UBUNTU: SAUCE: overlayfs: use shiftfs hacks only with shiftfs as underlay

Submitted by Andrei Vagin on April 19, 2020, 8:59 a.m.

Details

Message ID 20200419085918.900954-1-avagin@gmail.com
State New
Series "UBUNTU: SAUCE: overlayfs: use shiftfs hacks only with shiftfs as underlay"
Headers show

Commit Message

Andrei Vagin April 19, 2020, 8:59 a.m.
The hack was introduced in ("UBUNTU: SAUCE: overlayfs: allow with
shiftfs as underlay") and it broke checkpoint/restore of docker
contains:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1857257

The following script can be used to trigger the issue:
  #!/bin/bash

  cat > test.py << EOF
  import sys

  f = open("/proc/self/maps")

  for l in f.readlines():
    if "python" not in l:
      continue
    print(l)
    s = l.split()
    start, end = s[0].split("-")
    fname = s[-1]
    print(start, end, fname)
    break
  else:
    sys.exit(1)

  test_file1 = open(fname)
  test_file2 = open("/proc/self/map_files/%s-%s" % (start, end))

  fdinfo1 = open("/proc/self/fdinfo/%d" % test_file1.fileno()).read()
  fdinfo2 = open("/proc/self/fdinfo/%d" % test_file2.fileno()).read()

  if fdinfo1 != fdinfo2:
    print("FAIL")
    print(test_file1)
    print(fdinfo1)
    print(test_file2)
    print(fdinfo2)
    sys.exit(1)
  print("PASS")
  EOF
  sudo docker run -it --privileged --rm -v `pwd`:/mnt python python /mnt/test.py

Cc: Christian Brauner <christian.brauner@ubuntu.com>
Cc: Stefan Bader <stefan.bader@canonical.com>
Cc: Connor Kuehl <connor.kuehl@canonical.com>
Cc: Kleber Sacilotto de Souza <kleber.souza@canonical.com>
Fixes: 58009298c6bd ("UBUNTU: SAUCE: overlayfs: allow with shiftfs as underlay")
Signed-off-by: Andrei Vagin <avagin@gmail.com>
---
 fs/overlayfs/file.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

Patch hide | download patch | download mbox

diff --git a/fs/overlayfs/file.c b/fs/overlayfs/file.c
index 7b86d6080c22..c57cfe50f6cc 100644
--- a/fs/overlayfs/file.c
+++ b/fs/overlayfs/file.c
@@ -12,6 +12,7 @@ 
 #include <linux/xattr.h>
 #include <linux/uio.h>
 #include <linux/uaccess.h>
+#include <linux/magic.h>
 #include "overlayfs.h"
 
 static char ovl_whatisit(struct inode *inode, struct inode *realinode)
@@ -35,8 +36,12 @@  static struct file *ovl_open_realfile(const struct file *file,
 
 	old_cred = ovl_override_creds(inode->i_sb);
 	ovl_path_real(file->f_path.dentry, &realpath);
-	realfile = open_with_fake_path(&realpath, flags, realinode,
-				       current_cred());
+	if (realpath.dentry->d_sb->s_magic == SHIFTFS_MAGIC)
+		realfile = open_with_fake_path(&realpath, flags, realinode,
+					       current_cred());
+	else
+		realfile = open_with_fake_path(&file->f_path, flags, realinode,
+					       current_cred());
 	revert_creds(old_cred);
 
 	pr_debug("open(%p[%pD2/%c], 0%o) -> (%p, 0%o)\n",

Comments

Adrian Reber April 19, 2020, 10:54 a.m.
Andrei, thanks for the fix. For CRIU this would be a really important
fix to have. Almost every week we have some CRIU user asking us why
checkpoint/restore no longer works in Travis with Docker and Ubuntu in
general and we always point our users to that bug and are telling users
to downgrade the kernel or switch to another graph driver for Docker.

If this fix would be applied that would help us really a lot.

		Adrian

On Sun, Apr 19, 2020 at 01:59:18AM -0700, Andrei Vagin wrote:
> The hack was introduced in ("UBUNTU: SAUCE: overlayfs: allow with
> shiftfs as underlay") and it broke checkpoint/restore of docker
> contains:
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1857257
> 
> The following script can be used to trigger the issue:
>   #!/bin/bash
> 
>   cat > test.py << EOF
>   import sys
> 
>   f = open("/proc/self/maps")
> 
>   for l in f.readlines():
>     if "python" not in l:
>       continue
>     print(l)
>     s = l.split()
>     start, end = s[0].split("-")
>     fname = s[-1]
>     print(start, end, fname)
>     break
>   else:
>     sys.exit(1)
> 
>   test_file1 = open(fname)
>   test_file2 = open("/proc/self/map_files/%s-%s" % (start, end))
> 
>   fdinfo1 = open("/proc/self/fdinfo/%d" % test_file1.fileno()).read()
>   fdinfo2 = open("/proc/self/fdinfo/%d" % test_file2.fileno()).read()
> 
>   if fdinfo1 != fdinfo2:
>     print("FAIL")
>     print(test_file1)
>     print(fdinfo1)
>     print(test_file2)
>     print(fdinfo2)
>     sys.exit(1)
>   print("PASS")
>   EOF
>   sudo docker run -it --privileged --rm -v `pwd`:/mnt python python /mnt/test.py
> 
> Cc: Christian Brauner <christian.brauner@ubuntu.com>
> Cc: Stefan Bader <stefan.bader@canonical.com>
> Cc: Connor Kuehl <connor.kuehl@canonical.com>
> Cc: Kleber Sacilotto de Souza <kleber.souza@canonical.com>
> Fixes: 58009298c6bd ("UBUNTU: SAUCE: overlayfs: allow with shiftfs as underlay")
> Signed-off-by: Andrei Vagin <avagin@gmail.com>
> ---
>  fs/overlayfs/file.c | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/fs/overlayfs/file.c b/fs/overlayfs/file.c
> index 7b86d6080c22..c57cfe50f6cc 100644
> --- a/fs/overlayfs/file.c
> +++ b/fs/overlayfs/file.c
> @@ -12,6 +12,7 @@
>  #include <linux/xattr.h>
>  #include <linux/uio.h>
>  #include <linux/uaccess.h>
> +#include <linux/magic.h>
>  #include "overlayfs.h"
>  
>  static char ovl_whatisit(struct inode *inode, struct inode *realinode)
> @@ -35,8 +36,12 @@ static struct file *ovl_open_realfile(const struct file *file,
>  
>  	old_cred = ovl_override_creds(inode->i_sb);
>  	ovl_path_real(file->f_path.dentry, &realpath);
> -	realfile = open_with_fake_path(&realpath, flags, realinode,
> -				       current_cred());
> +	if (realpath.dentry->d_sb->s_magic == SHIFTFS_MAGIC)
> +		realfile = open_with_fake_path(&realpath, flags, realinode,
> +					       current_cred());
> +	else
> +		realfile = open_with_fake_path(&file->f_path, flags, realinode,
> +					       current_cred());
>  	revert_creds(old_cred);
>  
>  	pr_debug("open(%p[%pD2/%c], 0%o) -> (%p, 0%o)\n",
> -- 
> 2.17.1
> 
> _______________________________________________
> CRIU mailing list
> CRIU@openvz.org
> https://lists.openvz.org/mailman/listinfo/criu
Christian Brauner April 19, 2020, 11:18 a.m.
On Sun, Apr 19, 2020 at 01:59:18AM -0700, Andrei Vagin wrote:
> The hack was introduced in ("UBUNTU: SAUCE: overlayfs: allow with
> shiftfs as underlay") and it broke checkpoint/restore of docker
> contains:
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1857257
> 
> The following script can be used to trigger the issue:
>   #!/bin/bash
> 
>   cat > test.py << EOF
>   import sys
> 
>   f = open("/proc/self/maps")
> 
>   for l in f.readlines():
>     if "python" not in l:
>       continue
>     print(l)
>     s = l.split()
>     start, end = s[0].split("-")
>     fname = s[-1]
>     print(start, end, fname)
>     break
>   else:
>     sys.exit(1)
> 
>   test_file1 = open(fname)
>   test_file2 = open("/proc/self/map_files/%s-%s" % (start, end))
> 
>   fdinfo1 = open("/proc/self/fdinfo/%d" % test_file1.fileno()).read()
>   fdinfo2 = open("/proc/self/fdinfo/%d" % test_file2.fileno()).read()
> 
>   if fdinfo1 != fdinfo2:
>     print("FAIL")
>     print(test_file1)
>     print(fdinfo1)
>     print(test_file2)
>     print(fdinfo2)
>     sys.exit(1)
>   print("PASS")
>   EOF
>   sudo docker run -it --privileged --rm -v `pwd`:/mnt python python /mnt/test.py
> 
> Cc: Christian Brauner <christian.brauner@ubuntu.com>
> Cc: Stefan Bader <stefan.bader@canonical.com>
> Cc: Connor Kuehl <connor.kuehl@canonical.com>
> Cc: Kleber Sacilotto de Souza <kleber.souza@canonical.com>
> Fixes: 58009298c6bd ("UBUNTU: SAUCE: overlayfs: allow with shiftfs as underlay")
> Signed-off-by: Andrei Vagin <avagin@gmail.com>

Thanks for looking into this. Seems ok to me.

Acked-by: Christian Brauner <christian.brauner@ubuntu.com>
Andrei Vagin April 23, 2020, 12:02 a.m.
On Sun, Apr 19, 2020 at 4:19 AM Christian Brauner
<christian.brauner@ubuntu.com> wrote:
>
> On Sun, Apr 19, 2020 at 01:59:18AM -0700, Andrei Vagin wrote:
> > The hack was introduced in ("UBUNTU: SAUCE: overlayfs: allow with
> > shiftfs as underlay") and it broke checkpoint/restore of docker
> > contains:
> > https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1857257
> >
> > The following script can be used to trigger the issue:
...
> > Cc: Christian Brauner <christian.brauner@ubuntu.com>
> > Cc: Stefan Bader <stefan.bader@canonical.com>
> > Cc: Connor Kuehl <connor.kuehl@canonical.com>
> > Cc: Kleber Sacilotto de Souza <kleber.souza@canonical.com>
> > Fixes: 58009298c6bd ("UBUNTU: SAUCE: overlayfs: allow with shiftfs as underlay")
> > Signed-off-by: Andrei Vagin <avagin@gmail.com>
>
> Thanks for looking into this. Seems ok to me.
>
> Acked-by: Christian Brauner <christian.brauner@ubuntu.com>

Hi Christian,

Do you know why this patch isn't in the mailing list archive?
Do you know when we can expect to see this patch in the ubuntu kernel?

Thanks!
Andrei