Error (pie-util-vdso.c:190): vdso: Not all dynamic entries are present

Submitted by Dmitry Safonov on June 7, 2016, 12:24 p.m.

Details

Message ID f30205ef-103f-3263-610c-de9a785879bc@virtuozzo.com
State Rejected
Series "Error (pie-util-vdso.c:190): vdso: Not all dynamic entries are present"
Headers show

Commit Message

Dmitry Safonov June 7, 2016, 12:24 p.m.
On 06/07/2016 01:29 AM, Tycho Andersen wrote:
> Hi guys,
>
> With kernel 4.6 (and maybe earlier, although not with 4.4), I'm
> getting the error in the subject line. The full restore log is,
>
> (00.000044) File tty[8800:e] will be restored from inherit fd 5
> (00.004389) Pagemap is fully functional
> (00.004422) Found task size of 7ffffffff000
> (00.004489) cpu: fpu:1 fxsr:1 xsave:0
> (00.004590) vdso: Parsing at 7ffcc11ca000 7ffcc11cb000
> (00.004597) vdso: PT_LOAD p_vaddr: 0
> (00.004601) Error (pie-util-vdso.c:190): vdso: Not all dynamic entries are present
>
> The symptom is that the phdr we get for PT_DYNAMIC has p_filesz == 0,
> so the loop in parse_elf_dynamic() never executes.
>
> Any ideas?
>

Hi Tycho!

Could you try with the following diff?

--->8---
  			goto err_oob;

Patch hide | download patch | download mbox

diff --git a/criu/pie/util-vdso.c b/criu/pie/util-vdso.c
index 13b2e5316db7..2033cdd82f90 100644
--- a/criu/pie/util-vdso.c
+++ b/criu/pie/util-vdso.c
@@ -160,7 +160,10 @@  static int parse_elf_dynamic(uintptr_t mem, size_t 
size, Phdr_t *dynamic,
  	if (__ptr_oob(addr, mem, size))
  		goto err_oob;

-	for (i = 0; i < dynamic->p_filesz / sizeof(*d);
+	pr_debug("Dynamic segment's file size %lu, memory size %lu\n",
+			(unsigned long)dynamic->p_filesz,
+			(unsigned long)dynamic->p_memsz);
+	for (i = 0; i < dynamic->p_memsz / sizeof(*d);
  			i++, addr += sizeof(Dyn_t)) {
  		if (__ptr_struct_end_oob(addr, sizeof(Dyn_t), mem, size))

Comments

Dmitry Safonov June 7, 2016, 12:40 p.m.
On 06/07/2016 03:24 PM, Dmitry Safonov wrote:
> On 06/07/2016 01:29 AM, Tycho Andersen wrote:
>> Hi guys,
>>
>> With kernel 4.6 (and maybe earlier, although not with 4.4), I'm
>> getting the error in the subject line. The full restore log is,
>>
>> (00.000044) File tty[8800:e] will be restored from inherit fd 5
>> (00.004389) Pagemap is fully functional
>> (00.004422) Found task size of 7ffffffff000
>> (00.004489) cpu: fpu:1 fxsr:1 xsave:0
>> (00.004590) vdso: Parsing at 7ffcc11ca000 7ffcc11cb000
>> (00.004597) vdso: PT_LOAD p_vaddr: 0
>> (00.004601) Error (pie-util-vdso.c:190): vdso: Not all dynamic entries
>> are present
>>
>> The symptom is that the phdr we get for PT_DYNAMIC has p_filesz == 0,
>> so the loop in parse_elf_dynamic() never executes.
>>
>> Any ideas?
>>
>
> Hi Tycho!
>
> Could you try with the following diff?

Oh, I guess, it will not work, according to docs:

If p_memsz is greater than p_filesz, the memory range from
(p_vaddr + p_filesz) to (p_vaddr + p_memsz) is zero-filled
when the segment is loaded.

Well, let me think.
Dmitry Safonov June 7, 2016, 12:53 p.m.
On 06/07/2016 03:40 PM, Dmitry Safonov wrote:
> On 06/07/2016 03:24 PM, Dmitry Safonov wrote:
>> On 06/07/2016 01:29 AM, Tycho Andersen wrote:
>>> Hi guys,
>>>
>>> With kernel 4.6 (and maybe earlier, although not with 4.4), I'm
>>> getting the error in the subject line. The full restore log is,
>>>
>>> (00.000044) File tty[8800:e] will be restored from inherit fd 5
>>> (00.004389) Pagemap is fully functional
>>> (00.004422) Found task size of 7ffffffff000
>>> (00.004489) cpu: fpu:1 fxsr:1 xsave:0
>>> (00.004590) vdso: Parsing at 7ffcc11ca000 7ffcc11cb000
>>> (00.004597) vdso: PT_LOAD p_vaddr: 0
>>> (00.004601) Error (pie-util-vdso.c:190): vdso: Not all dynamic entries
>>> are present
>>>
>>> The symptom is that the phdr we get for PT_DYNAMIC has p_filesz == 0,
>>> so the loop in parse_elf_dynamic() never executes.
>>>
>>> Any ideas?
>>>
>>
>> Hi Tycho!
>>
>> Could you try with the following diff?
>
> Oh, I guess, it will not work, according to docs:
>
> If p_memsz is greater than p_filesz, the memory range from
> (p_vaddr + p_filesz) to (p_vaddr + p_memsz) is zero-filled
> when the segment is loaded.
>
> Well, let me think.
>

Oh, it would be nice, if you gave it a shot anyway -
this zero-padding is valid for loadable segments, not
dynamic.

And could you gave `uname -a` - so I could reproduce it in VM maybe?
Tycho Andersen June 7, 2016, 2:43 p.m.
Hi Dmitry,

On Tue, Jun 07, 2016 at 03:53:47PM +0300, Dmitry Safonov wrote:
> On 06/07/2016 03:40 PM, Dmitry Safonov wrote:
> >On 06/07/2016 03:24 PM, Dmitry Safonov wrote:
> >>On 06/07/2016 01:29 AM, Tycho Andersen wrote:
> >>>Hi guys,
> >>>
> >>>With kernel 4.6 (and maybe earlier, although not with 4.4), I'm
> >>>getting the error in the subject line. The full restore log is,
> >>>
> >>>(00.000044) File tty[8800:e] will be restored from inherit fd 5
> >>>(00.004389) Pagemap is fully functional
> >>>(00.004422) Found task size of 7ffffffff000
> >>>(00.004489) cpu: fpu:1 fxsr:1 xsave:0
> >>>(00.004590) vdso: Parsing at 7ffcc11ca000 7ffcc11cb000
> >>>(00.004597) vdso: PT_LOAD p_vaddr: 0
> >>>(00.004601) Error (pie-util-vdso.c:190): vdso: Not all dynamic entries
> >>>are present
> >>>
> >>>The symptom is that the phdr we get for PT_DYNAMIC has p_filesz == 0,
> >>>so the loop in parse_elf_dynamic() never executes.
> >>>
> >>>Any ideas?
> >>>
> >>
> >>Hi Tycho!
> >>
> >>Could you try with the following diff?
> >
> >Oh, I guess, it will not work, according to docs:
> >
> >If p_memsz is greater than p_filesz, the memory range from
> >(p_vaddr + p_filesz) to (p_vaddr + p_memsz) is zero-filled
> >when the segment is loaded.
> >
> >Well, let me think.
> >
> 
> Oh, it would be nice, if you gave it a shot anyway -
> this zero-padding is valid for loadable segments, not
> dynamic.

Sure,

(00.000127) Probing sock diag modules
(00.000198) Done probing
(00.003191) ========================================
(00.003233) Dumping processes (pid: 6554)
(00.003243) ========================================
(00.003276) Running pre-dump scripts
(00.003351) Pagemap is fully functional
(00.003392) Found anon-shmem device at 5
(00.003410) Reset 14739's dirty tracking
(00.003447)  ... done
(00.003480) Dirty track supported on kernel
(00.003538) Found task size of 7ffffffff000
(00.003579) irmap: Searching irmap cache in work dir
(00.003609) No irmap-cache image
(00.003627) irmap: Searching irmap cache in parent
(00.003640) irmap: No irmap cache
(00.003657) cpu: fpu:1 fxsr:1 xsave:0
(00.003730) vdso: Parsing at 7fff6cdc7000 7fff6cdc8000
(00.003742) vdso: PT_LOAD p_vaddr: 0
(00.003750) vdso: Dynamic segment's file size 0, memory size 0
(00.003757) Error (pie-util-vdso.c:194): vdso: Not all dynamic entries
are present
(00.003770) Unlock network
(00.003783) Error (cr-dump.c:1600): Dumping FAILED.

> And could you gave `uname -a` - so I could reproduce it in VM maybe?

Sure, it's:

Linux dev 4.6.0-8-generic #9 SMP Mon Jun 6 14:21:30 MDT 2016 x86_64 x86_64 x86_64 GNU/Linux

which is really 66b7a6c3fa35ea617c60d938bd2028eb6141c2f0 from
git://kernel.ubuntu.com/ubuntu/unstable.git

I put the debs I built here:

http://files.tycho.ws/tmp/vdso/

if you want to just download those. Let me know if you want me to try
something else.

Tycho
Dmitry Safonov June 7, 2016, 3:13 p.m.
On 06/07/2016 05:43 PM, Tycho Andersen wrote:
> Hi Dmitry,
>
> On Tue, Jun 07, 2016 at 03:53:47PM +0300, Dmitry Safonov wrote:
>> On 06/07/2016 03:40 PM, Dmitry Safonov wrote:
>>> On 06/07/2016 03:24 PM, Dmitry Safonov wrote:
>>>> On 06/07/2016 01:29 AM, Tycho Andersen wrote:
>>>>> Hi guys,
>>>>>
>>>>> With kernel 4.6 (and maybe earlier, although not with 4.4), I'm
>>>>> getting the error in the subject line. The full restore log is,
>>>>>
>>>>> (00.000044) File tty[8800:e] will be restored from inherit fd 5
>>>>> (00.004389) Pagemap is fully functional
>>>>> (00.004422) Found task size of 7ffffffff000
>>>>> (00.004489) cpu: fpu:1 fxsr:1 xsave:0
>>>>> (00.004590) vdso: Parsing at 7ffcc11ca000 7ffcc11cb000
>>>>> (00.004597) vdso: PT_LOAD p_vaddr: 0
>>>>> (00.004601) Error (pie-util-vdso.c:190): vdso: Not all dynamic entries
>>>>> are present
>>>>>
>>>>> The symptom is that the phdr we get for PT_DYNAMIC has p_filesz == 0,
>>>>> so the loop in parse_elf_dynamic() never executes.
>>>>>
>>>>> Any ideas?
>>>>>
>>>>
>>>> Hi Tycho!
>>>>
>>>> Could you try with the following diff?
>>>
>>> Oh, I guess, it will not work, according to docs:
>>>
>>> If p_memsz is greater than p_filesz, the memory range from
>>> (p_vaddr + p_filesz) to (p_vaddr + p_memsz) is zero-filled
>>> when the segment is loaded.
>>>
>>> Well, let me think.
>>>
>>
>> Oh, it would be nice, if you gave it a shot anyway -
>> this zero-padding is valid for loadable segments, not
>> dynamic.
>
> Sure,
>
> (00.000127) Probing sock diag modules
> (00.000198) Done probing
> (00.003191) ========================================
> (00.003233) Dumping processes (pid: 6554)
> (00.003243) ========================================
> (00.003276) Running pre-dump scripts
> (00.003351) Pagemap is fully functional
> (00.003392) Found anon-shmem device at 5
> (00.003410) Reset 14739's dirty tracking
> (00.003447)  ... done
> (00.003480) Dirty track supported on kernel
> (00.003538) Found task size of 7ffffffff000
> (00.003579) irmap: Searching irmap cache in work dir
> (00.003609) No irmap-cache image
> (00.003627) irmap: Searching irmap cache in parent
> (00.003640) irmap: No irmap cache
> (00.003657) cpu: fpu:1 fxsr:1 xsave:0
> (00.003730) vdso: Parsing at 7fff6cdc7000 7fff6cdc8000
> (00.003742) vdso: PT_LOAD p_vaddr: 0
> (00.003750) vdso: Dynamic segment's file size 0, memory size 0
> (00.003757) Error (pie-util-vdso.c:194): vdso: Not all dynamic entries
> are present
> (00.003770) Unlock network
> (00.003783) Error (cr-dump.c:1600): Dumping FAILED.

Thanks.

>> And could you gave `uname -a` - so I could reproduce it in VM maybe?
>
> Sure, it's:
>
> Linux dev 4.6.0-8-generic #9 SMP Mon Jun 6 14:21:30 MDT 2016 x86_64 x86_64 x86_64 GNU/Linux
>
> which is really 66b7a6c3fa35ea617c60d938bd2028eb6141c2f0 from
> git://kernel.ubuntu.com/ubuntu/unstable.git
>
> I put the debs I built here:
>
> http://files.tycho.ws/tmp/vdso/
>
> if you want to just download those. Let me know if you want me to try
> something else.

Ok, I fetched your kernel.
Here is it:
[ubuntu-unstable]$ git checkout 66b7a6c3fa35ea617c60d938bd2028eb6141c2f0
[ubuntu-unstable]$ make clean
...
[ubuntu-unstable]$ make -j5 arch/x86/entry/vdso/
...
[ubuntu-unstable]$ readelf --program-headers arch/x86/entry/vdso/vdso64.so

Elf file type is EXEC (Executable file)
Entry point 0x5c0
There are 4 program headers, starting at offset 64

Program Headers:
   Type           Offset             VirtAddr           PhysAddr
                  FileSiz            MemSiz              Flags  Align
   LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
                  0x0000000000000979 0x0000000000000979  R E    1000
   DYNAMIC        0x0000000000000000 0x0000000000000000 0x0000000000000000
                  0x0000000000000000 0x0000000000000000  R      8
readelf: Error: no .dynamic section in the dynamic segment
   NOTE           0x0000000000000460 0x0000000000000460 0x0000000000000460
                  0x000000000000003c 0x000000000000003c  R      4
   GNU_EH_FRAME   0x000000000000049c 0x000000000000049c 0x000000000000049c
                  0x0000000000000034 0x0000000000000034  R      4

  Section to Segment mapping:
   Segment Sections...
    00     .rodata .note .eh_frame_hdr .eh_frame .text .altinstructions 
.altinstr_replacement
    01
    02     .note
    03     .eh_frame_hdr


See? There is no .dynamic section :)

Ok, lets find, what broke it...
[ubuntu-unstable]$ git checkout f8830b2956a3~1
[ubuntu-unstable]$ make clean
...
[ubuntu-unstable]$ make -j5 arch/x86/entry/vdso/
...
[ubuntu-unstable]$ readelf --program-headers arch/x86/entry/vdso/vdso64.so

Elf file type is DYN (Shared object file)
Entry point 0x910
There are 4 program headers, starting at offset 64

Program Headers:
   Type           Offset             VirtAddr           PhysAddr
                  FileSiz            MemSiz              Flags  Align
   LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
                  0x0000000000000cb9 0x0000000000000cb9  R E    1000
   DYNAMIC        0x0000000000000360 0x0000000000000360 0x0000000000000360
                  0x0000000000000110 0x0000000000000110  R      8
   NOTE           0x00000000000007b0 0x00000000000007b0 0x00000000000007b0
                  0x000000000000003c 0x000000000000003c  R      4
   GNU_EH_FRAME   0x00000000000007ec 0x00000000000007ec 0x00000000000007ec
                  0x0000000000000034 0x0000000000000034  R      4

  Section to Segment mapping:
   Segment Sections...
    00     .hash .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_d 
.dynamic .rodata .note .eh_frame_hdr .eh_frame .text .altinstructions 
.altinstr_replacement
    01     .dynamic
    02     .note
    03     .eh_frame_hdr


And here it is. Not sure that this commit f8830b2956a3 ("disable -pie
when gcc has it enabled by default") has broke it, but it looks quite
suspicious.

Anyway, I'm not sure, maybe we could re-arrange vdso code in CRIU
that it wouldn't need .dynamic? I'm not sure, maybe I'll check it
when I'll have some more time :)
Dmitry Safonov June 7, 2016, 3:36 p.m.
On 06/07/2016 06:13 PM, Dmitry Safonov wrote:
> On 06/07/2016 05:43 PM, Tycho Andersen wrote:
>> Hi Dmitry,
>>
>> On Tue, Jun 07, 2016 at 03:53:47PM +0300, Dmitry Safonov wrote:
>>> On 06/07/2016 03:40 PM, Dmitry Safonov wrote:
>>>> On 06/07/2016 03:24 PM, Dmitry Safonov wrote:
>>>>> On 06/07/2016 01:29 AM, Tycho Andersen wrote:
>>>>>> Hi guys,
>>>>>>
>>>>>> With kernel 4.6 (and maybe earlier, although not with 4.4), I'm
>>>>>> getting the error in the subject line. The full restore log is,
>>>>>>
>>>>>> (00.000044) File tty[8800:e] will be restored from inherit fd 5
>>>>>> (00.004389) Pagemap is fully functional
>>>>>> (00.004422) Found task size of 7ffffffff000
>>>>>> (00.004489) cpu: fpu:1 fxsr:1 xsave:0
>>>>>> (00.004590) vdso: Parsing at 7ffcc11ca000 7ffcc11cb000
>>>>>> (00.004597) vdso: PT_LOAD p_vaddr: 0
>>>>>> (00.004601) Error (pie-util-vdso.c:190): vdso: Not all dynamic
>>>>>> entries
>>>>>> are present
>>>>>>
>>>>>> The symptom is that the phdr we get for PT_DYNAMIC has p_filesz == 0,
>>>>>> so the loop in parse_elf_dynamic() never executes.
>>>>>>
>>>>>> Any ideas?
>>>>>>
>>>>>
>>>>> Hi Tycho!
>>>>>
>>>>> Could you try with the following diff?
>>>>
>>>> Oh, I guess, it will not work, according to docs:
>>>>
>>>> If p_memsz is greater than p_filesz, the memory range from
>>>> (p_vaddr + p_filesz) to (p_vaddr + p_memsz) is zero-filled
>>>> when the segment is loaded.
>>>>
>>>> Well, let me think.
>>>>
>>>
>>> Oh, it would be nice, if you gave it a shot anyway -
>>> this zero-padding is valid for loadable segments, not
>>> dynamic.
>>
>> Sure,
>>
>> (00.000127) Probing sock diag modules
>> (00.000198) Done probing
>> (00.003191) ========================================
>> (00.003233) Dumping processes (pid: 6554)
>> (00.003243) ========================================
>> (00.003276) Running pre-dump scripts
>> (00.003351) Pagemap is fully functional
>> (00.003392) Found anon-shmem device at 5
>> (00.003410) Reset 14739's dirty tracking
>> (00.003447)  ... done
>> (00.003480) Dirty track supported on kernel
>> (00.003538) Found task size of 7ffffffff000
>> (00.003579) irmap: Searching irmap cache in work dir
>> (00.003609) No irmap-cache image
>> (00.003627) irmap: Searching irmap cache in parent
>> (00.003640) irmap: No irmap cache
>> (00.003657) cpu: fpu:1 fxsr:1 xsave:0
>> (00.003730) vdso: Parsing at 7fff6cdc7000 7fff6cdc8000
>> (00.003742) vdso: PT_LOAD p_vaddr: 0
>> (00.003750) vdso: Dynamic segment's file size 0, memory size 0
>> (00.003757) Error (pie-util-vdso.c:194): vdso: Not all dynamic entries
>> are present
>> (00.003770) Unlock network
>> (00.003783) Error (cr-dump.c:1600): Dumping FAILED.
>
> Thanks.
>
>>> And could you gave `uname -a` - so I could reproduce it in VM maybe?
>>
>> Sure, it's:
>>
>> Linux dev 4.6.0-8-generic #9 SMP Mon Jun 6 14:21:30 MDT 2016 x86_64
>> x86_64 x86_64 GNU/Linux
>>
>> which is really 66b7a6c3fa35ea617c60d938bd2028eb6141c2f0 from
>> git://kernel.ubuntu.com/ubuntu/unstable.git
>>
>> I put the debs I built here:
>>
>> http://files.tycho.ws/tmp/vdso/
>>
>> if you want to just download those. Let me know if you want me to try
>> something else.
>
> Ok, I fetched your kernel.
> Here is it:
> [ubuntu-unstable]$ git checkout 66b7a6c3fa35ea617c60d938bd2028eb6141c2f0
> [ubuntu-unstable]$ make clean
> ...
> [ubuntu-unstable]$ make -j5 arch/x86/entry/vdso/
> ...
> [ubuntu-unstable]$ readelf --program-headers arch/x86/entry/vdso/vdso64.so
>
> Elf file type is EXEC (Executable file)
> Entry point 0x5c0
> There are 4 program headers, starting at offset 64
>
> Program Headers:
>   Type           Offset             VirtAddr           PhysAddr
>                  FileSiz            MemSiz              Flags  Align
>   LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
>                  0x0000000000000979 0x0000000000000979  R E    1000
>   DYNAMIC        0x0000000000000000 0x0000000000000000 0x0000000000000000
>                  0x0000000000000000 0x0000000000000000  R      8
> readelf: Error: no .dynamic section in the dynamic segment
>   NOTE           0x0000000000000460 0x0000000000000460 0x0000000000000460
>                  0x000000000000003c 0x000000000000003c  R      4
>   GNU_EH_FRAME   0x000000000000049c 0x000000000000049c 0x000000000000049c
>                  0x0000000000000034 0x0000000000000034  R      4
>
>  Section to Segment mapping:
>   Segment Sections...
>    00     .rodata .note .eh_frame_hdr .eh_frame .text .altinstructions
> .altinstr_replacement
>    01
>    02     .note
>    03     .eh_frame_hdr
>
>
> See? There is no .dynamic section :)
>
> Ok, lets find, what broke it...
> [ubuntu-unstable]$ git checkout f8830b2956a3~1
> [ubuntu-unstable]$ make clean
> ...
> [ubuntu-unstable]$ make -j5 arch/x86/entry/vdso/
> ...
> [ubuntu-unstable]$ readelf --program-headers arch/x86/entry/vdso/vdso64.so
>
> Elf file type is DYN (Shared object file)
> Entry point 0x910
> There are 4 program headers, starting at offset 64
>
> Program Headers:
>   Type           Offset             VirtAddr           PhysAddr
>                  FileSiz            MemSiz              Flags  Align
>   LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
>                  0x0000000000000cb9 0x0000000000000cb9  R E    1000
>   DYNAMIC        0x0000000000000360 0x0000000000000360 0x0000000000000360
>                  0x0000000000000110 0x0000000000000110  R      8
>   NOTE           0x00000000000007b0 0x00000000000007b0 0x00000000000007b0
>                  0x000000000000003c 0x000000000000003c  R      4
>   GNU_EH_FRAME   0x00000000000007ec 0x00000000000007ec 0x00000000000007ec
>                  0x0000000000000034 0x0000000000000034  R      4
>
>  Section to Segment mapping:
>   Segment Sections...
>    00     .hash .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_d
> .dynamic .rodata .note .eh_frame_hdr .eh_frame .text .altinstructions
> .altinstr_replacement
>    01     .dynamic
>    02     .note
>    03     .eh_frame_hdr
>
>
> And here it is. Not sure that this commit f8830b2956a3 ("disable -pie
> when gcc has it enabled by default") has broke it, but it looks quite
> suspicious.
>
> Anyway, I'm not sure, maybe we could re-arrange vdso code in CRIU
> that it wouldn't need .dynamic? I'm not sure, maybe I'll check it
> when I'll have some more time :)

But it's very unclear, what can we do if there is no .dynamic
We really need symbols and their location information.
So, vDSO looks quite useless without it. I'm surprised that
your glibc works with that (is it?). Maybe it fails back to raw
syscall instructions instead of fast syscalls - I don't know
if you have check that after that patch.
Tycho Andersen June 7, 2016, 4:25 p.m.
Hi Dmitry,

On Tue, Jun 07, 2016 at 06:36:49PM +0300, Dmitry Safonov wrote:
> >Ok, I fetched your kernel.
> >Here is it:
> >[ubuntu-unstable]$ git checkout 66b7a6c3fa35ea617c60d938bd2028eb6141c2f0
> >[ubuntu-unstable]$ make clean
> >...
> >[ubuntu-unstable]$ make -j5 arch/x86/entry/vdso/
> >...
> >[ubuntu-unstable]$ readelf --program-headers arch/x86/entry/vdso/vdso64.so
> >
> >Elf file type is EXEC (Executable file)
> >Entry point 0x5c0
> >There are 4 program headers, starting at offset 64
> >
> >Program Headers:
> >  Type           Offset             VirtAddr           PhysAddr
> >                 FileSiz            MemSiz              Flags  Align
> >  LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
> >                 0x0000000000000979 0x0000000000000979  R E    1000
> >  DYNAMIC        0x0000000000000000 0x0000000000000000 0x0000000000000000
> >                 0x0000000000000000 0x0000000000000000  R      8
> >readelf: Error: no .dynamic section in the dynamic segment
> >  NOTE           0x0000000000000460 0x0000000000000460 0x0000000000000460
> >                 0x000000000000003c 0x000000000000003c  R      4
> >  GNU_EH_FRAME   0x000000000000049c 0x000000000000049c 0x000000000000049c
> >                 0x0000000000000034 0x0000000000000034  R      4
> >
> > Section to Segment mapping:
> >  Segment Sections...
> >   00     .rodata .note .eh_frame_hdr .eh_frame .text .altinstructions
> >.altinstr_replacement
> >   01
> >   02     .note
> >   03     .eh_frame_hdr
> >
> >
> >See? There is no .dynamic section :)
> >
> >Ok, lets find, what broke it...
> >[ubuntu-unstable]$ git checkout f8830b2956a3~1
> >[ubuntu-unstable]$ make clean
> >...
> >[ubuntu-unstable]$ make -j5 arch/x86/entry/vdso/
> >...
> >[ubuntu-unstable]$ readelf --program-headers arch/x86/entry/vdso/vdso64.so
> >
> >Elf file type is DYN (Shared object file)
> >Entry point 0x910
> >There are 4 program headers, starting at offset 64
> >
> >Program Headers:
> >  Type           Offset             VirtAddr           PhysAddr
> >                 FileSiz            MemSiz              Flags  Align
> >  LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
> >                 0x0000000000000cb9 0x0000000000000cb9  R E    1000
> >  DYNAMIC        0x0000000000000360 0x0000000000000360 0x0000000000000360
> >                 0x0000000000000110 0x0000000000000110  R      8
> >  NOTE           0x00000000000007b0 0x00000000000007b0 0x00000000000007b0
> >                 0x000000000000003c 0x000000000000003c  R      4
> >  GNU_EH_FRAME   0x00000000000007ec 0x00000000000007ec 0x00000000000007ec
> >                 0x0000000000000034 0x0000000000000034  R      4
> >
> > Section to Segment mapping:
> >  Segment Sections...
> >   00     .hash .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_d
> >.dynamic .rodata .note .eh_frame_hdr .eh_frame .text .altinstructions
> >.altinstr_replacement
> >   01     .dynamic
> >   02     .note
> >   03     .eh_frame_hdr
> >
> >
> >And here it is. Not sure that this commit f8830b2956a3 ("disable -pie
> >when gcc has it enabled by default") has broke it, but it looks quite
> >suspicious.
> >
> >Anyway, I'm not sure, maybe we could re-arrange vdso code in CRIU
> >that it wouldn't need .dynamic? I'm not sure, maybe I'll check it
> >when I'll have some more time :)
> 
> But it's very unclear, what can we do if there is no .dynamic
> We really need symbols and their location information.
> So, vDSO looks quite useless without it. I'm surprised that
> your glibc works with that (is it?). Maybe it fails back to raw
> syscall instructions instead of fast syscalls - I don't know
> if you have check that after that patch.

Thanks for the analysis. I just tested, and sure enough it does fall
back to real syscalls, so that seems like a big problem :)

I'll check with our kernel guys and see, but based on looking at the
code we do need the .dynamic section so we can reposition things.

Tycho
Dmitry Safonov June 7, 2016, 4:38 p.m.
On 06/07/2016 07:25 PM, Tycho Andersen wrote:
> Hi Dmitry,
>
> On Tue, Jun 07, 2016 at 06:36:49PM +0300, Dmitry Safonov wrote:
>>
>> But it's very unclear, what can we do if there is no .dynamic
>> We really need symbols and their location information.
>> So, vDSO looks quite useless without it. I'm surprised that
>> your glibc works with that (is it?). Maybe it fails back to raw
>> syscall instructions instead of fast syscalls - I don't know
>> if you have check that after that patch.
>
> Thanks for the analysis. I just tested, and sure enough it does fall
> back to real syscalls, so that seems like a big problem :)
>
> I'll check with our kernel guys and see, but based on looking at the
> code we do need the .dynamic section so we can reposition things.

Well, of course we could handle it by just blindly map vDSO back on
restore - if we can't parse it. But that has no use for CRIU: there
isn't any such vDSO, as the whole idea of it erases without information
about syscalls it provides.
Tycho Andersen June 7, 2016, 4:56 p.m.
On Tue, Jun 07, 2016 at 07:38:34PM +0300, Dmitry Safonov wrote:
> On 06/07/2016 07:25 PM, Tycho Andersen wrote:
> >Hi Dmitry,
> >
> >On Tue, Jun 07, 2016 at 06:36:49PM +0300, Dmitry Safonov wrote:
> >>
> >>But it's very unclear, what can we do if there is no .dynamic
> >>We really need symbols and their location information.
> >>So, vDSO looks quite useless without it. I'm surprised that
> >>your glibc works with that (is it?). Maybe it fails back to raw
> >>syscall instructions instead of fast syscalls - I don't know
> >>if you have check that after that patch.
> >
> >Thanks for the analysis. I just tested, and sure enough it does fall
> >back to real syscalls, so that seems like a big problem :)
> >
> >I'll check with our kernel guys and see, but based on looking at the
> >code we do need the .dynamic section so we can reposition things.
> 
> Well, of course we could handle it by just blindly map vDSO back on
> restore - if we can't parse it. But that has no use for CRIU: there
> isn't any such vDSO, as the whole idea of it erases without information
> about syscalls it provides.

Yes, I think the real fix is to make sure vDSO is offered in our
kernels ;-). Thanks a bunch!

Tycho