[0/7] Add architecture support for s390x

Submitted by Michael Holzheu on June 29, 2017, 2:23 p.m.

Details

Message ID 20170629162316.12b61b29@TP-holzheu
State New
Headers show

Patch hide | download patch | download mbox

=== Run 1/1 ================ zdtm/static/scm00

========================== Run zdtm/static/scm00 in h ==========================
Start test
./scm00 --pidfile=scm00.pid --outfile=scm00.out
Run criu dump
=[log]=> dump/zdtm/static/scm00/26/1/dump.log
------------------------ grep Error ------------------------
(00.002506) 26 fdinfo 5: pos:                0 flags:                2/0
(00.002509)     Searching for socket c786 (family 1.0)
(00.002510)     Searching for socket c785 (family 1.0)
(00.002515) No filter for socket
(00.002523) Error (criu/sk-queue.c:78): Control messages in queue, not supported
(00.002531) ----------------------------------------
(00.002539) Error (criu/cr-dump.c:1427): Dump files (pid: 26) failed with -1
(00.002897) ptrace_set_regs: pid=26
(00.002951) Unlock network
(00.002955) Unfreezing tasks into 1
(00.002957)     Unseizing 26 into 1
(00.002969) Error (criu/cr-dump.c:1797): Dumping FAILED.
------------------------ ERROR OVER ------------------------
Send the 15 signal to  26
Wait for zdtm/static/scm00(26) to die for 0.100000
Removing dump/zdtm/static/scm00/26
========================= Test zdtm/static/scm00 PASS ==========================

> 
> >  - zdtm/static/sock_peercred
> >  - zdtm/static/socket_snd_addr
> 
> And this one is marked as 'noauto' which means, that the test is a placeholder
> for the functionality we want to support, so it does fail now.

Ok
 
> How did you run the tests?

We start the tests with "./zdtm.py run -a -x  $exclude_list" because
"make test" stops for the first failing test. Is there a better way to
do it?

Michael
---
Subject: [PATCH] Try to enable travis for s390x

Signed-off-by: Michael Holzheu <holzheu@linux.vnet.ibm.com>
---
 .travis.yml                        | 2 ++
 scripts/build/Dockerfile.s390x.hdr | 4 ++++
 scripts/build/Makefile             | 5 +++--
 scripts/build/binfmt_misc          | 4 ++++
 4 files changed, 13 insertions(+), 2 deletions(-)
 create mode 100644 scripts/build/Dockerfile.s390x.hdr

diff --git a/.travis.yml b/.travis.yml
index 70d666d..69aa141 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -11,10 +11,12 @@  env:
   - TR_ARCH=armv7hf
   - TR_ARCH=aarch64
   - TR_ARCH=ppc64le
+  - TR_ARCH=s390x
   - TR_ARCH=armv7hf     CLANG=1
   - TR_ARCH=aarch64     CLANG=1
   - TR_ARCH=ppc64le     CLANG=1
   - TR_ARCH=alpine      CLANG=1
+  - TR_ARCH=s390x       CLANG=1
 script:
   - sudo make -C scripts/travis $TR_ARCH
 after_success:
diff --git a/scripts/build/Dockerfile.s390x.hdr b/scripts/build/Dockerfile.s390x.hdr
new file mode 100644
index 0000000..ae41239
--- /dev/null
+++ b/scripts/build/Dockerfile.s390x.hdr
@@ -0,0 +1,4 @@ 
+FROM s390x/ubuntu:xenial
+
+ENV QEMU_CPU z990.4
+COPY scripts/build/qemu-user-static/usr/bin/qemu-s390x-static /usr/bin/qemu-s390x-static
diff --git a/scripts/build/Makefile b/scripts/build/Makefile
index dc00bfb..fef8c68 100644
--- a/scripts/build/Makefile
+++ b/scripts/build/Makefile
@@ -1,4 +1,4 @@ 
-ARCHES ?= armv7hf aarch64 ppc64le # x86_64
+ARCHES ?= armv7hf aarch64 ppc64le s390x # x86_64
 all: $(ARCHES)
 
 Dockerfile.%: Dockerfile.%.hdr Dockerfile.tmpl
@@ -25,7 +25,8 @@  aarch64-clang: DB_ARGS=--build-arg CC=clang-3.6
 aarch64-clang: aarch64
 ppc64le-clang: ppc64le
 x86_64-clang: x86_64
-.PHONY: alpine-clang armv7hf-clang aarch64-clang ppc64le-clang x86_64-clang
+s390x-clang: s390x
+.PHONY: alpine-clang armv7hf-clang aarch64-clang ppc64le-clang x86_64-clang s390x-clang
 
 clean:
 	rm -rf qemu-user-static
diff --git a/scripts/build/binfmt_misc b/scripts/build/binfmt_misc
index a5c922e..ab4f035 100755
--- a/scripts/build/binfmt_misc
+++ b/scripts/build/binfmt_misc
@@ -8,3 +8,7 @@  test -f /proc/sys/fs/binfmt_misc/aarch64 ||
 
 test -f /proc/sys/fs/binfmt_misc/ppc64le ||
 	echo ':ppc64le:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x15\x00:\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\x00:/usr/bin/qemu-ppc64le-static:' > /proc/sys/fs/binfmt_misc/register
+
+test -f /proc/sys/fs/binfmt_misc/s390x ||
+	echo ':s390x:M::\x7fELF\x02\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x16:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff:/usr/bin/qemu-s390x-static:' > /proc/sys/fs/binfmt_misc/register
+

Comments

Pavel Emelyanov June 29, 2017, 2:33 p.m.
On 06/29/2017 05:23 PM, Michael Holzheu wrote:
> Am Thu, 29 Jun 2017 12:48:28 +0300
> schrieb Pavel Emelyanov <xemul@virtuozzo.com>:
> 
>> On 06/28/2017 07:11 PM, Michael Holzheu wrote:
>>> Hello CRIU mailing list,
>>>
>>> This set of patches adds the s390x (64 bit mainframe) backend support to
>>> the CRIU checkpoint/restore tool.
>>
>> That's really cool :) thanks a lot!
>>
>> There's one, quite big, thing I'd like to discuss :) So far the most active community
>> members are x86 guys, but we already have 3 more arches (arm, aarch64 and ppc64le). 
>> In order not to break non-x86 heavily we've set up travis-ci jobs to constantly check
>> that criu compiles on non-x64 [1]. Also we have arm box and access to ppc64 VM from 
>> IBM and a set of jenkins jobs [2] that run tests on them.
>>
>> Would you also at least add the former piece, so that we don't break compilation of your
>> code. And ideally, we'd like to have the latter thing too.
> 
> Yesterday we tried to setup the travis qemu stuff, but for some reason
> this failed. I attached our current patch to this E-mail.

OK. So we'll at least have the compilation checker :)

> Regarding [2] I will ask if we can provide something for s390.

That's awesome!

>>> The patches apply to the "criu-dev" branch on top of commit eee68d7a0
>>> ("aarch/vdso: include common/compiler.h before use __maybe_unused).
>>>
>>> On s390 all tests of the zdtm testsuite succeed on Ubuntu 16.04 with kernel
>>> 4.8.0-34-generic execpt for the following:
>>>
>>>  - zdtm/static/del_standalone_un
>>>  - zdtm/static/deleted_unix_sock
>>>  - zdtm/static/mnt_ext_dev
>>>  - zdtm/static/overmount_dev
>>>  - zdtm/static/overmount_fifo
>>>  - zdtm/static/overmount_file
>>>  - zdtm/static/overmount_sock
>>>  - zdtm/static/pthread02
>>>  - zdtm/static/ptrace_sig
>>>  - zdtm/static/scm00
>>
>> Erm ... this test is marked with crfail in its .desc file, which means, that
>> dump MUST fail :)
> 
> Good to know, from the output below we just thought something is wrong :-)
> 
> ~/criu/test (ibm_criu-dev)# ./zdtm.py run -t zdtm/static/scm00
> === Run 1/1 ================ zdtm/static/scm00
> 
> ========================== Run zdtm/static/scm00 in h ==========================
> Start test
> ./scm00 --pidfile=scm00.pid --outfile=scm00.out
> Run criu dump
> =[log]=> dump/zdtm/static/scm00/26/1/dump.log
> ------------------------ grep Error ------------------------
> (00.002506) 26 fdinfo 5: pos:                0 flags:                2/0
> (00.002509)     Searching for socket c786 (family 1.0)
> (00.002510)     Searching for socket c785 (family 1.0)
> (00.002515) No filter for socket
> (00.002523) Error (criu/sk-queue.c:78): Control messages in queue, not supported
> (00.002531) ----------------------------------------
> (00.002539) Error (criu/cr-dump.c:1427): Dump files (pid: 26) failed with -1
> (00.002897) ptrace_set_regs: pid=26
> (00.002951) Unlock network
> (00.002955) Unfreezing tasks into 1
> (00.002957)     Unseizing 26 into 1
> (00.002969) Error (criu/cr-dump.c:1797): Dumping FAILED.
> ------------------------ ERROR OVER ------------------------
> Send the 15 signal to  26
> Wait for zdtm/static/scm00(26) to die for 0.100000
> Removing dump/zdtm/static/scm00/26
> ========================= Test zdtm/static/scm00 PASS ==========================

This very line says that something indeed went wrong and that's what was expected :D

>>
>>>  - zdtm/static/sock_peercred
>>>  - zdtm/static/socket_snd_addr
>>
>> And this one is marked as 'noauto' which means, that the test is a placeholder
>> for the functionality we want to support, so it does fail now.
> 
> Ok
>  
>> How did you run the tests?
> 
> We start the tests with "./zdtm.py run -a -x  $exclude_list" because
> "make test" stops for the first failing test. Is there a better way to
> do it?

No, that's the correct command :) You should always look at the very last
line of each test, it says whether the test passed or not. Also, you may
want to use the --keep-going option, so that all tests are run and then
at the very end a summary is printed.

-- Pavel
Michael Holzheu June 30, 2017, 2:41 p.m.
Am Thu, 29 Jun 2017 17:33:51 +0300
schrieb Pavel Emelyanov <xemul@virtuozzo.com>:

> On 06/29/2017 05:23 PM, Michael Holzheu wrote:
> > Am Thu, 29 Jun 2017 12:48:28 +0300
> > schrieb Pavel Emelyanov <xemul@virtuozzo.com>:
> > 
> >> On 06/28/2017 07:11 PM, Michael Holzheu wrote:
> >>> Hello CRIU mailing list,
> >>>
> >>> This set of patches adds the s390x (64 bit mainframe) backend support to
> >>> the CRIU checkpoint/restore tool.
> >>
> >> That's really cool :) thanks a lot!
> >>
> >> There's one, quite big, thing I'd like to discuss :) So far the most active community
> >> members are x86 guys, but we already have 3 more arches (arm, aarch64 and ppc64le). 
> >> In order not to break non-x86 heavily we've set up travis-ci jobs to constantly check
> >> that criu compiles on non-x64 [1]. Also we have arm box and access to ppc64 VM from 
> >> IBM and a set of jenkins jobs [2] that run tests on them.
> >>
> >> Would you also at least add the former piece, so that we don't break compilation of your
> >> code. And ideally, we'd like to have the latter thing too.
> > 
> > Yesterday we tried to setup the travis qemu stuff, but for some reason
> > this failed. I attached our current patch to this E-mail.
> 
> OK. So we'll at least have the compilation checker :)

I just learned that qemu on s390x can only emulate older z900 mainframe machines.
Very likely the Travis Ubuntu Docker image for s390 uses newer instructions,
which is probably the reason why it currently does not work.

Michael