test/zdtm: do not allow running with --lazy-pages on kernel without uffd

Submitted by Mike Rapoport on July 9, 2017, 6:12 a.m.

Details

Message ID 1499580757-11322-1-git-send-email-rppt@linux.vnet.ibm.com
State New
Series "test/zdtm: do not allow running with --lazy-pages on kernel without uffd"
Headers show

Commit Message

Mike Rapoport July 9, 2017, 6:12 a.m.
Raise an exception for kernels that do not have userfaultfd. For the
kernels that have userfaultfd but do not provide non-cooperative events
(4.3 - 4.10) warn about incomplete feature.

Fixes: #363

Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
---
 test/zdtm.py | 10 ++++++++++
 1 file changed, 10 insertions(+)

Patch hide | download patch | download mbox

diff --git a/test/zdtm.py b/test/zdtm.py
index 2ee76cd..e6d2701 100755
--- a/test/zdtm.py
+++ b/test/zdtm.py
@@ -1767,6 +1767,16 @@  def run_tests(opts):
 		if subprocess.Popen(["ip", "netns", "exec", "zdtm_netns", "ip", "link", "set", "up", "dev", "lo"]).wait():
 			raise Exception("ip link set up dev lo")
 
+	if opts['lazy_pages'] or opts['remote_lazy_pages']:
+		check = subprocess.Popen([criu_bin, "check", "--feature", "lazy_pages"], stderr = subprocess.PIPE)
+		ret = check.wait()
+		err = ''.join(check.stderr.readlines())
+		if ret:
+			if "UFFD is not supported" in err:
+				raise Exception("UFFD is not supported, cannot run with --lazy-pages")
+			# Most test will work with 4.3 - 4.10
+			print "WARNING: UFFD is missing essentail features"
+
 	l = launcher(opts, len(torun))
 	try:
 		for t in torun:

Comments

Pavel Emelyanov July 10, 2017, 8:32 a.m.
On 07/09/2017 09:12 AM, Mike Rapoport wrote:
> Raise an exception for kernels that do not have userfaultfd. For the
> kernels that have userfaultfd but do not provide non-cooperative events
> (4.3 - 4.10) warn about incomplete feature.
> 
> Fixes: #363
> 
> Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
> ---
>  test/zdtm.py | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/test/zdtm.py b/test/zdtm.py
> index 2ee76cd..e6d2701 100755
> --- a/test/zdtm.py
> +++ b/test/zdtm.py
> @@ -1767,6 +1767,16 @@ def run_tests(opts):
>  		if subprocess.Popen(["ip", "netns", "exec", "zdtm_netns", "ip", "link", "set", "up", "dev", "lo"]).wait():
>  			raise Exception("ip link set up dev lo")
>  
> +	if opts['lazy_pages'] or opts['remote_lazy_pages']:
> +		check = subprocess.Popen([criu_bin, "check", "--feature", "lazy_pages"], stderr = subprocess.PIPE)

There's a helper for that:

        if opts['pre'] or opts['snaps']:
                if not criu.check("mem_dirty_track"):
                        print "Tracking memory is not available"
                        return

so you can do it with

	if not criu.check("lazy_pages"):
		<bail-out>

> +		ret = check.wait()
> +		err = ''.join(check.stderr.readlines())
> +		if ret:
> +			if "UFFD is not supported" in err:
> +				raise Exception("UFFD is not supported, cannot run with --lazy-pages")
> +			# Most test will work with 4.3 - 4.10
> +			print "WARNING: UFFD is missing essentail features"
> +
>  	l = launcher(opts, len(torun))
>  	try:
>  		for t in torun:
>
Mike Rapoport July 10, 2017, 8:48 a.m.
On Mon, Jul 10, 2017 at 11:32:09AM +0300, Pavel Emelyanov wrote:
> On 07/09/2017 09:12 AM, Mike Rapoport wrote:
> > Raise an exception for kernels that do not have userfaultfd. For the
> > kernels that have userfaultfd but do not provide non-cooperative events
> > (4.3 - 4.10) warn about incomplete feature.
> > 
> > Fixes: #363
> > 
> > Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
> > ---
> >  test/zdtm.py | 10 ++++++++++
> >  1 file changed, 10 insertions(+)
> > 
> > diff --git a/test/zdtm.py b/test/zdtm.py
> > index 2ee76cd..e6d2701 100755
> > --- a/test/zdtm.py
> > +++ b/test/zdtm.py
> > @@ -1767,6 +1767,16 @@ def run_tests(opts):
> >  		if subprocess.Popen(["ip", "netns", "exec", "zdtm_netns", "ip", "link", "set", "up", "dev", "lo"]).wait():
> >  			raise Exception("ip link set up dev lo")
> >  
> > +	if opts['lazy_pages'] or opts['remote_lazy_pages']:
> > +		check = subprocess.Popen([criu_bin, "check", "--feature", "lazy_pages"], stderr = subprocess.PIPE)
> 
> There's a helper for that:
> 
>         if opts['pre'] or opts['snaps']:
>                 if not criu.check("mem_dirty_track"):
>                         print "Tracking memory is not available"
>                         return
> 
> so you can do it with
> 
> 	if not criu.check("lazy_pages"):
> 		<bail-out>

It will return 1 even if uffd is present, but it still does not have
non-cooperative events. For instance, in travis ;-)
 
> > +		ret = check.wait()
> > +		err = ''.join(check.stderr.readlines())
> > +		if ret:
> > +			if "UFFD is not supported" in err:
> > +				raise Exception("UFFD is not supported, cannot run with --lazy-pages")
> > +			# Most test will work with 4.3 - 4.10
> > +			print "WARNING: UFFD is missing essentail features"
> > +
> >  	l = launcher(opts, len(torun))
> >  	try:
> >  		for t in torun:
> > 
>
Pavel Emelyanov July 10, 2017, 9:20 a.m.
On 07/10/2017 11:48 AM, Mike Rapoport wrote:
> On Mon, Jul 10, 2017 at 11:32:09AM +0300, Pavel Emelyanov wrote:
>> On 07/09/2017 09:12 AM, Mike Rapoport wrote:
>>> Raise an exception for kernels that do not have userfaultfd. For the
>>> kernels that have userfaultfd but do not provide non-cooperative events
>>> (4.3 - 4.10) warn about incomplete feature.
>>>
>>> Fixes: #363
>>>
>>> Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
>>> ---
>>>  test/zdtm.py | 10 ++++++++++
>>>  1 file changed, 10 insertions(+)
>>>
>>> diff --git a/test/zdtm.py b/test/zdtm.py
>>> index 2ee76cd..e6d2701 100755
>>> --- a/test/zdtm.py
>>> +++ b/test/zdtm.py
>>> @@ -1767,6 +1767,16 @@ def run_tests(opts):
>>>  		if subprocess.Popen(["ip", "netns", "exec", "zdtm_netns", "ip", "link", "set", "up", "dev", "lo"]).wait():
>>>  			raise Exception("ip link set up dev lo")
>>>  
>>> +	if opts['lazy_pages'] or opts['remote_lazy_pages']:
>>> +		check = subprocess.Popen([criu_bin, "check", "--feature", "lazy_pages"], stderr = subprocess.PIPE)
>>
>> There's a helper for that:
>>
>>         if opts['pre'] or opts['snaps']:
>>                 if not criu.check("mem_dirty_track"):
>>                         print "Tracking memory is not available"
>>                         return
>>
>> so you can do it with
>>
>> 	if not criu.check("lazy_pages"):
>> 		<bail-out>
> 
> It will return 1 even if uffd is present, but it still does not have
> non-cooperative events. For instance, in travis ;-)

Then there should be one more feature in criu, not just lazy_pages :) Parsing
stdout of it will break some day soon.

-- Pavel

>>> +		ret = check.wait()
>>> +		err = ''.join(check.stderr.readlines())
>>> +		if ret:
>>> +			if "UFFD is not supported" in err:
>>> +				raise Exception("UFFD is not supported, cannot run with --lazy-pages")
>>> +			# Most test will work with 4.3 - 4.10
>>> +			print "WARNING: UFFD is missing essentail features"
>>> +
>>>  	l = launcher(opts, len(torun))
>>>  	try:
>>>  		for t in torun:
>>>
>>
>
Mike Rapoport July 10, 2017, 11:15 a.m.
On Mon, Jul 10, 2017 at 12:20:00PM +0300, Pavel Emelyanov wrote:
> On 07/10/2017 11:48 AM, Mike Rapoport wrote:
> > On Mon, Jul 10, 2017 at 11:32:09AM +0300, Pavel Emelyanov wrote:
> >> On 07/09/2017 09:12 AM, Mike Rapoport wrote:
> >>> Raise an exception for kernels that do not have userfaultfd. For the
> >>> kernels that have userfaultfd but do not provide non-cooperative events
> >>> (4.3 - 4.10) warn about incomplete feature.
> >>>
> >>> Fixes: #363
> >>>
> >>> Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
> >>> ---
> >>>  test/zdtm.py | 10 ++++++++++
> >>>  1 file changed, 10 insertions(+)
> >>>
> >>> diff --git a/test/zdtm.py b/test/zdtm.py
> >>> index 2ee76cd..e6d2701 100755
> >>> --- a/test/zdtm.py
> >>> +++ b/test/zdtm.py
> >>> @@ -1767,6 +1767,16 @@ def run_tests(opts):
> >>>  		if subprocess.Popen(["ip", "netns", "exec", "zdtm_netns", "ip", "link", "set", "up", "dev", "lo"]).wait():
> >>>  			raise Exception("ip link set up dev lo")
> >>>  
> >>> +	if opts['lazy_pages'] or opts['remote_lazy_pages']:
> >>> +		check = subprocess.Popen([criu_bin, "check", "--feature", "lazy_pages"], stderr = subprocess.PIPE)
> >>
> >> There's a helper for that:
> >>
> >>         if opts['pre'] or opts['snaps']:
> >>                 if not criu.check("mem_dirty_track"):
> >>                         print "Tracking memory is not available"
> >>                         return
> >>
> >> so you can do it with
> >>
> >> 	if not criu.check("lazy_pages"):
> >> 		<bail-out>
> > 
> > It will return 1 even if uffd is present, but it still does not have
> > non-cooperative events. For instance, in travis ;-)
> 
> Then there should be one more feature in criu, not just lazy_pages :) Parsing
> stdout of it will break some day soon.

Hmm, I wouldn't want criu check to report that lazy-pages-something is
available for kernels older than 4.11...
 
> -- Pavel
> 
> >>> +		ret = check.wait()
> >>> +		err = ''.join(check.stderr.readlines())
> >>> +		if ret:
> >>> +			if "UFFD is not supported" in err:
> >>> +				raise Exception("UFFD is not supported, cannot run with --lazy-pages")
> >>> +			# Most test will work with 4.3 - 4.10
> >>> +			print "WARNING: UFFD is missing essentail features"
> >>> +
> >>>  	l = launcher(opts, len(torun))
> >>>  	try:
> >>>  		for t in torun:
> >>>
> >>
> > 
>
Mike Rapoport July 12, 2017, 8:43 a.m.
On Mon, Jul 10, 2017 at 09:59:02PM -0000, Patchwork wrote:
> == Series Details ==
> 
> Series: test/zdtm: do not allow running with --lazy-pages on kernel without uffd
> URL   : https://patchwork.criu.org/series/1740/
> State : failure
> 
> == Logs ==
> 
> For more details see: https://travis-ci.org/criupatchwork/criu/builds/252148009?utm_source=github_status&utm_medium=notification
> 

What would that mean:
profiling:/home/travis/build/criupatchwork/criu/criu/namespaces.gcda:Merge mismatch for function 45
?
Pavel Emelyanov Aug. 7, 2017, 2:56 p.m.
On 07/10/2017 02:15 PM, Mike Rapoport wrote:
> On Mon, Jul 10, 2017 at 12:20:00PM +0300, Pavel Emelyanov wrote:
>> On 07/10/2017 11:48 AM, Mike Rapoport wrote:
>>> On Mon, Jul 10, 2017 at 11:32:09AM +0300, Pavel Emelyanov wrote:
>>>> On 07/09/2017 09:12 AM, Mike Rapoport wrote:
>>>>> Raise an exception for kernels that do not have userfaultfd. For the
>>>>> kernels that have userfaultfd but do not provide non-cooperative events
>>>>> (4.3 - 4.10) warn about incomplete feature.
>>>>>
>>>>> Fixes: #363
>>>>>
>>>>> Signed-off-by: Mike Rapoport <rppt@linux.vnet.ibm.com>
>>>>> ---
>>>>>  test/zdtm.py | 10 ++++++++++
>>>>>  1 file changed, 10 insertions(+)
>>>>>
>>>>> diff --git a/test/zdtm.py b/test/zdtm.py
>>>>> index 2ee76cd..e6d2701 100755
>>>>> --- a/test/zdtm.py
>>>>> +++ b/test/zdtm.py
>>>>> @@ -1767,6 +1767,16 @@ def run_tests(opts):
>>>>>  		if subprocess.Popen(["ip", "netns", "exec", "zdtm_netns", "ip", "link", "set", "up", "dev", "lo"]).wait():
>>>>>  			raise Exception("ip link set up dev lo")
>>>>>  
>>>>> +	if opts['lazy_pages'] or opts['remote_lazy_pages']:
>>>>> +		check = subprocess.Popen([criu_bin, "check", "--feature", "lazy_pages"], stderr = subprocess.PIPE)
>>>>
>>>> There's a helper for that:
>>>>
>>>>         if opts['pre'] or opts['snaps']:
>>>>                 if not criu.check("mem_dirty_track"):
>>>>                         print "Tracking memory is not available"
>>>>                         return
>>>>
>>>> so you can do it with
>>>>
>>>> 	if not criu.check("lazy_pages"):
>>>> 		<bail-out>
>>>
>>> It will return 1 even if uffd is present, but it still does not have
>>> non-cooperative events. For instance, in travis ;-)
>>
>> Then there should be one more feature in criu, not just lazy_pages :) Parsing
>> stdout of it will break some day soon.
> 
> Hmm, I wouldn't want criu check to report that lazy-pages-something is
> available for kernels older than 4.11...

Sure. This means, that we need criu check --feature uffd and --feature uffd-noncoop or smth
like that.

-- Pavel