[7/7] travis: enable ccache for docker/qemu builds

Submitted by Kir Kolyshkin on March 15, 2017, 11:17 p.m.

Details

Message ID 20170315231724.9323-8-kir@openvz.org
State New
Series "travis build speedup and fixes"
Headers show

Commit Message

Kir Kolyshkin March 15, 2017, 11:17 p.m.
As we compile-test non-x86_64 architectures under qemu emulation,
it works pretty slow.
Dmitry Safonov suggested, and Andrey Vagin initially implemented
supporting ccache for such builds. This patch is based heavily
on Andrey's work -- all the bugs added are purely mine though.

Performance results: in an ideal environment (two builds of the same
code, one with cold (empty) ccache, another with the hot one)
I saw compile time improvements of 4x to 5x, and total test run time
improvement up to 2x to 2.5x. In layman terms, the complete test run
that was taking more than 50 minutes now takes about 25!

Notes on handling .ccache directory:

1. Before running docker build, .ccache directory (saved in between
   runs by Travis) is moved to criu source code root, from where it
   is copied by docker together with criu sources.

2. In Dockerfile, .ccache gets moved to /tmp, and CCACHE_DIR
   is set accordingly.

3. After running docker build, .ccache is copied out from docker
   container back to the host (from where it is saved by Travis).

Ccache envorinment notes:

1. CCACHE_NOCOMPRESS is needed because otherwise tons of time is spent
   on compression/decompression (big performance hit under qemu).

2. CCACHE_CPP2 is required with clang, see detailed explanation at
   http://petereisentraut.blogspot.com/2011/09/ccache-and-clang-part-2.html
   The logic of setting CCACHE_CPP2 in Dockerfile is somewhat fancy;
   unfortunately I was not able to come up with a simpler approach.

Misc:

1. Travis runs "ccache -s" after the build is completed. Also, ccache -s
   is called by Dockerfile before make, so you can compare before/after
   numbers.

2. make invocations are surrounded by "date" calls so you can see the
   timings. For the sake of accurate timings, a separate compilation
   of criu/parasite-syscall.o (see commit 48f680df) is commented out.

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
---
 scripts/build/Dockerfile.alpine | 13 +++++++++++--
 scripts/build/Dockerfile.tmpl   | 14 +++++++++++---
 scripts/build/Makefile          |  6 +++++-
 3 files changed, 27 insertions(+), 6 deletions(-)

Patch hide | download patch | download mbox

diff --git a/scripts/build/Dockerfile.alpine b/scripts/build/Dockerfile.alpine
index 86738a74..51da014 100644
--- a/scripts/build/Dockerfile.alpine
+++ b/scripts/build/Dockerfile.alpine
@@ -1,5 +1,7 @@ 
 FROM alpine:3.5
 ARG CC=gcc
+ARG ENV1=FOOBAR
+
 RUN apk update && apk add \
                 build-base \
                 coreutils \
@@ -12,11 +14,18 @@  RUN apk update && apk add \
                 libnl3-dev \
                 pkgconfig \
                 libnet-dev \
+                ccache \
                 $CC
 
 COPY . /criu
 WORKDIR /criu
-RUN make mrproper && make -j $(nproc) CC=$CC
+RUN mv .ccache /tmp
+ENV CC="ccache $CC"
+ENV CCACHE_DIR=/tmp/.ccache
+ENV CCACHE_NOCOMPRESS=1
+ENV $ENV1 yes
+RUN ccache -s
+RUN date && make mrproper && make -j $(nproc) CC="$CC" && date
 
 # to run tests
 RUN apk add	py-yaml \
@@ -26,4 +35,4 @@  RUN apk add	py-yaml \
 		iproute2
 
 RUN pip install protobuf
-RUN make -C test/zdtm/static env00
+RUN date && make -C test/zdtm/static env00 && date
diff --git a/scripts/build/Dockerfile.tmpl b/scripts/build/Dockerfile.tmpl
index 6e35f87..bab8aad 100644
--- a/scripts/build/Dockerfile.tmpl
+++ b/scripts/build/Dockerfile.tmpl
@@ -1,4 +1,5 @@ 
 ARG CC=gcc
+ARG ENV1=FOOBAR
 
 RUN apt-get update && apt-get install -y \
                 build-essential \
@@ -16,15 +17,22 @@  RUN apt-get update && apt-get install -y \
                 pkg-config \
                 git-core \
 		libnet-dev \
+		ccache \
                 $CC
 
 COPY . /criu
 WORKDIR /criu
+RUN mv .ccache /tmp
+ENV CC="ccache $CC"
+ENV CCACHE_DIR=/tmp/.ccache
+ENV CCACHE_NOCOMPRESS=1
+ENV $ENV1 yes
 
 RUN make mrproper
-RUN make -j $(nproc) CC=$CC V=1 criu/parasite-syscall.o
-RUN make -j $(nproc) CC=$CC V=1
+RUN ccache -s
+#RUN make -j $(nproc) CC=$CC V=1 criu/parasite-syscall.o
+RUN date; make -j $(nproc) CC="$CC" V=1; date
 RUN make mrproper
 RUN bash -c 'CLEAN="$(git clean -ndx --exclude=scripts/build --exclude=.config)"; echo "${CLEAN}"; test -z "${CLEAN}"; exit $?'
-RUN make -j $(nproc) CC=$CC -C test/zdtm
+RUN date; make -j $(nproc) CC="$CC" -C test/zdtm; date
 #RUN make test/compel/handle_binary && ./test/compel/handle_binary
diff --git a/scripts/build/Makefile b/scripts/build/Makefile
index fc8317b..b17bf10 100644
--- a/scripts/build/Makefile
+++ b/scripts/build/Makefile
@@ -25,7 +25,10 @@  binfmt_misc:
 $(QEMU_ARCHES): qemu-user-static binfmt_misc
 
 $(TARGETS):
-	docker build  -t criu-$@ -f Dockerfile.$@ $(DB_CC) ../..
+	mkdir -p $(HOME)/.ccache
+	mv $(HOME)/.ccache ../../
+	docker build  -t criu-$@ -f Dockerfile.$@ $(DB_CC) $(DB_ENV) ../..
+	docker run criu-$@ tar c -C /tmp .ccache | tar x -C $(HOME)
 .PHONY: $(TARGETS)
 
 # Clang builds add some Docker build env
@@ -35,6 +38,7 @@  endef
 $(foreach t,$(TARGETS),$(eval $(call CLANG_DEP,$(t))))
 
 %-clang:	DB_CC=--build-arg CC=clang-3.8
+%-clang:	DB_ENV=--build-arg ENV1=CCACHE_CPP2
 alpine-clang:	DB_CC=--build-arg CC=clang
 aarch64-clang:	DB_CC=--build-arg CC=clang-3.6
 .PHONY: $(TARGETS_CLANG)

Comments

Dmitry Safonov March 16, 2017, 2:54 p.m.
On 03/16/2017 02:17 AM, Kir Kolyshkin wrote:
> As we compile-test non-x86_64 architectures under qemu emulation,
> it works pretty slow.
> Dmitry Safonov suggested, and Andrey Vagin initially implemented
> supporting ccache for such builds. This patch is based heavily
> on Andrey's work -- all the bugs added are purely mine though.
>
> Performance results: in an ideal environment (two builds of the same
> code, one with cold (empty) ccache, another with the hot one)
> I saw compile time improvements of 4x to 5x, and total test run time
> improvement up to 2x to 2.5x. In layman terms, the complete test run
> that was taking more than 50 minutes now takes about 25!
>
> Notes on handling .ccache directory:
>
> 1. Before running docker build, .ccache directory (saved in between
>    runs by Travis) is moved to criu source code root, from where it
>    is copied by docker together with criu sources.
>
> 2. In Dockerfile, .ccache gets moved to /tmp, and CCACHE_DIR
>    is set accordingly.
>
> 3. After running docker build, .ccache is copied out from docker
>    container back to the host (from where it is saved by Travis).
>
> Ccache envorinment notes:
>
> 1. CCACHE_NOCOMPRESS is needed because otherwise tons of time is spent
>    on compression/decompression (big performance hit under qemu).
>
> 2. CCACHE_CPP2 is required with clang, see detailed explanation at
>    http://petereisentraut.blogspot.com/2011/09/ccache-and-clang-part-2.html
>    The logic of setting CCACHE_CPP2 in Dockerfile is somewhat fancy;
>    unfortunately I was not able to come up with a simpler approach.
>
> Misc:
>
> 1. Travis runs "ccache -s" after the build is completed. Also, ccache -s
>    is called by Dockerfile before make, so you can compare before/after
>    numbers.
>
> 2. make invocations are surrounded by "date" calls so you can see the
>    timings. For the sake of accurate timings, a separate compilation
>    of criu/parasite-syscall.o (see commit 48f680df) is commented out.

Hmm, so this separate compilation is a test that building not whole
criu, but a object file works - it was broken some time ago and I added
this to check that any changes to makefiles does not break it.

I expected that building an object that anyway is needed by criu
wouldn't enlarge compilation time much. Did you test how much time is
saved here?

The other questions:
- is exact build time needed? I mean, patches that speed up compilation
can provide this information, but is it always needed? (I don't object,
just questioning)
- is building some object file needed? Usally I don't use it and
recompile all criu, but maybe there are users? Should we keep the test
then to ensure that that ability will not broke?

>
> Signed-off-by: Kir Kolyshkin <kir@openvz.org>
> ---
>  scripts/build/Dockerfile.alpine | 13 +++++++++++--
>  scripts/build/Dockerfile.tmpl   | 14 +++++++++++---
>  scripts/build/Makefile          |  6 +++++-
>  3 files changed, 27 insertions(+), 6 deletions(-)
>
> diff --git a/scripts/build/Dockerfile.alpine b/scripts/build/Dockerfile.alpine
> index 86738a74..51da014 100644
> --- a/scripts/build/Dockerfile.alpine
> +++ b/scripts/build/Dockerfile.alpine
> @@ -1,5 +1,7 @@
>  FROM alpine:3.5
>  ARG CC=gcc
> +ARG ENV1=FOOBAR
> +
>  RUN apk update && apk add \
>                  build-base \
>                  coreutils \
> @@ -12,11 +14,18 @@ RUN apk update && apk add \
>                  libnl3-dev \
>                  pkgconfig \
>                  libnet-dev \
> +                ccache \
>                  $CC
>
>  COPY . /criu
>  WORKDIR /criu
> -RUN make mrproper && make -j $(nproc) CC=$CC
> +RUN mv .ccache /tmp
> +ENV CC="ccache $CC"
> +ENV CCACHE_DIR=/tmp/.ccache
> +ENV CCACHE_NOCOMPRESS=1
> +ENV $ENV1 yes
> +RUN ccache -s
> +RUN date && make mrproper && make -j $(nproc) CC="$CC" && date
>
>  # to run tests
>  RUN apk add	py-yaml \
> @@ -26,4 +35,4 @@ RUN apk add	py-yaml \
>  		iproute2
>
>  RUN pip install protobuf
> -RUN make -C test/zdtm/static env00
> +RUN date && make -C test/zdtm/static env00 && date
> diff --git a/scripts/build/Dockerfile.tmpl b/scripts/build/Dockerfile.tmpl
> index 6e35f87..bab8aad 100644
> --- a/scripts/build/Dockerfile.tmpl
> +++ b/scripts/build/Dockerfile.tmpl
> @@ -1,4 +1,5 @@
>  ARG CC=gcc
> +ARG ENV1=FOOBAR
>
>  RUN apt-get update && apt-get install -y \
>                  build-essential \
> @@ -16,15 +17,22 @@ RUN apt-get update && apt-get install -y \
>                  pkg-config \
>                  git-core \
>  		libnet-dev \
> +		ccache \
>                  $CC
>
>  COPY . /criu
>  WORKDIR /criu
> +RUN mv .ccache /tmp
> +ENV CC="ccache $CC"
> +ENV CCACHE_DIR=/tmp/.ccache
> +ENV CCACHE_NOCOMPRESS=1
> +ENV $ENV1 yes
>
>  RUN make mrproper
> -RUN make -j $(nproc) CC=$CC V=1 criu/parasite-syscall.o
> -RUN make -j $(nproc) CC=$CC V=1
> +RUN ccache -s
> +#RUN make -j $(nproc) CC=$CC V=1 criu/parasite-syscall.o
> +RUN date; make -j $(nproc) CC="$CC" V=1; date
>  RUN make mrproper
>  RUN bash -c 'CLEAN="$(git clean -ndx --exclude=scripts/build --exclude=.config)"; echo "${CLEAN}"; test -z "${CLEAN}"; exit $?'
> -RUN make -j $(nproc) CC=$CC -C test/zdtm
> +RUN date; make -j $(nproc) CC="$CC" -C test/zdtm; date
>  #RUN make test/compel/handle_binary && ./test/compel/handle_binary
> diff --git a/scripts/build/Makefile b/scripts/build/Makefile
> index fc8317b..b17bf10 100644
> --- a/scripts/build/Makefile
> +++ b/scripts/build/Makefile
> @@ -25,7 +25,10 @@ binfmt_misc:
>  $(QEMU_ARCHES): qemu-user-static binfmt_misc
>
>  $(TARGETS):
> -	docker build  -t criu-$@ -f Dockerfile.$@ $(DB_CC) ../..
> +	mkdir -p $(HOME)/.ccache
> +	mv $(HOME)/.ccache ../../
> +	docker build  -t criu-$@ -f Dockerfile.$@ $(DB_CC) $(DB_ENV) ../..
> +	docker run criu-$@ tar c -C /tmp .ccache | tar x -C $(HOME)
>  .PHONY: $(TARGETS)
>
>  # Clang builds add some Docker build env
> @@ -35,6 +38,7 @@ endef
>  $(foreach t,$(TARGETS),$(eval $(call CLANG_DEP,$(t))))
>
>  %-clang:	DB_CC=--build-arg CC=clang-3.8
> +%-clang:	DB_ENV=--build-arg ENV1=CCACHE_CPP2
>  alpine-clang:	DB_CC=--build-arg CC=clang
>  aarch64-clang:	DB_CC=--build-arg CC=clang-3.6
>  .PHONY: $(TARGETS_CLANG)
>
Kir Kolyshkin March 16, 2017, 4:41 p.m.
On 03/16/2017 07:54 AM, Dmitry Safonov wrote:
> On 03/16/2017 02:17 AM, Kir Kolyshkin wrote:
>> As we compile-test non-x86_64 architectures under qemu emulation,
>> it works pretty slow.
>> Dmitry Safonov suggested, and Andrey Vagin initially implemented
>> supporting ccache for such builds. This patch is based heavily
>> on Andrey's work -- all the bugs added are purely mine though.
>>
>> Performance results: in an ideal environment (two builds of the same
>> code, one with cold (empty) ccache, another with the hot one)
>> I saw compile time improvements of 4x to 5x, and total test run time
>> improvement up to 2x to 2.5x. In layman terms, the complete test run
>> that was taking more than 50 minutes now takes about 25!
>>
>> Notes on handling .ccache directory:
>>
>> 1. Before running docker build, .ccache directory (saved in between
>>    runs by Travis) is moved to criu source code root, from where it
>>    is copied by docker together with criu sources.
>>
>> 2. In Dockerfile, .ccache gets moved to /tmp, and CCACHE_DIR
>>    is set accordingly.
>>
>> 3. After running docker build, .ccache is copied out from docker
>>    container back to the host (from where it is saved by Travis).
>>
>> Ccache envorinment notes:
>>
>> 1. CCACHE_NOCOMPRESS is needed because otherwise tons of time is spent
>>    on compression/decompression (big performance hit under qemu).
>>
>> 2. CCACHE_CPP2 is required with clang, see detailed explanation at
>> http://petereisentraut.blogspot.com/2011/09/ccache-and-clang-part-2.html
>>    The logic of setting CCACHE_CPP2 in Dockerfile is somewhat fancy;
>>    unfortunately I was not able to come up with a simpler approach.
>>
>> Misc:
>>
>> 1. Travis runs "ccache -s" after the build is completed. Also, ccache -s
>>    is called by Dockerfile before make, so you can compare before/after
>>    numbers.
>>
>> 2. make invocations are surrounded by "date" calls so you can see the
>>    timings. For the sake of accurate timings, a separate compilation
>>    of criu/parasite-syscall.o (see commit 48f680df) is commented out.
>
> Hmm, so this separate compilation is a test that building not whole
> criu, but a object file works - it was broken some time ago and I added
> this to check that any changes to makefiles does not break it.
>
> I expected that building an object that anyway is needed by criu
> wouldn't enlarge compilation time much. Did you test how much time is
> saved here?

It's not about saving compile time, it's about more accurate measurements
of a compile time. Since this is clean build, "make criu/parasite-syscall.o"
leads to building images/, compel/, libsoccr/, and criu/pie/ -- that takes
about 1/2 of all criu compile time.

>
> The other questions:
> - is exact build time needed? I mean, patches that speed up compilation
> can provide this information, but is it always needed? (I don't object,
> just questioning)

Surely it's only needed when you try to optimize it.

> - is building some object file needed? Usally I don't use it and
> recompile all criu, but maybe there are users? Should we keep the test
> then to ensure that that ability will not broke?

I think it is needed.

OK, I have an idea of how to add the test back, will send a patch soon.

>
>>
>> Signed-off-by: Kir Kolyshkin <kir@openvz.org>
>> ---
>>  scripts/build/Dockerfile.alpine | 13 +++++++++++--
>>  scripts/build/Dockerfile.tmpl   | 14 +++++++++++---
>>  scripts/build/Makefile          |  6 +++++-
>>  3 files changed, 27 insertions(+), 6 deletions(-)
>>
>> diff --git a/scripts/build/Dockerfile.alpine 
>> b/scripts/build/Dockerfile.alpine
>> index 86738a74..51da014 100644
>> --- a/scripts/build/Dockerfile.alpine
>> +++ b/scripts/build/Dockerfile.alpine
>> @@ -1,5 +1,7 @@
>>  FROM alpine:3.5
>>  ARG CC=gcc
>> +ARG ENV1=FOOBAR
>> +
>>  RUN apk update && apk add \
>>                  build-base \
>>                  coreutils \
>> @@ -12,11 +14,18 @@ RUN apk update && apk add \
>>                  libnl3-dev \
>>                  pkgconfig \
>>                  libnet-dev \
>> +                ccache \
>>                  $CC
>>
>>  COPY . /criu
>>  WORKDIR /criu
>> -RUN make mrproper && make -j $(nproc) CC=$CC
>> +RUN mv .ccache /tmp
>> +ENV CC="ccache $CC"
>> +ENV CCACHE_DIR=/tmp/.ccache
>> +ENV CCACHE_NOCOMPRESS=1
>> +ENV $ENV1 yes
>> +RUN ccache -s
>> +RUN date && make mrproper && make -j $(nproc) CC="$CC" && date
>>
>>  # to run tests
>>  RUN apk add    py-yaml \
>> @@ -26,4 +35,4 @@ RUN apk add    py-yaml \
>>          iproute2
>>
>>  RUN pip install protobuf
>> -RUN make -C test/zdtm/static env00
>> +RUN date && make -C test/zdtm/static env00 && date
>> diff --git a/scripts/build/Dockerfile.tmpl 
>> b/scripts/build/Dockerfile.tmpl
>> index 6e35f87..bab8aad 100644
>> --- a/scripts/build/Dockerfile.tmpl
>> +++ b/scripts/build/Dockerfile.tmpl
>> @@ -1,4 +1,5 @@
>>  ARG CC=gcc
>> +ARG ENV1=FOOBAR
>>
>>  RUN apt-get update && apt-get install -y \
>>                  build-essential \
>> @@ -16,15 +17,22 @@ RUN apt-get update && apt-get install -y \
>>                  pkg-config \
>>                  git-core \
>>          libnet-dev \
>> +        ccache \
>>                  $CC
>>
>>  COPY . /criu
>>  WORKDIR /criu
>> +RUN mv .ccache /tmp
>> +ENV CC="ccache $CC"
>> +ENV CCACHE_DIR=/tmp/.ccache
>> +ENV CCACHE_NOCOMPRESS=1
>> +ENV $ENV1 yes
>>
>>  RUN make mrproper
>> -RUN make -j $(nproc) CC=$CC V=1 criu/parasite-syscall.o
>> -RUN make -j $(nproc) CC=$CC V=1
>> +RUN ccache -s
>> +#RUN make -j $(nproc) CC=$CC V=1 criu/parasite-syscall.o
>> +RUN date; make -j $(nproc) CC="$CC" V=1; date
>>  RUN make mrproper
>>  RUN bash -c 'CLEAN="$(git clean -ndx --exclude=scripts/build 
>> --exclude=.config)"; echo "${CLEAN}"; test -z "${CLEAN}"; exit $?'
>> -RUN make -j $(nproc) CC=$CC -C test/zdtm
>> +RUN date; make -j $(nproc) CC="$CC" -C test/zdtm; date
>>  #RUN make test/compel/handle_binary && ./test/compel/handle_binary
>> diff --git a/scripts/build/Makefile b/scripts/build/Makefile
>> index fc8317b..b17bf10 100644
>> --- a/scripts/build/Makefile
>> +++ b/scripts/build/Makefile
>> @@ -25,7 +25,10 @@ binfmt_misc:
>>  $(QEMU_ARCHES): qemu-user-static binfmt_misc
>>
>>  $(TARGETS):
>> -    docker build  -t criu-$@ -f Dockerfile.$@ $(DB_CC) ../..
>> +    mkdir -p $(HOME)/.ccache
>> +    mv $(HOME)/.ccache ../../
>> +    docker build  -t criu-$@ -f Dockerfile.$@ $(DB_CC) $(DB_ENV) ../..
>> +    docker run criu-$@ tar c -C /tmp .ccache | tar x -C $(HOME)
>>  .PHONY: $(TARGETS)
>>
>>  # Clang builds add some Docker build env
>> @@ -35,6 +38,7 @@ endef
>>  $(foreach t,$(TARGETS),$(eval $(call CLANG_DEP,$(t))))
>>
>>  %-clang:    DB_CC=--build-arg CC=clang-3.8
>> +%-clang:    DB_ENV=--build-arg ENV1=CCACHE_CPP2
>>  alpine-clang:    DB_CC=--build-arg CC=clang
>>  aarch64-clang:    DB_CC=--build-arg CC=clang-3.6
>>  .PHONY: $(TARGETS_CLANG)
>>
>
>
Dmitry Safonov March 16, 2017, 5:14 p.m.
On 03/16/2017 07:41 PM, Kir Kolyshkin wrote:
> On 03/16/2017 07:54 AM, Dmitry Safonov wrote:
>> On 03/16/2017 02:17 AM, Kir Kolyshkin wrote:
>>> As we compile-test non-x86_64 architectures under qemu emulation,
>>> it works pretty slow.
>>> Dmitry Safonov suggested, and Andrey Vagin initially implemented
>>> supporting ccache for such builds. This patch is based heavily
>>> on Andrey's work -- all the bugs added are purely mine though.
>>>
>>> Performance results: in an ideal environment (two builds of the same
>>> code, one with cold (empty) ccache, another with the hot one)
>>> I saw compile time improvements of 4x to 5x, and total test run time
>>> improvement up to 2x to 2.5x. In layman terms, the complete test run
>>> that was taking more than 50 minutes now takes about 25!
>>>
>>> Notes on handling .ccache directory:
>>>
>>> 1. Before running docker build, .ccache directory (saved in between
>>>    runs by Travis) is moved to criu source code root, from where it
>>>    is copied by docker together with criu sources.
>>>
>>> 2. In Dockerfile, .ccache gets moved to /tmp, and CCACHE_DIR
>>>    is set accordingly.
>>>
>>> 3. After running docker build, .ccache is copied out from docker
>>>    container back to the host (from where it is saved by Travis).
>>>
>>> Ccache envorinment notes:
>>>
>>> 1. CCACHE_NOCOMPRESS is needed because otherwise tons of time is spent
>>>    on compression/decompression (big performance hit under qemu).
>>>
>>> 2. CCACHE_CPP2 is required with clang, see detailed explanation at
>>> http://petereisentraut.blogspot.com/2011/09/ccache-and-clang-part-2.html
>>>    The logic of setting CCACHE_CPP2 in Dockerfile is somewhat fancy;
>>>    unfortunately I was not able to come up with a simpler approach.
>>>
>>> Misc:
>>>
>>> 1. Travis runs "ccache -s" after the build is completed. Also, ccache -s
>>>    is called by Dockerfile before make, so you can compare before/after
>>>    numbers.
>>>
>>> 2. make invocations are surrounded by "date" calls so you can see the
>>>    timings. For the sake of accurate timings, a separate compilation
>>>    of criu/parasite-syscall.o (see commit 48f680df) is commented out.
>>
>> Hmm, so this separate compilation is a test that building not whole
>> criu, but a object file works - it was broken some time ago and I added
>> this to check that any changes to makefiles does not break it.
>>
>> I expected that building an object that anyway is needed by criu
>> wouldn't enlarge compilation time much. Did you test how much time is
>> saved here?
>
> It's not about saving compile time, it's about more accurate measurements
> of a compile time. Since this is clean build, "make
> criu/parasite-syscall.o"
> leads to building images/, compel/, libsoccr/, and criu/pie/ -- that takes
> about 1/2 of all criu compile time.

Yes, but those built objects will be used on the next `make`, no need
to rebuild them again. There is only overhead of twice-make processing,
which I expect to be low.

>
>>
>> The other questions:
>> - is exact build time needed? I mean, patches that speed up compilation
>> can provide this information, but is it always needed? (I don't object,
>> just questioning)
>
> Surely it's only needed when you try to optimize it.
>
>> - is building some object file needed? Usally I don't use it and
>> recompile all criu, but maybe there are users? Should we keep the test
>> then to ensure that that ability will not broke?
>
> I think it is needed.
>
> OK, I have an idea of how to add the test back, will send a patch soon.

Ok, good, thanks!

>
>>
>>>
>>> Signed-off-by: Kir Kolyshkin <kir@openvz.org>
>>> ---
>>>  scripts/build/Dockerfile.alpine | 13 +++++++++++--
>>>  scripts/build/Dockerfile.tmpl   | 14 +++++++++++---
>>>  scripts/build/Makefile          |  6 +++++-
>>>  3 files changed, 27 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/scripts/build/Dockerfile.alpine
>>> b/scripts/build/Dockerfile.alpine
>>> index 86738a74..51da014 100644
>>> --- a/scripts/build/Dockerfile.alpine
>>> +++ b/scripts/build/Dockerfile.alpine
>>> @@ -1,5 +1,7 @@
>>>  FROM alpine:3.5
>>>  ARG CC=gcc
>>> +ARG ENV1=FOOBAR
>>> +
>>>  RUN apk update && apk add \
>>>                  build-base \
>>>                  coreutils \
>>> @@ -12,11 +14,18 @@ RUN apk update && apk add \
>>>                  libnl3-dev \
>>>                  pkgconfig \
>>>                  libnet-dev \
>>> +                ccache \
>>>                  $CC
>>>
>>>  COPY . /criu
>>>  WORKDIR /criu
>>> -RUN make mrproper && make -j $(nproc) CC=$CC
>>> +RUN mv .ccache /tmp
>>> +ENV CC="ccache $CC"
>>> +ENV CCACHE_DIR=/tmp/.ccache
>>> +ENV CCACHE_NOCOMPRESS=1
>>> +ENV $ENV1 yes
>>> +RUN ccache -s
>>> +RUN date && make mrproper && make -j $(nproc) CC="$CC" && date
>>>
>>>  # to run tests
>>>  RUN apk add    py-yaml \
>>> @@ -26,4 +35,4 @@ RUN apk add    py-yaml \
>>>          iproute2
>>>
>>>  RUN pip install protobuf
>>> -RUN make -C test/zdtm/static env00
>>> +RUN date && make -C test/zdtm/static env00 && date
>>> diff --git a/scripts/build/Dockerfile.tmpl
>>> b/scripts/build/Dockerfile.tmpl
>>> index 6e35f87..bab8aad 100644
>>> --- a/scripts/build/Dockerfile.tmpl
>>> +++ b/scripts/build/Dockerfile.tmpl
>>> @@ -1,4 +1,5 @@
>>>  ARG CC=gcc
>>> +ARG ENV1=FOOBAR
>>>
>>>  RUN apt-get update && apt-get install -y \
>>>                  build-essential \
>>> @@ -16,15 +17,22 @@ RUN apt-get update && apt-get install -y \
>>>                  pkg-config \
>>>                  git-core \
>>>          libnet-dev \
>>> +        ccache \
>>>                  $CC
>>>
>>>  COPY . /criu
>>>  WORKDIR /criu
>>> +RUN mv .ccache /tmp
>>> +ENV CC="ccache $CC"
>>> +ENV CCACHE_DIR=/tmp/.ccache
>>> +ENV CCACHE_NOCOMPRESS=1
>>> +ENV $ENV1 yes
>>>
>>>  RUN make mrproper
>>> -RUN make -j $(nproc) CC=$CC V=1 criu/parasite-syscall.o
>>> -RUN make -j $(nproc) CC=$CC V=1
>>> +RUN ccache -s
>>> +#RUN make -j $(nproc) CC=$CC V=1 criu/parasite-syscall.o
>>> +RUN date; make -j $(nproc) CC="$CC" V=1; date
>>>  RUN make mrproper
>>>  RUN bash -c 'CLEAN="$(git clean -ndx --exclude=scripts/build
>>> --exclude=.config)"; echo "${CLEAN}"; test -z "${CLEAN}"; exit $?'
>>> -RUN make -j $(nproc) CC=$CC -C test/zdtm
>>> +RUN date; make -j $(nproc) CC="$CC" -C test/zdtm; date
>>>  #RUN make test/compel/handle_binary && ./test/compel/handle_binary
>>> diff --git a/scripts/build/Makefile b/scripts/build/Makefile
>>> index fc8317b..b17bf10 100644
>>> --- a/scripts/build/Makefile
>>> +++ b/scripts/build/Makefile
>>> @@ -25,7 +25,10 @@ binfmt_misc:
>>>  $(QEMU_ARCHES): qemu-user-static binfmt_misc
>>>
>>>  $(TARGETS):
>>> -    docker build  -t criu-$@ -f Dockerfile.$@ $(DB_CC) ../..
>>> +    mkdir -p $(HOME)/.ccache
>>> +    mv $(HOME)/.ccache ../../
>>> +    docker build  -t criu-$@ -f Dockerfile.$@ $(DB_CC) $(DB_ENV) ../..
>>> +    docker run criu-$@ tar c -C /tmp .ccache | tar x -C $(HOME)
>>>  .PHONY: $(TARGETS)
>>>
>>>  # Clang builds add some Docker build env
>>> @@ -35,6 +38,7 @@ endef
>>>  $(foreach t,$(TARGETS),$(eval $(call CLANG_DEP,$(t))))
>>>
>>>  %-clang:    DB_CC=--build-arg CC=clang-3.8
>>> +%-clang:    DB_ENV=--build-arg ENV1=CCACHE_CPP2
>>>  alpine-clang:    DB_CC=--build-arg CC=clang
>>>  aarch64-clang:    DB_CC=--build-arg CC=clang-3.6
>>>  .PHONY: $(TARGETS_CLANG)
>>>
>>
>>
>