'Docker: In Dockerfile, copy files temporarily, but not for final image

I have a Java service I'd like to package, and the only thing the final docker image needs is the JAR file and a config file. However, I need to run my gradle command first to build the JAR, but I don't want all the things that gradle uses to be in the result docker image.

Here's my current DockerFile:

RUN apt-get update && apt-get install -y openjdk-7-jdk
COPY . /
RUN ./gradlew shadowJar
CMD ["java", "-jar", "service/build/libs/service.jar", "server", "service/service.yml"]

You can see I have to COPY everything first so that I can run ./gradlew (otherwise it says the command cannot be found). But in the end, all I need is the service.jar and service.yml files.

I'm probably missing something, but how I can make everything available during the ./gradlew build step, but only have the result image include the service.jar and service.yml.

docker

Solution 1:^[1]

Docker introduced so called "multi stage builds". With them, you can avoid having unnecessary files in your image. Your Dockerfile would look like this:

FROM ubuntu AS build
COPY . /src
# run your build

FROM ubuntu
RUN mkdir -p /dist/
COPY --from=build /src/my.object /dist/

The principle is simple. In the first FROM you name your build. In the second FROM you can use the COPY --from=-parameter to copy files from your first build to your second. The second build is the one which results in an usable image later.

If you want to test the results of your build instead the resulting image, you can use docker build --target build myimage:build .. The resulting image only includes the first FROM in your Dockerfile.

Try to use the same base image für your "build" and the final image. Different base images might result in strange bugs or even segfaults.

For more information: https://docs.docker.com/develop/develop-images/multistage-build/#use-multi-stage-builds

Solution 2:^[2]

Building an image works as follows.

... The docker build command will use whatever directory contains the Dockerfile as the build context (including all of its subdirectories). The build context will be sent to the Docker daemon before building the image, which means if you use / as the source repository, the entire contents of your hard drive will get sent to the daemon ...

See https://docs.docker.com/reference/builder/

I see no way to achieve what you want. There are two options:

Having all the build dependencies inside the image and build your JAR file inside the container. That bloats your image.
I would recommend to build your JAR separately and just ADD the executable and config files when they are build. That means all build dependencies must be available on your development environment, but your image is as small as it can be.

Solution 3:^[3]

Another workaround would be to use a web server to pull in the data.

Start a web server, serving your temporarily needed files. You may use
```
python3 -m http.server 8000 --bind 127.0.0.1 &
```
to serve the current directory, see https://docs.python.org/3/library/http.server.html#http-server-cli

In your Dockerfile get the file, perform actions, delete the files in a single RUN

RUN wget http://localhost:8000/big.tar && tar xf big.tar && ... && rm big.tar

Build image with --network host to be able to access localhost

Solution 4:^[4]

There is an experimental build flag --squash to summarize everything into one layer.

For example, docker build --squash -t <tag> .

The documentation https://docs.docker.com/engine/reference/commandline/image_build/#options.

The Github discussion https://github.com/moby/moby/issues/332.

How to enable experimental features in the daemon? https://github.com/docker/docker-ce/blob/master/components/cli/experimental/README.md

Solution 5:^[5]

I adopted various comments and came up with this solution using DOCKER_BUILDKIT. I copy somefile.tar.gz temporarily into stage 0, but that file is not present in the final image. The overall container size is reduced.

# syntax=docker/dockerfile:experimental

# File: Dockerfile
# Build:
#     DOCKER_BUILDKIT=1 docker build -t sometag -f Dockerfile $(pwd)

# =============================================================================
# STAGE 0 ---------------------------------------------------------------------
# =============================================================================
FROM somebase as devel

COPY somefile.tar.gz /somemodules/
RUN cd /somemodules && \
    tar -xf somefile.tar.gz

# =============================================================================
# STAGE 1 ---------------------------------------------------------------------
# =============================================================================
FROM same_or_another_base

RUN apt-get update -y && \
    apt-get install -y --no-install-recommends \
        rsync && \
    rm -rf /var/lib/apt/lists/*

RUN  --mount=from=devel,src=/somemodules,dst=/somemodules \    
    rsync -I -K -a /somemodules/somefile/* /usr/local/somemodules/

RUN echo "/usr/local/somemodules/lib" >> /etc/ld.so.conf.d/somemodules.conf
ENV PATH=/usr/local/somemodules/bin:$PATH

# a bunch of other stuff . . .

RUN ldconfig

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1
Solution 2	xh3b4sd
Solution 3	Adam
Solution 4	Phizaz
Solution 5	Al Conrad

'Docker: In Dockerfile, copy files temporarily, but not for final image

Solution 1:[1]

Solution 2:[2]

Solution 3:[3]

Solution 4:[4]

Solution 5:[5]