docker executor vs docker dind image

Question

I am a newbie in gitlabci. I want to understand why do we need docker dind image in order to build a docker image in GitLab CI jobs. Why can't we use the docker executor and run docker commands under scripts?

When we register docker executor gitlab runner, we choose one image.. Again inside gitlabci, we choose an image under image: or services: fields. So does that mean this GitLab CI job container runs inside the docker executor container?

Answer 1

why do we need docker dind image in order to build a docker image in GitLab CI jobs. Why can't we use the docker executor and run docker commands under scripts?

This partly depends on how you have configured your GitLab runner.

Why docker doesn't work inside containers

When you invoke docker commands, they are really talking to a docker daemon which is needed to perform builds and carry out other docker commands. Typically, jobs running under the docker executor do not have access to any docker daemon by default. It's the same kind of problem you would face if you tried to run docker inside of a docker container you started locally.

Even if I can run docker successfully on my host:

$ docker run --rm docker /bin/sh -c 'hello from container $HOSTNAME'
hello from container 2b51479b11b1

I cannot run docker inside the container

$ docker run --rm docker /bin/sh -c 'docker info'
errors pretty printing info
Client:
 Context:    default
 Debug Mode: false

Server:
ERROR: error during connect: Get "http://docker:2375/v1.24/info": dial tcp: lookup docker on 192.168.65.5:53: no such host

The same error would happen trying to run any other significant docker command like build, run, etc.

An exception to this would be if you configured your GitLab runner to run containers in privileged mode and mount /var/run/docker.sock to all your jobs (this would not be advisable) in which case all your jobs could talk directly to the docker daemon on the host. Another exception might be if you use the shell executor instead and you have docker installed on the host where the runner is running.

How the dind service fixes this

The docker:dind service is a daemon that is created just for your job. This is incredibly important because it can prevent concurrent jobs from stepping on one another or being able to escalate access where they might not otherwise have it.

When the build starts, the GitLab runner will create two containers: your job container and the docker:dind container; they are linked together. When your job invokes docker commands, your job connects to the docker:dind container, which then carries out the requested commands.

Any containers created by your job (say, by invoking docker run or docker build as part of your job) are managed by the daemon running on the docker:dind container, not the host daemon. If you run docker ps inside the job, you'll notice that none of the containers run on the host daemon are listed, despite the fact that if you ran docker ps on the host, you would see the job container, the dind container, and any other running containers.

---

To clarify your other questions:

When we register docker executor gitlab runner, we choose one image

The image specified in your runner configuration is simply the default docker image to be used if a job doesn't declare any image: key. It does not affect how the runner runs in any way.

inside gitlabci, we choose an image under image: or services: fields

When the docker executor runs your job, it uses docker run to do so. The image: key determines which image is used to run your job. Similarly, services: define the image used for service containers -- service containers are siblings to the job container and are connected with links.

So does that mean this GitLab CI job container runs inside the docker executor container?

No. I'd also like to clear up: the runner/executor doesn't run in a container, necessarily. Runners might be installed as a Windows service, or simply even a process running directly on a system. You can use runners that happen to be inside containers, but it doesn't materially affect how jobs are run.

In any case, the containers where your job run are generally always going to be run directly by the host docker daemon.