Gitlab Runner Image with GCP credentials

Question

I am trying to teach my Gitlab Runner image to get custom builder images from my private Docker Registry (GCR running in the Google Cloud).

What did not work out? I created a custom Gitlab Runner image with the ServiceAccount properly set. I started in in non-privileged mode but the wormhole pattern (via docker.sock). On exec-ing into that container (which is based on gitlab/gitlab-runner:v11.3.0) I had to recognise that I cannot do any docker commands in there (neither as root nor as gitlab-user). How the gitlab-runner starts the builder containers afterwards is way above my cognitive capabilities. ;)

# got started via eu.gcr.io/my-project/gitlab-runner:0.0.5 which got taught the GCR credentials

stages:
    - build

build:
    image: docker pull eu.gcr.io/my-project/gitlab-builder-docker:0.0.2
    stage: build
    script: 
        # only for test if I have access to private docker registry
        - docker pull eu.gcr.io/my-project/gitlab-builder-docker:0.0.1 

What worked out? According to this tutorial you can authenticate via in a before_script block in your .gitlab-ci.yml files. That worked out.

# got started via gitlab/gitlab-runner:v11.3.0 

stages:
    - build

before_script:
    - apk add --update curl python which bash
    - curl -sSL https://sdk.cloud.google.com | bash
    - export PATH="$PATH:/root/google-cloud-sdk/bin"
    - gcloud components install docker-credential-gcr
    - gcloud auth activate-service-account --key-file=/key.json
    - gcloud auth configure-docker --quiet

build:
    image: docker:18.03.1-ce
    stage: build
        # only for test if I have access to private docker registry
        - docker pull eu.gcr.io/my-project/gitlab-builder-docker:0.0.1 

The Question This means that I have to do this (install gcloud & authenticate) in each build run - I would prefer to have done this in the gitlab-runner image. Do you have an idea how to achieve this?

Answer 1

Finally I found a way to get this done.

Teach the vanilla gitlab-runner how to pull from your private GCR Docker Repo

GCP

1. Create Service account with no permissions in IAM & Admin
2. Download Json Key
3. Add Permissions in Storage Browser
   1. Select bucket holding your images (eg eu.artifacts.my-project.appspot.com)
   2. Grant permission Storage Object Admin to the service account

Local Docker Container

1. Launch a library/docker container and exec into it (with Docker Wormhole Pattern docker.sock volume mount)
2. Login into GCR via (Check the url of your repo, in my case its located in Europe, therefore the eu prefix in the url) docker login -u _json_key --password-stdin https://eu.gcr.io < /etc/gitlab-runner/<MY_KEY>.json
3. Verify if it works via some docker pull <MY_GCR_IMAGE>
4. Copy the content of ~/.docker/config.json

Gitlab config.toml configuration

1. Add the following into your config.toml file [[runners]] environment = ["DOCKER_AUTH_CONFIG={ \"auths\": { \"myregistryurl.com:port\": { \"auth\": \"<TOKEN-FROM-DOCKER-CONFIG-FILE>\" } } }"]

Vanilla Gitlab Runner Container

1. Run the runner eg like this docker run -it \ --name gitlab-runner \ --rm \ -v <FOLDER-CONTAININNG-GITLAB-RUNNER-CONFIG-FILE>:/etc/gitlab-runner:ro \ -v /var/run/docker.sock:/var/run/docker.sock \ gitlab/gitlab-runner:v11.3.0

Your .gitlab-ci.yml file

1. Verify the done work via a .gitlab-ci.yml
   1. Use an image which is located in your private GCP Container Registry

Teach your builder images how to push to your private GCR Docker Repo

GCP

1. Add permissions to your service account
   1. Grant permission Storage Legacy Bucket Reader to your service account in the Storage Browser

Custom Docker Builder Image

1. Add your Service Account key file to your your custom image FROM docker:18.03.1-ce ADD key.json /<MY_KEY>.json

Your .gitlab-ci.yml file

1. Add the following script into your before_script section docker login -u _json_key --password-stdin https://eu.gcr.io < /key.json

Final Thoughts

Now the vanilla gitlab-runner can pull your custom images from your private GCR Docker Repo. Furthermore those pullable custom images are also capable of talking to your private GCR Docker Repo and eg push the resulting images of your build pipeline.

That was quite complicated stuff. Maybe Gitlab enhances the support for this usecase in the future.

Answer 2

have you tried to use google cloudbuild? i had the same problem and solved it like this:

echo ${GCR_AUTH_KEY} > key.json
gcloud auth activate-service-account --key-file key.json
gcloud auth configure-docker

gcloud builds submit . --config=cloudbuild.yaml --substitutions _CI_PROJECT_NAME=$CI_PROJECT_NAME,_CI_COMMIT_TAG=${CI_COMMIT_TAG},_CI_PROJECT_NAMESPACE=${CI_PROJECT_NAMESPACE}

cloudbuild.yaml:

steps:
 - name: gcr.io/cloud-builders/docker
    id: builder
    args: 
      - 'build'
      - '-t'
      - 'eu.gcr.io/projectID/$_CI_PROJECT_NAMESPACE-$_CI_PROJECT_NAME:$_CI_COMMIT_TAG'
      - '.'
 - name: gcr.io/cloud-builders/docker
    id: tag-runner-image
    args: 
      - 'tag'
      - 'eu.gcr.io/projectID/$_CI_PROJECT_NAMESPACE-$_CI_PROJECT_NAME:$_CI_COMMIT_TAG'
      - 'eu.gcr.io/projectID/$_CI_PROJECT_NAMESPACE-$_CI_PROJECT_NAME:latest'
images:
 - 'eu.gcr.io/projectID/$_CI_PROJECT_NAMESPACE-$_CI_PROJECT_NAME:$_CI_COMMIT_TAG'
 - 'eu.gcr.io/projectID/$_CI_PROJECT_NAMESPACE-$_CI_PROJECT_NAME:latest'

just use google/cloud-sdk:alpine as image in the gitlab-ci stage

Answer 3

I also finally found a way to do this. Please check this answer. It's nearly the same question, but doesn't allow any keys to be stored within the image.

Answer 4

This example config worked for me in values.yaml:

config: |
[[runners]]
  [runners.docker]
    image = "google/cloud-sdk:alpine"
  [runners.kubernetes]
    namespace = "{{.Release.Namespace}}"
    image = "google/cloud-sdk:alpine"
    [runners.cache]
      Type = "gcs"
      Path = "runner"
      Shared = true
      [runners.cache.gcs]
        BucketName = "runners-cache"
    [[runners.kubernetes.volumes.secret]]
      name = "service-account-credentials"
      mount_path = "keys"
      read_only = true

Where service-account-credentials is a secret containing credentials.json

then in .gitlab-ci.yml you can do:

gcloud auth activate-service-account --key-file=/keys/credentials.json

Hope it helps