'What kubernetes permissions does GitLab runner kubernetes executor need?

I've installed GitLab runner on a kubernetes cluster under a namespace gitlab-runner. Like so

# cat <<EOF | kubectl create -f -
{
  "apiVersion": "v1",
  "kind": "Namespace",
  "metadata": {
    "name": "gitlab-runner",
    "labels": {
      "name": "gitlab-runner"
    }
  }
}

# helm repo add gitlab https://charts.gitlab.io
# cat <<EOF|helm install --namespace gitlab-runner gitlab-runner -f - gitlab/gitlab-runner
gitlabUrl: https://gitlab.mycompany.com
runnerRegistrationToken: "c................Z"

The GitLab runner properly registers with the GitLab project but all jobs fail.

A quick look into the GitLab runner logs tells me that the service account used by the GitLab runner lack the proper permissions:

# kubectl logs --namespace gitlabrunner gitlab-runner-gitlab-runner-xxxxxxxxx
ERROR: Job failed (system failure): pods is forbidden: User "system:serviceaccount:gitlabrunner:default" cannot create resource "pods" in API group "" in the namespace "gitlab-runner"  duration=42.095493ms job=37482 project=yyy runner=xxxxxxx

What permission does the gitlab runner kubernetes executor need?

kubernetesgitlab-cigitlab-ci-runner


Solution 1:[1]

I couldn't find in the GitLab runner documentation a list of permissions but I try adding permissions one by one and I compiled a list of the permission required for basic functioning.

The gitlab runner will use the service account system:serviceaccount:gitlab-runner:default so we need to create a role and assign that role to that service account.

# cat <<EOF | kubectl create -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: gitlab-runner
  namespace: gitlab-runner
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["list", "get", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["create"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get"]

# kubectl create rolebinding --namespace=gitlab-runner gitlab-runner-binding --role=gitlab-runne r --serviceaccount=gitlab-runner:default

With that role assigned to the service account, GitLab runner will be able to create, execute and delete the pod and also access the logs.

Solution 2:[2]

Unfortunately I couldn't find this in the official docs either just like @RubenLaguna stated. However, the default values.yaml of the kubernetes gitlab runner helm chart lets you define these RBAC rules nicely and does list some examples which I started with.

In my case I had to add a few and went with the following:

rbac:
  create: true
  rules:
    - apiGroups: [""]
      resources: ["pods", "secrets", "configmaps"]
      verbs: ["get", "list", "watch", "create", "patch", "delete", "update"]
    - apiGroups: [""]
      resources: ["pods/exec", "pods/attach"]
      verbs: ["create", "patch", "delete"]
    - apiGroups: [""]
      resources: ["pods/log"]
      verbs: ["get"]

Solution 3:[3]

If you installed gitlab runners using helm chart then gitlab runners does not use the service account created by the helm chart instead it uses the default service account. There is a bug related to this. https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/353 To work around this problem we created a ClusterRoleBinding as below.

kind: ClusterRoleBinding
metadata:
  name: gitlab-runner-role-default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: <helm-created-cluster-role-here>
subjects:
- kind: ServiceAccount
  name: default
  namespace: gitlab-runner

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 RubenLaguna
Solution 2 MoRe
Solution 3 armourbear

Related Questions