ArgoCD & Traefik 2.x: How to configure argocd-server Deployment to run with TLS disabled (where to put --insecure flag)

Question

We have a setup with Traefik as the Ingress Controller / CRD and ArgoCD. We installed ArgoCD into our EKS setup as described in the Argo getting stared guide:

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

Now as the docs state the IngressRoute object to configure Traefik correctly looks like this:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: argocd-server
  namespace: argocd
spec:
  entryPoints:
    - websecure
  routes:
    - kind: Rule
      match: Host(`argocd.tekton-argocd.de`)
      priority: 10
      services:
        - name: argocd-server
          port: 80
    - kind: Rule
      match: Host(`argocd.tekton-argocd.de`) && Headers(`Content-Type`, `application/grpc`)
      priority: 11
      services:
        - name: argocd-server
          port: 80
          scheme: h2c
  tls:
    certResolver: default
    

Right now there's a bug in the docs - so be sure to remove the options: {} in order to let Traefik accept the configuration.

Traefik shows everything is fine in the dashboard:

But if we try to access the ArgoCD dashboard at https://argocd.tekton-argocd.de we get multiple HTTP 307 redirects and can't access the dashboard in the end. You can see the redirects inside the developer tools:

Searching for a solution we already found this issue where the problem is described:

The problem is that by default Argo-CD handles TLS termination itself and always redirects HTTP requests to HTTPS. Combine that with an ingress controller that also handles TLS termination and always communicates with the backend service with HTTP and you get Argo-CD's server always responding with a redirects to HTTPS.

Also the solution is sketched:

So one of the solutions would be to disable HTTPS on Argo-CD, which you can do by using the --insecure flag on argocd-server.

But how can we configure the argocd-server Deployment to add the --insecure flag to the argocd-server command - as it is also stated inside the ArgoCD docs?

Answer 1

0. Why a declarative ArgoCD setup with Kustomize is a great way to configure custom parameters

There are multiple options on how to configure ArgoCD. A great way is to use a declarative approach, which should be the default Kubernetes-style. Skimming the ArgoCD docs there's a additional configuration section where the possible flags of the ConfigMap argocd-cmd-params-cm can be found. The flags are described in argocd-cmd-params-cm.yaml. One of them is the flag server.insecure

## Server properties
# Run server without TLS
server.insecure: "false"

The argocd-server deployment which ships with https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml will use this parameter, if it is defined in the argocd-cmd-params-cm ConfigMap.

In order to declaratively configure the ArgoCD configuration, the ArgoCD docs have a great section on how to do that with Kustomize. In fact the ArgoCD team itself uses this approach to deploy their own ArgoCD instances - a live deployment is available here https://cd.apps.argoproj.io/ and the configuration used can be found on GitHub.

Adopting this to our use case, we need to switch our ArgoCD installation from simply using kubectl apply -f to a Kustomize-based installation. The ArgoCD docs also have a section on how to do this. Here are the brief steps:

1. Create a argocd/installation directory with a new file kustomization.yaml

We slightly enhance the kustomization.yaml proposed in the docs:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - https://raw.githubusercontent.com/argoproj/argo-cd/v2.3.3/manifests/install.yaml

## changes to config maps
patchesStrategicMerge:
  - argocd-cmd-params-cm-patch.yml

namespace: argocd

Since the docs state

It is recommended to include the manifest as a remote resource and apply additional customizations using Kustomize patches.

we use the patchesStrategicMerge configuration key, which contains another new file we need to create called argocd-cmd-params-cm-patch.yml.

2. Create a new file argocd-cmd-params-cm-patch.yml

This new file only contains the configuration we want to change inside the ConfigMap argocd-cmd-params-cm:

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cmd-params-cm
data:
  server.insecure: "true"

3. Install ArgoCD using the Kustomization files & kubectl apply -k

There's a separate kustomize CLI one can install e.g. via brew install kustomize. But as Kustomize is build into kubectl we only have to use kubectl apply -k and point that to our newly created argocd/installation directory like this. We just also need to make sure that the argocd namespace is created:

kubectl create namespace argocd --dry-run=client -o yaml | kubectl apply -f -    
kubectl apply -k argocd/installation

This will install ArgoCD and configure the argocd-server deployment to use the --insecure flag as needed to stop Argo from handling the TLS termination itself and giving that responsibility to Traefik. Now accessing https://argocd.tekton-argocd.de should open the ArgoCD dashboard as expected:

Answer 2

You can use this traefik rule:

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: argocd-server
  namespace: argocd
spec:
  entryPoints:
    - websecure
  routes:
    - kind: Rule
      match: Host(`argocd.example.com`)
      priority: 10
      services:
        - name: argocd-server
          port: 80
    - kind: Rule
      match: Host(`argocd.example.com`) && Headers(`Content-Type`, `application/grpc`)
      priority: 11
      services:
        - name: argocd-server
          port: 80
          scheme: h2c
  tls: {}

It worked for me using traefik 2.6.3.

The troubleshot is at tls: {}.