Terraform fails to create ingress (could not find the requested resource ingresses.extensions)

Question

I'm using minikube locally. The following is the .tf file I use to create my kubernetes cluster:

provider "kubernetes" {
  config_path = "~/.kube/config"
}

resource "kubernetes_namespace" "tfs" {
  metadata {
    name = "tfs" # terraform-sandbox
  }
}

resource "kubernetes_deployment" "golang_webapp" {
  metadata {
    name      = "golang-webapp"
    namespace = "tfs"
    labels = {
      app = "webapp"
    }
  }
  spec {
    replicas = 3
    selector {
      match_labels = {
        app = "webapp"
      }
    }
    template {
      metadata {
        labels = {
          app = "webapp"
        }
      }
      spec {
        container {
          image             = "golang-docker-example"
          name              = "golang-webapp"
          image_pull_policy = "Never" # this is set so that kuberenetes wont try to download the image but use the localy built one
          liveness_probe {
            http_get {
              path = "/"
              port = 8080
            }
            initial_delay_seconds = 15
            period_seconds        = 15
          }

          readiness_probe {
            http_get {
              path = "/"
              port = 8080
            }
            initial_delay_seconds = 3
            period_seconds        = 3
          }
        }
      }
    }
  }
}


resource "kubernetes_service" "golang_webapp" {
  metadata {
    name      = "golang-webapp"
    namespace = "tfs"
    labels = {
      app = "webapp_ingress"
    }
  }
  spec {
    selector = {
      app = kubernetes_deployment.golang_webapp.metadata.0.labels.app
    }
    port {
      port        = 8080
      target_port = 8080
      protocol    = "TCP"
    }
    # type = "ClusterIP"
    type = "NodePort"
  }
}

resource "kubernetes_ingress" "main_ingress" {
  metadata {
    name      = "main-ingress"
    namespace = "tfs"
  }

  spec {
    rule {
      http {
        path {
          backend {
            service_name = "golang-webapp"
            service_port = 8080
          }
          path = "/golang-webapp"
        }
      }
    }
  }
}

When executing terraform apply, I am successfully able to create all of the resources except for the ingress.

The error is:

Error: Failed to create Ingress 'tfs/main-ingress' because: the server could not find the requested resource (post ingresses.extensions)

with kubernetes_ingress.main_ingress,
   on main.tf line 86, in resource "kubernetes_ingress" "main_ingress":
   86: resource "kubernetes_ingress" "main_ingress" {

When I try to create an ingress service with kubectl using the same configuration as the one above (only in .yaml and using the kubectl apply command) it works, so it seems that kubectl & minikube are able to create this type of ingress, but terraform cant for some reason...

Thanks in advance for any help!

Edit 1:

adding the .yaml that I'm able to create the ingress with

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  namespace: tfs
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: golang-webapp
                port:
                  number: 8080

Answer 1

I think the issue can be related to the ingress classname. May be you need to explicitely provide it in your .tf:

metadata {
    name = "example"
    annotations = {
      "kubernetes.io/ingress.class" = "nginx or your classname"
    }

Or may be it's ingresses.extensions that does not exist in your cluster. Can you provide the .yaml that executed correctly ?

Answer 2

The kubernetes_ingress resource generate an ingress with an apiVersion which is not supported by your kubernetes cluster. You have to use [kubernetes_ingress_v1][1] resource which looks similar to kubernetes_ingress resource with some diferences. For your example, it will be like this :

resource "kubernetes_ingress_v1" "jenkins-ingress" {
   metadata {
      name        = "example-ingress"
      namespace   = "tfs"
      annotations = {
        "nginx.ingress.kubernetes.io/rewrite-target" = "/$1"
      }
   }
   spec {
      rule {
        http {
         path {
           path = "/"
           backend {
             service {
               name = "golang-webapp"
               port {
                 number = 8080
               }
             }
           }
        }
      }
    }
  }
}

Answer 3

Something like this should help using kubernetes_ingress_v1

locals{
  ingress_rules = [
    {
      service_path = "/"
      service_name = "golang-webapp"
      service_port = 8080
    }
}

resource "kubernetes_ingress_v1" "jenkins-ingress" {

  metadata {
    annotations = var.ingress_annotations
    name        = "example-ingress"
    namespace   = "tfs"
    labels      = var.labels
  }
  spec {
    ingress_class_name = var.ingress_class_name
    rule {
      http {
        dynamic "path" {
          for_each = local.ingress_rules
          content {
            backend {
              service {
                name = path.value.service_name
                port {
                  number = path.value.service_port
                }
              }
            }
            path = path.value.service_path
          }
        }
      }
    }
    tls {
      secret_name = "tls-secret"
    }
  }
}