'Migrate docker-compose nginx.conf to kubernetes nginx ingress
I am trying to convert my docker-compose nginx.conf to kubernetes nginx ingress controller. This is my nginx.conf that i am using with docker-compose.
server {
listen 3000;
root /srv/www/static;
sendfile on;
gzip on;
gzip_comp_level 1;
gzip_proxied any;
gzip_vary on;
gzip_types application/json text/plain;
location /api/main {
rewrite /api/main(.*)$1 break;
proxy_pass http://main:4000;
proxy_set_header content-type "application/json";
}
location /api/login {
rewrite /api/login(.*)$1 break;
proxy_pass http://login:4000;
proxy_set_header content-type "application/json";
}
}
This is the nginx ingress controller. I have the ingress addon enabled.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: test.com
http:
paths:
- path: /api/login(.*)
pathType: Prefix
backend:
service:
name: login-app
port:
number: 4000
- path: /api/main(.*)
pathType: Prefix
backend:
service:
name: main-app
port:
number: 4000
kubectl get ingress:
NAME CLASS HOSTS ADDRESS PORTS AGE
test-ingress <none> test.com 80 20h
kubectl describe ingress test-ingress:
Name: test-ingress
Namespace: default
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
test.com
/api/login(.*) login-app:4000 (10.1.146.185:4000)
/api/main(.*) main-app:4000 (10.1.146.186:4000)
Annotations: nginx.ingress.kubernetes.io/rewrite-target: /$1
Events: <none>
If I go to test.com:80 I get an nginx 404 error so I believe its up and running but any request that I make to an endpoint also returns 404. I also defined test.com in my local machine etc/hostname and I am also using microk8s --classic instead of minikube.
The way I have it set up is I need to be able to send a request to an endpoint and have url path forwarded to the corresponding service. so test.com/api/main/orders/orderStatus would get rewritten to main-service/api/main/orders/orderStatus but this returns a 404 error I think it might be the way I have the rewrite annotations for kubernetes set up because my nginx.conf for docker-compose works fine.
kubectl -n ingress logs nginx-ingress-microk8s-controller-5h5qm:
NGINX Ingress controller
Release: v0.35.0
Build: 54ad65e32bcab32791ab18531a838d1c0f0811ef
Repository: https://github.com/kubernetes/ingress-nginx
nginx version: nginx/1.19.2
-------------------------------------------------------------------------------
I0210 20:46:20.300962 6 flags.go:205] Watching for Ingress class: public
W0210 20:46:20.301239 6 flags.go:208] Only Ingresses with class "public" will be processed by this Ingress controller
W0210 20:46:20.302300 6 flags.go:252] SSL certificate chain completion is disabled (--enable-ssl-chain-completion=false)
W0210 20:46:20.302548 6 client_config.go:552] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I0210 20:46:20.431594 6 main.go:231] Creating API client for https://10.152.183.1:443
I0210 20:46:20.492885 6 main.go:275] Running in Kubernetes cluster version v1.20+ (v1.20.1-34+e7db93d188d0d1) - git (clean) commit e7db93d188d0d12f2fe5336d1b85cdb94cb909d3 - platform linux/amd64
I0210 20:46:20.917856 6 main.go:105] SSL fake certificate created /etc/ingress-controller/ssl/default-fake-certificate.pem
I0210 20:46:21.002278 6 main.go:113] Enabling new Ingress features available since Kubernetes v1.18
I0210 20:46:22.130699 6 nginx.go:263] Starting NGINX Ingress controller
I0210 20:46:22.315064 6 event.go:278] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress", Name:"nginx-ingress-udp-microk8s-conf", UID:"73215902-b0b2-4da2-b402-5aa68d4a5be6", APIVersion:"v1", ResourceVersion:"668", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress/nginx-ingress-udp-microk8s-conf
I0210 20:46:22.315231 6 event.go:278] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress", Name:"nginx-load-balancer-microk8s-conf", UID:"8f8cd494-7d19-4168-8f05-e4992ea3f4de", APIVersion:"v1", ResourceVersion:"666", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress/nginx-load-balancer-microk8s-conf
I0210 20:46:22.401763 6 event.go:278] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress", Name:"nginx-ingress-tcp-microk8s-conf", UID:"885189d0-e5ca-47e2-98f4-8982d813606a", APIVersion:"v1", ResourceVersion:"667", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress/nginx-ingress-tcp-microk8s-conf
I0210 20:46:23.347072 6 store.go:363] ignoring add for ingress test-ingress based on annotation kubernetes.io/ingress.class with value
I0210 20:46:23.431950 6 nginx.go:307] Starting NGINX process
I0210 20:46:23.432052 6 leaderelection.go:242] attempting to acquire leader lease ingress/ingress-controller-leader-public...
I0210 20:46:23.444902 6 controller.go:141] Configuration changes detected, backend reload required.
I0210 20:46:25.899037 6 leaderelection.go:252] successfully acquired lease ingress/ingress-controller-leader-public
I0210 20:46:25.899076 6 status.go:86] new leader elected: nginx-ingress-microk8s-controller-5h5qm
I0210 20:46:26.061674 6 controller.go:157] Backend successfully reloaded.
I0210 20:46:26.061807 6 controller.go:166] Initial sync, sleeping for 1 second.
W0210 20:46:27.063786 6 controller.go:184] Dynamic reconfiguration failed: Post "http://127.0.0.1:10246/configuration/backends": dial tcp 127.0.0.1:10246: connect: connection refused
E0210 20:46:27.063892 6 controller.go:188] Unexpected failure reconfiguring NGINX:
Post "http://127.0.0.1:10246/configuration/backends": dial tcp 127.0.0.1:10246: connect: connection refused
W0210 20:46:27.063941 6 queue.go:130] requeuing initial-sync, err Post "http://127.0.0.1:10246/configuration/backends": dial tcp 127.0.0.1:10246: connect: connection refused
I0210 20:46:27.064158 6 controller.go:141] Configuration changes detected, backend reload required.
I0210 20:46:27.398413 6 controller.go:157] Backend successfully reloaded.
I0210 20:46:27.398500 6 controller.go:166] Initial sync, sleeping for 1 second.
Deployment example mainDep.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: main-app
spec:
replicas: 1
selector:
matchLabels:
app: main-app
template:
metadata:
labels:
app: main-app
spec:
containers:
- name: main-app
image: peking-main:latest
imagePullPolicy: Never
ports:
- containerPort: 4000
resources:
limits:
memory: "100Mi"
cpu: "100m"
Service example mainService.yaml
apiVersion: v1
kind: Service
metadata:
name: main-app
labels:
app: main-app
spec:
type: LoadBalancer
selector:
app: main-app
ports:
- port: 80
targetPort: 4000
kubectl get svc
login-app LoadBalancer 10.152.183.107 <pending> 80:30412/TCP 16m
main-app LoadBalancer 10.152.183.108 <pending> 80:31369/TCP 16m
kubectl describe svc main-app
Name: main-app
Namespace: default
Labels: app=main-app
Annotations: <none>
Selector: app=main-app
Type: LoadBalancer
IP Families: <none>
IP: 10.152.183.108
IPs: 10.152.183.108
Port: <unset> 80/TCP
TargetPort: 4000/TCP
NodePort: <unset> 31369/TCP
Endpoints: 10.1.146.171:4000
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
Solution 1:[1]
So I found a workaround. I changed every port I could find such as application and dockerfile to 80 and I changed the api version of the ingress to v1beta.
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: nginx
spec:
rules:
- host: test.com
http:
paths:
- path: /api/login
backend:
serviceName: login-app
servicePort: 80
- path: /api/main
backend:
serviceName: main-app
servicePort: 80
Although it works this will throw a warning whenever I apply it.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Andrew Messier |