'kube-proxy fails with CrashLoopBackOff in minikube
I am running k8s using minikube version v1.18.0 on Ubuntu 20. But kube-proxy fails with CrashLoopBackOff status. What could be the issue? I am using 1.18 version for k8s server and client with minikube. I tried reinstalling the cluster but the issue is persistent. logs are attached for get pod and describe pod.
/home/ravi> kubectl get pods --all-namespaces -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-system coredns-66bff467f8-gtsl7 0/1 Running 0 10m 172.17.0.2 minikube <none> <none>
kube-system etcd-minikube 1/1 Running 0 10m 192.168.49.2 minikube <none> <none>
kube-system kube-apiserver-minikube 1/1 Running 0 10m 192.168.49.2 minikube <none> <none>
kube-system kube-controller-manager-minikube 1/1 Running 0 10m 192.168.49.2 minikube <none> <none>
kube-system kube-proxy-d5dqf 0/1 CrashLoopBackOff 6 10m 192.168.49.2 minikube <none> <none>
kube-system kube-scheduler-minikube 1/1 Running 0 10m 192.168.49.2 minikube <none> <none>
kube-system storage-provisioner 0/1 CrashLoopBackOff 6 11m 192.168.49.2 minikube <none> <none>
/home/ravi>
/home/ravi>
/home/ravi>kubectl describe pod -n kube-system kube-proxy-d5dqf
Name: kube-proxy-d5dqf
Namespace: kube-system
Priority: 2000001000
Priority Class Name: system-node-critical
Node: minikube/192.168.49.2
Start Time: Fri, 13 May 2022 14:08:26 +0530
Labels: controller-revision-hash=5bdc57b48f
k8s-app=kube-proxy
pod-template-generation=1
Annotations: <none>
Status: Running
IP: 192.168.49.2
IPs:
IP: 192.168.49.2
Controlled By: DaemonSet/kube-proxy
Containers:
kube-proxy:
Container ID: docker://dc1554848405254d1ba463fc4ae2ec98fb7e2db1472d3b143b71256dcb7812c3
Image: k8s.gcr.io/kube-proxy:v1.18.20
Image ID: docker://sha256:27f8b8d51985f755cfb3ffea424fa34865cc0da63e99378d8202f923c3c5a8ba
Port: <none>
Host Port: <none>
Command:
/usr/local/bin/kube-proxy
--config=/var/lib/kube-proxy/config.conf
--hostname-override=$(NODE_NAME)
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Fri, 13 May 2022 14:19:30 +0530
Finished: Fri, 13 May 2022 14:19:30 +0530
Ready: False
Restart Count: 7
Environment:
NODE_NAME: (v1:spec.nodeName)
Mounts:
/lib/modules from lib-modules (ro)
/run/xtables.lock from xtables-lock (rw)
/var/lib/kube-proxy from kube-proxy (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-proxy-token-z2nb4 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
kube-proxy:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: kube-proxy
Optional: false
xtables-lock:
Type: HostPath (bare host directory volume)
Path: /run/xtables.lock
HostPathType: FileOrCreate
lib-modules:
Type: HostPath (bare host directory volume)
Path: /lib/modules
HostPathType:
kube-proxy-token-z2nb4:
Type: Secret (a volume populated by a Secret)
SecretName: kube-proxy-token-z2nb4
Optional: false
QoS Class: BestEffort
Node-Selectors: kubernetes.io/os=linux
Tolerations:
CriticalAddonsOnly
node.kubernetes.io/disk-pressure:NoSchedule
node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/network-unavailable:NoSchedule
node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/pid-pressure:NoSchedule
node.kubernetes.io/unreachable:NoExecute
node.kubernetes.io/unschedulable:NoSchedule
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 15m default-scheduler Successfully assigned kube-system/kube-proxy-d5dqf to minikube
Normal Pulled 13m (x5 over 15m) kubelet, minikube Container image "k8s.gcr.io/kube-proxy:v1.18.20" already present on machine
Normal Created 13m (x5 over 15m) kubelet, minikube Created container kube-proxy
Normal Started 13m (x5 over 15m) kubelet, minikube Started container kube-proxy
Warning BackOff 5m22s (x49 over 15m) kubelet, minikube Back-off restarting failed container
/home/ravi>
/home/ravi>kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.0", GitCommit:"9e991415386e4cf155a24b1da15becaa390438d8", GitTreeState:"clean", BuildDate:"2020-03-25T14:58:59Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.20", GitCommit:"1f3e19b7beb1cc0110255668c4238ed63dadb7ad", GitTreeState:"clean", BuildDate:"2021-06-16T12:51:17Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
/home/ravi>
/home/ravi/>kubectl logs -n kube-system kube-proxy-d5dqf
W0513 10:01:01.652298 1 server_others.go:559] Unknown proxy mode "", assuming iptables proxy
I0513 10:01:01.673427 1 node.go:136] Successfully retrieved node IP: 192.168.49.2
I0513 10:01:01.673482 1 server_others.go:186] Using iptables Proxier.
I0513 10:01:01.673968 1 server.go:583] Version: v1.18.20
I0513 10:01:01.674955 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_max' to 131072
F0513 10:01:01.675040 1 server.go:497] open /proc/sys/net/netfilter/nf_conntrack_max: permission denied
/home/ravi>
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|