Unable To Access Prometheus Dashboard/ Port Forwarding Doesn't Work

Question

I'm creating a K8 cluster and I want to display the cluster information in Grafana using Prometheus (as usual). I've followed various documentation that has been already posted, but nothing seems to fix the problem.

Prometheus dashboard doesn't load, I'm doing this in an AWS environment. Please find the steps that I'm using.

1. helm install stable prometheus-community/kube-prometheus-stack --namespace prometheus - SUCCESSFUL
2. kubectl edit svc stable-kube-prometheus-sta-prometheus -n prometheus - change to NodePort
3. kubectl edit svc stable-grafana -n prometheus - change to NodePort
4. kubectl port-forward -n monitoring svc/grafana 8000:80

I was able to do the port-forwarding once to the Grafana (first attempt). However, after deleting that deployment due to some complications, i proceeded with the above deployment again.
When it comes to the port-forwarding, i'm not getting the following response

Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

Infact I'm not getting anything. Is this port is already in use ? What am I doing wrong ? Please help.

Answer 1

It looks like a trivial mistake in your case: you picked the wrong namespace for port-forward cmd.

Use the one where you installed the whole release: prometheus

kubectl port-forward -n prometheus svc/grafana 8000:80

Answer 2

For me the following worked but the namespace and port depends on your installation :

kubectl port-forward -n knative-monitoring svc/grafana 3000:30802

If this doesn't work due to a different setup, here is what you can do to figure it out :

1. List all services kubectl get services --all-namespaces
2. Look for a service with the name grafana
3. Note the port of the grafana service shown to the right of the service name (eg: 30802).
4. Note the namespace the grafana service is in, it should be listed near the service name (eg: knative-monitoring).
5. Perform the port-forward :

kubectl port-forward -n YOUR_GRAFANA_NAMESPACE svc/grafana 3000:YOUR_GRAFANA_PORT

• Replace YOUR_GRAFANA_NAMESPACE with the namespace grafana is in (see step 4)

• Replace YOUR_GRAFANA_PORT with the port of grafana (see step 3)

• This will port-forward grafana to your local port 3000 (feel free to change 3000 to whatever free port you want).

6. Access Grafana dashboard http://localhost:3000/grafana/