How to easily switch gcloud / kubectl credentials

Question

At work we use Kubernetes hosted in GCP. I also have a side project hosted in my personal GCP account using Google App Engine (deploy using gcloud app deploy).

Often when I try to run a command such as kubectl logs -f service-name, I get an error like "Error from server (Forbidden): pods is forbidden: User "[email protected]" cannot list resource "pods" in API group "" in the namespace "WORK_NAMESPACE": Required "container.pods.list" permission." and then I have to fight with kubectl for hours trying to get it to work.

Can somebody please break it down for a slow person like me, how gcloud and kubectl work together, and how I can easily switch accounts so I can use gcloud commands for my personal projects and kubectl commands for my work projects? I'm happy to nuke my whole config and start from scratch if that's what it takes. I've found various kubectl and gcloud documentation but it doesn't make much sense or talks in circles.

Edit: this is on Linux.

Answer 1

Had the same problem and doing all of the:

gcloud auth login
gcloud auth list
gcloud config set account
gcloud projects list

didn't help. I knew gcloud switched fine as I was able to list other resources with it directly. But it seems kubectl can't pick those changes up automatically, as kubectl/gcloud integration relies on the pre-generated key, which has a 1h expiration(not sure if it's a default but it's what it is on my machine right now). So, on top of setting right user/project/account with gcloud, you should re-generate the creds:

gcloud container clusters get-credentials <my-cluster> --zone <clusters-zone>

Answer 2

I'm in the same boat as you - apps deployed in GKE for work and personal projects deployed in my personal GCP account.

gcloud stores a list of logged in accounts that you can switch between to communicate with associated projects. Take a look at these commands:

gcloud auth login
gcloud auth list
gcloud config set account
gcloud projects list

To work with a specific project under one of your accounts you need to set that configuration via gcloud config set project PROJECT_ID

kubectl has a list of "contexts" on your local machine in ~/.kube/config. Your current context is the cluster you want to run commands against - similar to the active account/project in gcloud.

Unlike gcloud these are cluster specific and store info on cluster endpoint, default namespaces, the current context, etc. You can have contexts from GCP, AWS, on-prem...anywhere you have a cluster. We have different clusters for dev, qa, and prod (thus different contexts) and switch between them a ton. Take a look at the [kubectx project][1] https://github.com/ahmetb/kubectx for an easier way to switch between contexts and namespaces.

kubectl will use the keys from whatever GCP account you are logged in with against the cluster that is set as your current context. i.e., from your error above, if your active account for gcloud is your personal but try to list pods from a cluster at work you will get an error. You either need to set the active account/project for gcloud to your work email or change the kubectl context to a cluster that is hosted in your personal GCP account/project.

Answer 3

For me updating the ~/.kube/config and setting the expiry to a date in past fixes it