I want to assign UAMI to kubelet, but it fails due to lack of permissions

Question

I'm trying to assign UAMI to an AKS kubelet using terraform, but I don't have permissions and it fails with the following error.

Error: creating Managed Kubernetes Cluster "ClusterName" (Resource Group "ResourceGroupName"): containerservice.ManagedClustersClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="CustomKubeletIdentityMissingPermissionError" Message="The cluster user assigned identity must be given permission to assign kubelet identity /subscriptions/***/resourceGroups/ResourceGroupName/providers/Microsoft.ManagedIdentity/userAssignedIdentities/UAMI. Check access result not allowed for action Microsoft.ManagedIdentity/userAssignedIdentities/assign/action.

I would like to grant permissions, but the error message does not mention the scope, so I do not know where to assign permissions. In addition, I am using the same UAMI that is currently assigned to the control plane, is there any problem? Thank you for your cooperation.

Answer 1

You need to provide the role Microsoft.ManagedIdentity/userAssignedIdentities/assign/action. As its not directly present in any built-in role definition in Azure , you will have to create a custom role and then assign it to the UAMI to set kublet identity.

I tried the same after receiving the error as below :

Terraform code:

provider"azurerm"{
    features{}
}
provider "random" {}
data "azurerm_subscription" "primary" {
}

data "azurerm_client_config" "example" {
}
data "azurerm_resource_group" "rg" {
      name     = "ansumantest"
}

resource "azurerm_user_assigned_identity" "UAMI" {
  resource_group_name = data.azurerm_resource_group.rg.name
  location            = data.azurerm_resource_group.rg.location
  name = "AKS-MI"
}

resource "random_uuid" "customrole" {}
resource "random_uuid" "roleassignment" {}
resource "azurerm_role_definition" "example" {
  role_definition_id = random_uuid.customrole.result
  name               = "CustomKubeletIdentityPermission"
  scope              = data.azurerm_subscription.primary.id

  permissions {
    actions     = ["Microsoft.ManagedIdentity/userAssignedIdentities/assign/action"]
    not_actions = []
  }

  assignable_scopes = [
    data.azurerm_subscription.primary.id,
  ]
}

resource "azurerm_role_assignment" "example" {
  name               = random_uuid.roleassignment.result
  scope              = data.azurerm_subscription.primary.id
  role_definition_id = azurerm_role_definition.example.role_definition_resource_id
  principal_id       = azurerm_user_assigned_identity.UAMI.principal_id
}
resource "azurerm_user_assigned_identity" "kubletIdentity" {
  resource_group_name = data.azurerm_resource_group.rg.name
  location            = data.azurerm_resource_group.rg.location
  name = "Kublet-MI"
}

resource "azurerm_kubernetes_cluster" "aks" {
  name                = "ansumantestaks"
  location            = data.azurerm_resource_group.rg.location
  resource_group_name = data.azurerm_resource_group.rg.name
  dns_prefix          = "ansumantestaks-dns"

  default_node_pool {
    name                = "system"
    node_count          = 1
    vm_size             = "Standard_B2ms"
    type                = "VirtualMachineScaleSets"
    availability_zones  = [1, 2, 3]
    enable_auto_scaling = false
    }
    identity{
       type = "UserAssigned"
       user_assigned_identity_id = azurerm_user_assigned_identity.UAMI.id
   }
   kubelet_identity {
     client_id = azurerm_user_assigned_identity.kubletIdentity.client_id
     object_id = azurerm_user_assigned_identity.kubletIdentity.principal_id
     user_assigned_identity_id = azurerm_user_assigned_identity.kubletIdentity.id
   }
   depends_on = [
     azurerm_role_assignment.example
   ]
}

Output:

Answer 2

I was able to achieve the same with a built-in role.

data "azurerm_resource_group" "main" {
  name = var.resource_group_name
}

resource "azurerm_user_assigned_identity" "this" {
  location            = data.azurerm_resource_group.main.location
  resource_group_name = data.azurerm_resource_group.main.name
  name                = "${var.cluster_name}-msi"
}

resource "azurerm_role_assignment" "this" {
  scope                = data.azurerm_resource_group.main.id
  role_definition_name = "Managed Identity Operator"
  principal_id         = azurerm_user_assigned_identity.this.principal_id
}

resource "azurerm_kubernetes_cluster" "this" {

  depends_on = [
    azurerm_role_assignment.msi_operator,
  ]

  name                    = var.cluster_name
  kubernetes_version      = var.kubernetes_version
  location                = data.azurerm_resource_group.main.location
  resource_group_name     = data.azurerm_resource_group.main.name
  dns_prefix              = var.prefix
  sku_tier                = var.sku_tier
  private_cluster_enabled = var.private_cluster_enabled

  kubelet_identity {
    user_assigned_identity_id = azurerm_user_assigned_identity.this.id
    client_id = azurerm_user_assigned_identity.this.client_id
    object_id = azurerm_user_assigned_identity.this.principal_id
  }
...
}

Answer 3

It's WRONG to say.... we need to create a Custome Role with "Microsoft.ManagedIdentity/userAssignedIdentities/assign/action. As its not directly present in any built-in role definition in Azure "

• It is present in the built in role "Managed Identity Operator". (Pls check the following screenshot from Azure)
• Most people find it difficult to find the correct scope So start with the subscription scope and it will do the job. (e.g: var.subscription_id)
• To be honest the scope should be set to the id of the Managed Identity itself.

So then the Terraform code should only have following role assignment, when you have the Managed Identity Created Already.

  resource "azurerm_role_assignment" "kubelet_identity" {
  scope                = azurerm_user_assigned_identity.module.id
  role_definition_name = "Managed Identity Operator"
  principal_id         = azurerm_user_assigned_identity.module.principal_id
  }