Prometheus query in Grafana with query variable

Question

My Grafana panel query is

sum(kube_pod_container_resource_limits_cpu_cores{node=~"$workers"}) / sum(kube_node_status_allocatable_cpu_cores{node=~"$workers"})

The variable of "workers" is defined as a Prometheus query variable

label_values(kube_node_role{role="worker"}, node)

This query will return a list of nodes that have the label of role="worker".

The question is because there might be a situation where one of the nodes doesn't contain any pod that defines CPU limits, so the metric of kube_pod_container_resource_limits_cpu_cores won't contain that host.

When using {node=~"$workers"} in the panel, the whole query will return N/A because sum(kube_pod_container_resource_limits_cpu_cores{node=~"$workers"}) return N/A.

I found out that the reason is when referring the variable $workers, it only pick up the first host, and coincidentally the first host doesn't have the metric of kube_pod_container_resource_limits_cpu_cores, so in the panel, it shows N/A.

How can I fix it or improve the query to get only workers CPU limits committment? Anything wrong to my set up?

PS: I try to define a new constant variable named with hosts that has all the nodes which return by the query of label_values(kube_node_role{role="worker"}, node), but manually combine them with |(the constant variable looks like hostA|hostB|hostC). By adjusting the query with using this variable sum(kube_pod_container_resource_limits_cpu_cores{node=~"$hosts"}) / sum(kube_node_status_allocatable_cpu_cores{node=~"$hosts"}), it could display correctly! Could it might be the Grafana inside mechannism problem that when using multi values from a query variable?

Debug screenshots
The first query only contains one host.

Answer 1

In Dashboard setting, choose the variable options as

Hide               = empty
Multi-value        = enable
Include All option = enable

Then select "all" from the dashboard label, so the query "node=~${workers}" could select all nodes that are filtered out by the query variable.

Answer 2

Here is a hack:

sum(kube_pod_container_resource_limits_cpu_cores{node=~"$workers"} or kube_node_status_allocatable_cpu_cores{node=~"$workers"}*0) / sum(kube_node_status_allocatable_cpu_cores{node=~"$workers"})

you can easily join node label values in kube_node_status_allocatable_cpu_cores by doing sum(a or b*0). By doing so, when you do query with host doesn't exist in a but exist in b, you will get a 0

An additional screenshoot to clear my point.