Prometheus - join series based on the key part of a label

Question

In 1.16 k8s deprecated the label kubernetes.io/role: worker I had previously been using as a filter to target graphs to specific node role using this in my grafana variables declaration

label_values(kube_node_labels, label_kubernetes_io_role) 
#join series based on the the label value
  sum(kube_pod_container_resource_requests_cpu_cores
  * on (node) group_left (label_kubernetes_io_role) kube_node_labels{label_kubernetes_io_role="$node_role"}

In this scary new world of using the label node-role.kubernetes.io/worker: "" this no longer works, nor can I necessarily regex it when the part I care about is in the key.

Is there a new way anybody knows of to filter queries by arbitrary labels on the node

Answer 1

label_values(kube_node_role, role)

sum(kube_pod_container_resource_requests_cpu_cores
  * on (node) group_left (role) kube_node_role{role="$node_role"})

kube_node_role out of kube-state-metrics can give you a relationship to join