Which libraries generate observability metrics for applications in the cluster to prometheus?

Question

I recently read "Cloud Native DevOps with Kubernetes" by O'reily and it mentions the RED and USE metrics which are a good starting point for gaining better observability into the system, however I'm having a hard time finding any information online about how to actually go about generating those metrics outside of re-inventing the wheel for each service.

Is there a set of libraries to actually generate RED and USE metrics within the application? Or do we have to create a set of utilities for each language and each service independently?

Also, are there known solutions for sending CrashLoopBack or Pending pod metrics to Prometheus or do these also need to be reinvented in house?

Answer 1

For Application specific calls you need to use a Prometheus client and create the metrics yourself. The Prometheus client will allow you to create a /metrics endpoint where you can output the current metric values which Prometheus will scrape.

For metrics that are not unique to your application, you can use Prometheus exporters that will run on either the node, container, or runtime environment, or third-party software. These will also create targets which you can tell Prometheus to scrape to gather the metric data.

Once the metric data is collected you'll need to write your own RED and USE queries in graphana to display those metrics.