What is Unit testing in the context of YML configuration Alerting rules?

Question

I read a very short blog posting about Unit testing alerts with Prometheus

Can you explain me why I need unit testing for *.yml? What is the benefit of unit testing for configuration files?

I not understand what are the gains when implementing that.

Answer 1

The important part of the blog post you link to isn't testing the syntax of the YAML file; it's testing the semantics of the Prometheus configuration.

The core configuration in that post includes a fragment:

- alert: MyAlert
  expr: avg without(instance)(up) < 0.75
  for: 2m
  labels:
    severity: page

How do you know this is the right configuration? If it's misconfigured then either your on-call team gets paged when the system is healthy, or it doesn't get paged when the system is down; both are bad.

The test setup shown in the blog post would let you test your Prometheus alerting setup, with artificial sample data (the "bar" instance is up for 5 minutes, then down for 5 minutes, and it's expected to page after it's been down for longer than 2 minutes), before you deploy it for real and without actually paging anyone. That gives you some confidence your monitoring setup will do what you expect, before you deploy it for real.