Alarms management with Elasticsearch and Kibana

Question

I'm working on my first implementation of the Elasticsearch-Logstash-Kibana stack. I have everything working, but now I need to:

• Set alarms on certain thresholds/events
• Send notifications on alarms
• (desirable) Manage alarms from within Kibana (i.e. alert/acknowledge)

I've read some partial solutions using another tools beside ELK, as Nagios, but I'm wondering what is the industry's "best" approach to this need.

Note: I've already taken a look at X-Pack and seems to be one of the acceptable solutions, but the licencing details and costs are still vague - not publicly available. I've contacted Elastic but need to answer a few questions with data I still don't have.

Answer 1

I have no experience with it myself, but Elastalert (http://elastalert.readthedocs.io/en/latest/elastalert.html) sounds like what you need (for the alarms/threshold part)

But besides that, it sounds like your interested in monitoring. In my opinion ELK is not a classic monitoring tool (more: data collection, data processing and data visualization). You should take a look at Icinga2 (https://www.icinga.com/products/icinga-2/) which can can be configured highly dynamical and is compatible with nagios plugins

Answer 2

You can use emergent AI based alerts by totalcloud. You can customize the thresholds and see a dry run of the alarms too.

Answer 3

Manage alarms through Kibana

That specifically is one of the licensing points for X-Pack, so there won't be much non-X-pack stuff out there that does this.

I've been doing alarms through Logstash config, but it isn't managed through X-Pack. We've had better luck managing logstash output {} blocks through GitHub-style pull-request workflows. It will require knowing how to write logstash code, and some sense of the object model of your events. But if people are fluent in writing Kibana dashboards, they already have that fluency; most of your training will be in scoping the logstash code.