Sending Exceptions to Elastic Stack via REST

Question

I am working with devs wishing to replace Exceptionless, a Real-time exception reporting tool.

They wish to send the exceptions from a web app that they would previously send to Exceptionless to our ELK stack.

I am only experienced with using Beats to ship data to ELK.

In the past I have written custom code to do similar things like this, but we shipped to Splunk, not ELK.

How friendly is the process of sending information to ELK via REST? Are their tips, tricks and/or pitfalls?

Is this a supported, recommended, and good way to do exception logging/reporting/analytics/etc.

Answer 1

It depends on what you're trying to accomplish but IMO if you're interested in reliably getting your exceptions you probably want something between Elasticsearch and your app - a queueing or logging system of some kind like Kafka or Redis.

Send your exceptions there and then drain that to Elasticsearch. Otherwise you run the risk of overwhelming ES's ability to accept writes at the very time you need it most (cascading failure, DDOS etc).

Answer 2

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html

POST test/_update/1
{
    "script" : {
        "source": "if (ctx._source.tags.contains(params.tag)) { ctx._source.tags.remove(ctx._source.tags.indexOf(params.tag)) }",
        "lang": "painless",
        "params" : {
            "tag" : "blue"
        }
    }
}