Should I use filebeat or an app-logging module with lumberjack capabilities to collect my logs

Question

I have a pretty basic elk setup which aggregates logs from around 15 VMs that are running various services.

All of them have configured filebeat syslog forwarder with a single input in logstash.

I have app servers where I run NodeJS web apps, which output JSON logs. In order to make my logs structured I have two options :

Option A

Tell the NodeJS apps to output the logs in a /var/app/app.log rotatable file. Add the app.log to my log propspect in filebeat and push to logstash, where I setup a filter on [source] =~ app.log to parse JSON.

Option B

Tell the NodeJS app to use a module ( e.g. node-bunyan-lumberjack ) which connects independently to logstash and pushes the logs there, without using filebeat.

My question is :

Which one is a more common setup and why?

Answer 1

Both are common and comes with their own pro and cons.

It depends if you can accept loosing logs if logstash is dead and your app restart and on the other hand if you can write logs to disk and consume place there.

Those are the basic points to choose for one or the other, you may end up with more caveats in term of IO/caching/etc.

There's no 'best' option, there's the best option for your organization/teams which we can't answer for you as we don't know your environment.