3

I have a pretty basic elk setup which aggregates logs from around 15 VMs that are running various services.

All of them have configured filebeat syslog forwarder with a single input in logstash.

I have app servers where I run NodeJS web apps, which output JSON logs. In order to make my logs structured I have two options :

Option A

Tell the NodeJS apps to output the logs in a /var/app/app.log rotatable file. Add the app.log to my log propspect in filebeat and push to logstash, where I setup a filter on [source] =~ app.log to parse JSON.

Option B

Tell the NodeJS app to use a module ( e.g. node-bunyan-lumberjack ) which connects independently to logstash and pushes the logs there, without using filebeat.

My question is :

Which one is a more common setup and why?

1 Answer 1

5

Both are common and comes with their own pro and cons.

It depends if you can accept loosing logs if logstash is dead and your app restart and on the other hand if you can write logs to disk and consume place there.

Those are the basic points to choose for one or the other, you may end up with more caveats in term of IO/caching/etc.

There's no 'best' option, there's the best option for your organization/teams which we can't answer for you as we don't know your environment.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.