3

From what I've seen, Prometheus scrapes data and metrics and displays them and Fluent Bit collects logs and metrics and can output them.

So, are they doing the same thing, i.e. should I choose one over the other or am I missing something?

Improve this question

1 Answer 1

Reset to default
2

This is a little off. You are comparing a "Time Series Database" to a "Log Shipper".

How Prometheus Works

Any service/entity prometheus wants to run monitor has to provide an "exporter". An exporter is just a http endpoint that has metrics in a simple text format. The exporter can be a standalone app, a web sever linking to a file, or something like spring boot literally just adding a url to serve prometheus metrics. Lots of other systems can pickup and use this format, not just prometheus.

You configure prometheus to "scrape" these metrics and store them in its own Time Series Database, usually at 30 second intervals. Then prometheus is basically your metrics database. It also integrates with alert manager to help you trigger alerts to slack, pager duty, etc.

How Fluent Bit Works

Fluent Bit is a log shipper. It is better compared with, say, logstash. It can basically take any kind of log data (text lines) and ship/route/filter/etc them to a destination system. It is not itself a time series database like prometheus though. Also prometheus is not a log shipper, it can't go make arbitrary routing decisions on the data being sent to it.

This may be worth a skim - https://logz.io/blog/fluentd-logstash/.

Improve this answer
5
  • So, they work in unison or is having Prometheus enough if I don't have any super-complex logging routing/strategies?
    – Grafana
    Commented Jan 10, 2022 at 22:10
  • Prometheus handles application metrics specifically and is a TSDB. Fluent Bit is a part of a logging solution (like ELK - Elastic Search, LogStash, Kibana), which is generally used to collect, centralize, and visualize application logs (like when you do logger.error("Oh No!");. Fluent Bit would replace LogStash in that stack for routing logs to Elastic Search (kind of a text database) for visualization in Kibana. You could route and analyze metrics in ELK, and you can do alerting, etc. But Prometheus/alert-manager/grafana are really designed for doing that on app metrics and excel at it.
    – John Humphreys
    Commented Jan 11, 2022 at 1:41
  • 1
    For example, we maintain prometheus + alert manager + thanos + cortex + grafana for application metrics, and a separate ELK stack (which may use fluent eventually) for application log analysis. Metrics != logs.
    – John Humphreys
    Commented Jan 11, 2022 at 1:43
  • 1
    Ah, thanks, that cleared it up. I'd vote the answer up if I could.
    – Grafana
    Commented Jan 11, 2022 at 16:26
  • Fluent Bit is also a "Metric Shipper": "Metrics also are unique in Fluent Bit in that metrics don’t go through the same pipelines that logs go through. So metrics are almost treated as an independent type." (calyptia.com/blog/logs-openmetrics-opentelemetry-fluent-bit). There is an experimental plugin attempting to cross this divide within Fluent Bit: Log to Metrics (docs.fluentbit.io/manual/pipeline/filters/log_to_metrics). The following Fluent Bit pipeline could work: Tail -> JSON -> Log to Metrics -> Prometheus Exporter (or Prometheus Remote Write)
    – n1ghtm4n4g3r
    Commented Jul 2, 2023 at 14:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.