Questions tagged [kafka]
Use this tag for questions about Apache Kafka - a distributed streaming (aka publish and subscribe or pub/sub) platform.
8 questions
3
votes
0
answers
455
views
Confluent Kafka helm chart - broker pods not being able to communicate to Zookeper overtime
We are running confluent helm chart on a lot of clusters, we occasionally get brokers pods failing after some time and no self-heal after is following, the error in the broker usually is the following:...
2
votes
1
answer
469
views
How to add LinkedIn's Burrow to a Docker Compose?
It appears LinkedIn doesn't have an official Burrow docker image on Docker Hub, but there are others who have forked it.
However, I can't find any examples of how to add any of them to a docker ...
- 141
2
votes
1
answer
2k
views
Multiple Kafka brokers behind a single OpenShift service?
Is it possible to have several brokers behind a single TCP/IP port?
I.e., offer a single endpoint for Kafka clients, and having a scalable amount of brokers behind that?
In my company, some people ...
- 4,906
1
vote
1
answer
3k
views
EKS - Get availability zone of pod deployment from pod
In my EKS cluster, I have a few node groups that cover availability zone A and B. The pods running on the nodes are consumers/producers on various Kafka topics (MSK). There isn't any internal ...
- 121
1
vote
1
answer
44
views
Kafka from client to server vs. "router" in the DC
I want to use Kafka as a message broker on the server side of a distributed application. We have clients that can be located around the world, and that send frequent messages to the server. Right ...
0
votes
0
answers
4
views
Proper way to horizontally scale Kafka 3 KRaft cluster
I am working with a Kafka 3.6.1 cluster (KRaft mode enabled) and would like some guidance on scaling Kafka brokers and controllers. Below are the details of my setup and the steps I followed, along ...
- 101
0
votes
1
answer
253
views
Confluent Kafka on K8s - offset is stuck, lag increases, what can be done?
We are using Confluent Kafka cluster on production K8s.
It looks like our consumers are stuck in a loop as offset is static and lag keeps going up.
When I restart zookeper/brokers (persistent volume) ...
0
votes
1
answer
393
views
AWS MSK: deploying all Kafka brokers in the same availability zone
When using AWS MSK, is it possible to deploy Kafka brokers in the same AWS availability zone?
From what I read, AWS enforces deployments across availability zones. If you use three brokers, it will ...
- 1,675