Questions tagged [kafka]
Use this tag for questions about Apache Kafka - a distributed streaming (aka publish and subscribe or pub/sub) platform.
11
questions
0
votes
0
answers
89
views
Kafka - Only one consumer in a consumer group gets messages even the topic has partitions
Here's my setup. I have a topic test_topic with partitions = 4 and replication factor = 1. I have two consumers that are on the same consumer group test_group.
My expectation: messages will get ...
- 101
0
votes
0
answers
17
views
Python scripts executor
I have multiple python scripts processing events from kafka sequentially. Each script has to be executed on remote machine (not all users have access to). I am looking for a way to automate script ...
- 11
0
votes
0
answers
12
views
How to pass data from one mongodb cluster to another upon changes
The project I'm working on has multiple mongodb clusters for each module of the project.
There is one main module that contains data that every other module requires to function. So far the current ...
1
vote
1
answer
2k
views
EKS - Get availability zone of pod deployment from pod
In my EKS cluster, I have a few node groups that cover availability zone A and B. The pods running on the nodes are consumers/producers on various Kafka topics (MSK). There isn't any internal ...
- 121
1
vote
0
answers
52
views
How does topography affect composite SLA?
I looked at How do you calculate the compound Service Level Agreement (SLA) for cloud services? and am still not sure how topography affects SLA. Suppose a request...
passes through one instance of ...
- 11
3
votes
0
answers
414
views
Confluent Kafka helm chart - broker pods not being able to communicate to Zookeper overtime
We are running confluent helm chart on a lot of clusters, we occasionally get brokers pods failing after some time and no self-heal after is following, the error in the broker usually is the following:...
0
votes
1
answer
218
views
Confluent Kafka on K8s - offset is stuck, lag increases, what can be done?
We are using Confluent Kafka cluster on production K8s.
It looks like our consumers are stuck in a loop as offset is static and lag keeps going up.
When I restart zookeper/brokers (persistent volume) ...
0
votes
1
answer
257
views
AWS MSK: deploying all Kafka brokers in the same availability zone
When using AWS MSK, is it possible to deploy Kafka brokers in the same AWS availability zone?
From what I read, AWS enforces deployments across availability zones. If you use three brokers, it will ...
- 1,675
2
votes
1
answer
450
views
How to add LinkedIn's Burrow to a Docker Compose?
It appears LinkedIn doesn't have an official Burrow docker image on Docker Hub, but there are others who have forked it.
However, I can't find any examples of how to add any of them to a docker ...
- 141
1
vote
1
answer
37
views
Kafka from client to server vs. "router" in the DC
I want to use Kafka as a message broker on the server side of a distributed application. We have clients that can be located around the world, and that send frequent messages to the server. Right ...
2
votes
1
answer
2k
views
Multiple Kafka brokers behind a single OpenShift service?
Is it possible to have several brokers behind a single TCP/IP port?
I.e., offer a single endpoint for Kafka clients, and having a scalable amount of brokers behind that?
In my company, some people ...
- 4,866