Questions tagged [kafka]

Use this tag for questions about Apache Kafka - a distributed streaming (aka publish and subscribe or pub/sub) platform.

Filter by
Sorted by
Tagged with
1 vote
0 answers
225 views

Kafka - Only one consumer in a consumer group gets messages even the topic has partitions

Here's my setup. I have a topic test_topic with partitions = 4 and replication factor = 1. I have two consumers that are on the same consumer group test_group. My expectation: messages will get ...
Noob_Guy's user avatar
  • 111
0 votes
0 answers
17 views

Python scripts executor

I have multiple python scripts processing events from kafka sequentially. Each script has to be executed on remote machine (not all users have access to). I am looking for a way to automate script ...
user2817340's user avatar
1 vote
1 answer
2k views

EKS - Get availability zone of pod deployment from pod

In my EKS cluster, I have a few node groups that cover availability zone A and B. The pods running on the nodes are consumers/producers on various Kafka topics (MSK). There isn't any internal ...
hi im Bacon's user avatar
1 vote
0 answers
58 views

How does topography affect composite SLA?

I looked at How do you calculate the compound Service Level Agreement (SLA) for cloud services? and am still not sure how topography affects SLA. Suppose a request... passes through one instance of ...
Noobian Noob's user avatar
3 votes
0 answers
422 views

Confluent Kafka helm chart - broker pods not being able to communicate to Zookeper overtime

We are running confluent helm chart on a lot of clusters, we occasionally get brokers pods failing after some time and no self-heal after is following, the error in the broker usually is the following:...
anVzdGFub3RoZXJodW1hbg's user avatar
0 votes
1 answer
234 views

Confluent Kafka on K8s - offset is stuck, lag increases, what can be done?

We are using Confluent Kafka cluster on production K8s. It looks like our consumers are stuck in a loop as offset is static and lag keeps going up. When I restart zookeper/brokers (persistent volume) ...
anVzdGFub3RoZXJodW1hbg's user avatar
0 votes
1 answer
309 views

AWS MSK: deploying all Kafka brokers in the same availability zone

When using AWS MSK, is it possible to deploy Kafka brokers in the same AWS availability zone? From what I read, AWS enforces deployments across availability zones. If you use three brokers, it will ...
Philipp Claßen's user avatar
2 votes
1 answer
457 views

How to add LinkedIn's Burrow to a Docker Compose?

It appears LinkedIn doesn't have an official Burrow docker image on Docker Hub, but there are others who have forked it. However, I can't find any examples of how to add any of them to a docker ...
Marcel Lamothe's user avatar
1 vote
1 answer
37 views

Kafka from client to server vs. "router" in the DC

I want to use Kafka as a message broker on the server side of a distributed application. We have clients that can be located around the world, and that send frequent messages to the server. Right ...
Ken - Enough about Monica's user avatar
2 votes
1 answer
2k views

Multiple Kafka brokers behind a single OpenShift service?

Is it possible to have several brokers behind a single TCP/IP port? I.e., offer a single endpoint for Kafka clients, and having a scalable amount of brokers behind that? In my company, some people ...
AnoE's user avatar
  • 4,876