1

As part of my architecture, I want to create a deployment/service on my kubernetes cluster which reads message queues and open dockerized subscribers and then subsequently relay message to the original subscriber (if its an RPC pattern).

I was wondering what would be the best workflow to ensure minimal latency (esp since some workflows are RPC).

  1. I can create kubernetes JOBs but that would mean i read logs for the output of pods which would be relayed and then subsequently delete the job . But I worry this might be an overkill
  2. I can volume share the docker socket and my service can then run my images using docker client. But I am not if thats the right way to do so given that i would like to have proper tracking/versioning of my docker images esp since nodes on clusters are disposable
  3. Are there existing frameworks to handle my situation.
1

After reading your Q&A I think that you are looking for a message broker like kafka, rabbitmq or Amazon MQ. Please let me know whether this helps.

| improve this answer | |
1

For K8s the go to solution for me was https://www.confluent.io/ helm chart. I think they cover your needs, you can read the technical doc for more.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.