0

I have a simple Django web application running on AWS EC2 instance (2 vCPU, 4Gi). For that type of deployment I have a performance (measured with Hey)

$ hey -t 30 -z 1m https://mydomain.com/my/endpoint/

about 30 reqs/s

When deployed with Kubernetes onto EKS cluster (same docker image) I have about 15 reqs/s for 1 pod with limits

    resources:
      requests:
        cpu: "500m"
        memory: "2Gi"
      limits:
        cpu: "1"
        memory: "4Gi"

I expect that scaling pods would double the performance but nothing happens (request rate stays the same)

$ kubectl scale deploy/myapp --replicas=2

I'm running Django app behind gunicorn like this (adjusting number of workers did not help)

gunicorn --config gunicorn.py myapp.wsgi:application
# gunicorn.py
import os

# tried different values from 1 to 5
# for 1 vCPU allocated for the pod it should be 3 as per docs
# (2 x $num_cores) + 1
# https://docs.gunicorn.org/en/stable/design.html#how-many-workers
workers = int(os.getenv('GUNICORN_WORKERS_APP', default=4))
daemon = False
bind = '0.0.0.0:8080'
max_requests = 2000

So couple of questions here:

  • how to properly debug this type of issue and what's the toolset I can use?
  • how service does load balancing to pods and is there any overhead introduced?
  • is there an overhead for resource management (cpu allocation/throttling)?
  • how performance tuning is done for k8s-driven applications?
1
  • 1
    An APM like New Relic would be a tool to see the pods load and response times. How services do load balancing is provided here:-kubernetes.io/docs/concepts/services-networking/service/… There could be some overhead but not such that two pods provide the same i/o as one. Could you check if there could be other factors like ingress-controller configurations and such? Jul 7 at 20:56
1

What does 'kubectl top node' return for your node usage?

Kubernetes is pretty low overhead and can certainly handle more than 15req/s. Does your application rely on a database? If so, where is that database living? Is that single node also running any other pods?

As others noted, an APM would be a helpful tool to diagnose unexpected performance issues.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.