I have a simple Django web application running on AWS EC2 instance (2 vCPU, 4Gi). For that type of deployment I have a performance (measured with Hey)
$ hey -t 30 -z 1m https://mydomain.com/my/endpoint/
about 30 reqs/s
When deployed with Kubernetes onto EKS cluster (same docker image) I have about 15 reqs/s
for 1 pod with limits
resources:
requests:
cpu: "500m"
memory: "2Gi"
limits:
cpu: "1"
memory: "4Gi"
I expect that scaling pods would double the performance but nothing happens (request rate stays the same)
$ kubectl scale deploy/myapp --replicas=2
I'm running Django app behind gunicorn
like this (adjusting number of workers did not help)
gunicorn --config gunicorn.py myapp.wsgi:application
# gunicorn.py
import os
# tried different values from 1 to 5
# for 1 vCPU allocated for the pod it should be 3 as per docs
# (2 x $num_cores) + 1
# https://docs.gunicorn.org/en/stable/design.html#how-many-workers
workers = int(os.getenv('GUNICORN_WORKERS_APP', default=4))
daemon = False
bind = '0.0.0.0:8080'
max_requests = 2000
So couple of questions here:
- how to properly debug this type of issue and what's the toolset I can use?
- how
service
does load balancing to pods and is there any overhead introduced? - is there an overhead for resource management (cpu allocation/throttling)?
- how performance tuning is done for k8s-driven applications?