Consider this scenario:

• 1 worker node with 4 CPU cores
• 2 pods there, one of which is some CPU intensive bg task, another one is a web server with unpredictable CPU load during the day.

I need to somehow tell k8s that my web server has higher priority than bg task in terms of CPU. Because I want to get is 100% load by bg task when there is no load on web server and e.g. 25/75% load when web server is on peak.

I tried to do something with resource requests and limits, but it works differently than what I expected. For example if I limit bg task to one core and then dedicate 3 cores to web server to handle highload peaks, it will obviously mean that CPU in my cluster is under loaded most of the time and I want to maximize utilization. If I do not set limits for CPU for both pods at all, on peaks web server works slower than needed.

Another approach which also does not work - is to schedule changes in resource requests to adapt to times when web server load is increasing, because peaks for web server are unpredictable and rather short, so I need a server up and running waiting for them.

So considering I want to use k8s as a cluster orchestrator, is there a way to define CPU priorities for particular pods?