I am trying to answer a question:

Do we need more worker machines to handle current load?

We have a variable number of jobs coming in and each job takes different time. Here is a snapshot of our system for a period 12 hours.

Jobs in 8 hours

But the number of jobs in system changes based on day and time.

How can I calculate throughput of our system and build monitoring around max load that our system can handle?

  • You're looking at math and queuing theory. Your variables will likely be median job-size, time to process that median job, and a number derived on how many simultaneous jobs your server can handle. That's a bit beyond me at the moment. – sysadmin1138 Aug 24 at 14:53
  • 4
    You may be asking the wrong question. What's your SLA or target for queue processing? – CodeGnome Aug 24 at 16:56
  • Actually I am defining SLA and this exercise is part of coming up with SLA metrics. – Rohit Aug 24 at 18:16
  • 2
    Your question lacks some information, the graph provided means nothing related to worker machines except numbers jobs in bucket of duration. Server capacity has to be determined on one or several servers resources (disks throughput, disk storage space, CPU usage, memory usage) based on what jobs relies on, and compare these resource usage with the capacity you can process the current data. You need to cross these two information over the time to see how your current worker capacity can process the workload in 3, 6, 12 months. I over-simplified naturally because of comments length limitation. – Baptiste Mille-Mathias Aug 25 at 17:38

(N.B. this answer is, in the spirit of the question, high-level and not very concrete...)

How can I calculate throughput of our system

If by "calculate" you mean to take some data points, dry run some formulas and get a pretty close approximation to your throughput - that is pretty much impossible, unless you have a lot more information - which may also include pretty random parts, hence quite hard to do theoretically.

If you mean "measure", then it could be as simple as checking your logfiles and counting lines per time unit.

and build monitoring around max load that our system can handle?

Unless you're NASA, that's probably a case for trial and error. Run your system(s) and see how much troughput you get. Increase worker nodes. See what happens.

If you already know the behaviour of your overal system - i.e., whether it scales linearly, whether there are bottlenecks like databases, lock contention and such, then you can take shortcuts or good guesses.

That's one of the reasons for doing the container-based scaling we do these days - you can throw a few more workers at it relatively easily.

How you actually (technically) do that depends on what platform you are using. AWS, Kubernetes/OpenShift etc. come with techniques or settings do do it automatically for you. I assume those work mainly with the metric of "free workers" - trying to get the "free:busy worker" ratio into some target corridor so that every new request has a very high likelihood to hit a free worker, at any time, thus getting (theoretical) constant time for each request.

KPI (key performance indicator) or software metrics could help. For starters after a bit googling, Kubernetes has self-healing mechanism in which whether thresholds could be set up to heal after exceeding some measures is something I never tried. I'd love to know if it is doable though.

Read: Monitoring Kubernetes performance metrics.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.