6

Is it possible to launch a jenkins agent from within a script in another job? I'm open to either shell or pipeline options.

To be clear I don't mean the agent the job is running on itself. I'm talking about spinning up a completely separate agent.

Sometimes our master gets slammed with unit test requests and will make jobs wait in the queue for 15-20+ minutes instead of just spinning up more agents. I want to see if there's a way to spin an agent up intentionally so I can tie it into a job that polls the queue for wait times.

Improve this question
6
  • Do you have some Jenkins slaves already?
    – 030
    Mar 5, 2018 at 16:50
  • Yes I do @030 I'm looking to spin up another of a type I already have.
    – Alex
    Mar 5, 2018 at 16:56
  • What is your infrastructure? Are you using some cloud solution or is bare metal? I recommend you to test some integration with docker or Kubernetes.
    – wolmi
    Mar 5, 2018 at 20:18
  • We're moving to k8 sometime in the next 6 months. I need a quick-fix for our current issue until we can switch over.
    – Alex
    Mar 5, 2018 at 20:20
  • In this case take a moment to calculate the cost of automated agent launch vs increase the actual resources for that period (if possible)
    – wolmi
    Mar 5, 2018 at 20:27

2 Answers 2

Reset to default
1

What you are looking for is called dynamic provisioning of Jenkins' slaves, and yes this is doable. Setting it up depends upon your desired infrastructure provider/resource. The easiest example in my opinion is using AWS EC2 instances. Install the Amazon EC2 Plugin for Jenkins, and configure it with the appropriate access keys, IAM roles, etc. to allow it to provision EC2 instances in a specified VPC. Here is a short (but slightly outdated) guide.

If you are moving to K8's and Docker, you may want to consider looking into the Docker Plugin for Jenkins. If you are using bare bones infrastructure, this should allow you to more effectively utilize what current hardware you have.

Improve this answer
4
  • I think you've misunderstood my question; we are already set up to automatically launch slaves. The issue is that the automatic launching is too slow; jenkins will wait too long to spin up new instances. I am looking for a way to launch them on my own schedule to supplement the automatic launches.
    – Alex
    Mar 7, 2018 at 14:15
  • If time is your issue, you could dedicate a handful of particular slaves to running unit tests and turn them on when you think the load will be greater than your current nodes can handle. Then turn them off when your job is finished executing. This will give you the quickest turn around time vs having to provision new instances. If the wait is still to long, you may just need to increase the minimum number of continuously running nodes to accommodate the long wait time.
    – Preston Martin ♦
    Mar 7, 2018 at 14:45
  • It's a good suggestion but the issue is we need single-use slaves - reusing pollutes our results. It's looking from the answers to this question that there isn't a way to programatically spin up a new agent so I guess we'll just have to deal with the wait until we switch to K8s
    – Alex
    Mar 7, 2018 at 14:48
  • You can use the docker plugin to spin up new containers from a docker image that is run the Jenkins agent without using kubernetes. If you truly need them to spin up quickly and need them to be fresh single-use slaves, then this may be your best approach.
    – Preston Martin ♦
    Mar 7, 2018 at 15:01
0

We have a jenkins master which has like 10 agents connected via ssh, usually they are down. But when a build starts, jenkins starts an agent automatically(It launches the jar file over ssh).

So I would believe jenkins should solve that problem with the build in logic. If the queue is full it starts up agents which are currently offline.

Improve this answer
3
  • We have our jenkins configured to launch automatically, the issue is the delay. I'd like to supplement the automatic launching with more manual launches.
    – Alex
    Mar 6, 2018 at 14:05
  • So what are you looking for ? we used to spin up pre configured VMs on Azure according to need, it is relatively fast since everything is already in place in the VM image.
    – Rsf
    Mar 6, 2018 at 14:51
  • I want a command / api call / etc that will let me force-launch a new agent, the same way you can by going to the nodes page and provisioning one manually. The wait-time jenkins has is too long for us; we want a more aggressive provisioning strategy than jenkins' conservative one. It waits too long to see if a previous agent frees up instead of just launching a new one.
    – Alex
    Mar 6, 2018 at 15:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.