2

I am trying to implement a maintenance job for cleaning up caches (e.g. Maven, NPM). In order not to disturb running jobs by deleting the caches the maintenance job should only run, if no other job is running on that node.

Any ideas are welcome that not include refactoring already implemented pipelines and by locking resources.

I also tried throttle concurrents plugin but without success.

Pipeline
def allNodes = nodesByLabel label: 'master'
def otherNodes = nodesByLabel label: '!master'
allNodes.addAll(otherNodes)

Map tasks = [:]

for (int i = 0; i < allNodes.size(); i++) {
    def label = allNodes[i]
    if ( label.isEmpty() ) {
        label = "master"
    }

    def stageName = "Clean ${type} cache on node: ${label}"
    tasks[label] = {
        node(label) {
            stage(stageName) {
                sh("rm -rf ${env.NPM_CACHE_DIR}")
            }
        }
    }
}

throttle(['jenkinsAdministrationCleanCache']) {
    parallel(tasks)
}
Throttle config
unclassified:
  throttleJobProperty:
    categories:
    - categoryName: "jenkinsAdministrationCleanCache"
      maxConcurrentTotal: 1
      maxConcurrentPerNode: 1

1 Answer 1

0

I have achieved same thing on one nodeof jenkins. Running below script every 30 minutes. If no jobs are running on node, i am making node offline and after performing tasks making node online again.

import requests, subprocess
import jenkins
from bs4 import BeautifulSoup
from datetime import datetime
import time


now = datetime.now()
timestamp = now.strftime("%d/%m/%Y %H:%M:%S")
node_name = "node_name"


jenkins_url="https://jekins/computer/"+node_name+"/"
node_url = "https://jenkins/"

jenkins_user="user"
jenkins_pwd = "password"
auth= (jenkins_user, jenkins_pwd)

page = requests.get(jenkins_url,auth=auth)

print("####### starting at {}".format(timestamp))

server = jenkins.Jenkins(jenkins_url, username=jenkins_user, password=jenkins_pwd) #for connecting to jenkins node page and reading jobs
node_disconnect = jenkins.Jenkins(node_url, username=jenkins_user, password=jenkins_pwd) #for disconnecting node before cleaning

"""
to read the node page of jenkins and find executors for current jobs running on it
"""
soup = BeautifulSoup(page.content, "html.parser")
node_executors = soup.find(id="executors")
jobs = node_executors.find("table", attrs={"class": "pane"})

jobs_running = []
for job in jobs.children:
    for td in job:
        if td.text != '' and not td.text.isnumeric():
            jobs_running.append(td.text)

print(jobs_running)
if len(list(set(jobs_running)))>1:
    print('jobs running')
else:
    print('idle')
    node_disconnect.disable_node(node_name)
    print(node_name, "is now disabled.")
    subprocess.Popen('rm -rf abc', shell=True)
    node_disconnect.enable_node(node_name)
    print(node_name, "is now enabled.")

print("####### completed ######")

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.