4

I have self-hosted build agents using Azure DevOps. We have the clean flag in the checkout step, however we run out of space sometimes.

What is the preferred way of cleaning the agents' work directories?

The problem is, that the clean process is in the beginning of a pipeline, and leaves everything behind. This is even worse if we have two parallel builds. (First clean then clones the repo, then the second cleans and clones the repo, then first build, then there is no space for the second build). Unless the first build is scheduled again, the build outputs stays on the agent.

We tried to schedule a deletion task every day, but it doesn't solve our problems.

We tried to add a script at the end of our yaml file that cleans the working dir. Is it recommended?

    - powershell: Remove-Item $(Pipeline.Workspace)/** -Force
      displayName: Clean after publish

Also it would be great if we could clean up even if the build fails, or someone cancels it midway.

3 Answers 3

2

I tried to use DeleteFiles@1, like @lionclub. But the implementation is very slow for large repositories.

So, I decided to use PowerShell:

- task: PowerShell@2
  displayName: 'Cleanup source folder'
  inputs:
    targetType: 'inline'
    script: 'Get-ChildItem -Recurse | foreach { Write-Host $_.FullName ; Remove-Item $_.FullName -Force -Recurse }'
    failOnStderr: true
    pwsh: true
    workingDirectory: '$(Build.SourcesDirectory)'

There are variations:

  1. Using failOnStderr or not
  2. Using PowerShell or PowerShell Core (pwsh: true)
  3. Add separate tasks for $(Build.ArtifactStagingDirectory) and $(Build.BinariesDirectory)
  4. Clear complete working folder e.g. by using variable $(Agent.BuildDirectory)
  5. Adding condition: always() to always clear folder, incl. failed runs
2
  • Would it not be quicker to skip the loop and just run Remove-Item $(Build.SourcesDirectory) -Force -Recurse
    – John
    Commented Nov 28, 2023 at 16:18
  • 1
    This is maybe another variation. My version reports progress. Please note that I reverted your edit and converted it into variation because I think that failed runs should not be cleared by default.
    – lg2de
    Commented Nov 29, 2023 at 10:44
2
- task: DeleteFiles@1
  displayName: 'clean *'
  inputs:
    SourceFolder: '$(Pipeline.Workspace)'
    Contents: '**/*'
    RemoveDotFiles: true
  condition: always()

will clean everything in '$(Pipeline.Workspace)/notClean/allClean'.

1

There is a clean option within the job properties. See the job definition.

jobs:
- job:
...
 workspace:  # Workspace options on the agent.
   clean: string # Which parts of the workspace should be scorched before fetching.  (outputs, resources, all)

As it would be called before fetching, there will be always space for the new build.

1
  • 1
    This does a clean of the workspace before your build runs, not after
    – Enrico
    Commented Sep 21, 2023 at 12:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.