When running azcopy copy
with huge repos, or when running azcopy bench
, the pods are killed with the error OOMKilled
. (azcopy v10)
The pod has to following resource limitations:
limits: { cpu: 1, memory: 2Gi }
DevOps Stack Exchange is a question and answer site for software engineers working on automated testing, continuous delivery, service integration and monitoring, and building SDLC infrastructure. It only takes a minute to sign up.
Sign up to join this communityWhen running azcopy copy
with huge repos, or when running azcopy bench
, the pods are killed with the error OOMKilled
. (azcopy v10)
The pod has to following resource limitations:
limits: { cpu: 1, memory: 2Gi }
azcopy
(v10) tries to use all available memory to buffer the blobs on transfer. It seems that the calculated memory size is incorrect and exceeds the pod memory limit.
This can be controlled with the option AZCOPY_BUFFER_GB
, as described in the azcopy env
output below.
INFO: Name: AZCOPY_BUFFER_GB
Description: Max number of GB that AzCopy should use for buffering data between network and disk. May include decimal point, e.g. 0.5. The default is based on machine size.
So for a pod limit of 2GiB, you can set the buffer a bit lower:
export AZCOPY_BUFFER_GB=1.5
azcopy bench ...