When running azcopy copy with huge repos, or when running azcopy bench, the pods are killed with the error OOMKilled. (azcopy v10)
The pod has to following resource limitations:
limits: { cpu: 1, memory: 2Gi }
Add a comment
|
azcopy (v10) tries to use all available memory to buffer the blobs on transfer. It seems that the calculated memory size is incorrect and exceeds the pod memory limit.
This can be controlled with the option AZCOPY_BUFFER_GB, as described in the azcopy env output below.
INFO: Name: AZCOPY_BUFFER_GB
Description: Max number of GB that AzCopy should use for buffering data between network and disk. May include decimal point, e.g. 0.5. The default is based on machine size.
So for a pod limit of 2GiB, you can set the buffer a bit lower:
export AZCOPY_BUFFER_GB=1.5
azcopy bench ...
