2

as one (sometimes failing) step of our pipeline we

  • setup environment via installation of docker which runs 3 containers through own docker-compose. Unfortunately we could not figure out how to impose any memory limits on individual containers in docker-compose v3 config.
  • run tests in sbt with SBT_OPTS="-Xmx1500m" and every second to third time step is failing with OOM

    definitions: services: docker: memory: 1024

Explanation on the memory usage is not clear on this case.

Could somebody please explain what's going on with the memory allocation to containers, docker host, and step env?

Interesting is that running the same tests, same containers for multiple times usually gives different results. Understandable that real load on the physical server changes over time and running times can differ - but memory?

Puzzled...

  • 2
    hi and welcome! does OOM stand here for "out of memory"? – Peter Dec 27 '18 at 13:09
  • 1
    yes, precisely: en.wikipedia.org/wiki/Out_of_memory – xvga Dec 27 '18 at 13:52
  • Am I reading correctly that you are setting a memory limit of 1024 megabytes on the docker container and then running sbt in the container with a max memory size of 1500 megabytes inside of the container? – James Shewey Jan 31 at 21:24
1

I suspect that Bitbucket pipelines is enforcing this memory limit, not docker-compose. This is nothing to do with the Linux kernel's OOM killer.

The page you linked is about memory limits in additional containers (e.g. for a database) used by the pipeline, not the pipeline itself.

Some suggestions:

  • Try editing bitbucket-pipelines.yml to include a size option for memory size - I would expect this is more useful than editing Docker-compose options which may not be respected. See this doc page.

  • Try running the build locally under Docker Compose to get an idea of memory usage.

It would help to include the details of your bitbucket-pipelines.yml, and the Bitbucket error message about being out of memory.

0

When you limit resources in docker-compose, the docker engine prevent containers to use more than the limits, docker do this by Linux's Cgroups. Its usage in docker-compose v3 is also straightforward: https://docs.docker.com/compose/compose-file/#resources

When your OS is in OOM state, this kills processes automatically and maybe your containers are good candidates in the state, as the problem I have faced on my Gitlab CI's build machine to build node projects!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.