This project has a pipeline that runs integration tests against a spun up instance of Gitlab. Each Job in the integration test stage has its own instance testing different build distributions (es, cjs, original source). Unfortunately, the shared runners have limited resources and inconsistently cause these jobs to fail because the spun up GitLab instances have insufficient resources. This occurs more often when I have more than one instance spin up at a time since the jobs run in parallel.
Gitlab Support has suggested using a dedicated runner OR refactoring my pipeline. Since I'm trying to maintain a $0 cost for this open-source project, the only feasible suggestion (that I'm aware of) is the latter one.
I've tested doing just this, specifically splitting these integration tests into separate stages to stop creating multiple GitLab instances at the same time. This, however, has not improved the results. The only other idea I can think of is running all the integration tests in the same script, instead of separate stages or jobs, which would only require spinning up one instance of GitLab. In this solution, I would have to add some sort of id's to the data being created in the tests to avoid having clashes (i.e., creating a project must use a unique name)
What I want to figure out is if I'm overlooking something? Are there any better ways to improve the pipeline to avoid the resource bottlenecks causing the crashing?