How do developers handle developing large scaled dockerized applications locally?

Question

I am currently working on a fairly large scale project (upwards of 250 docker containers), which has complexe dependencies on multiple APIs and front-end applications. The staging and production infrastructures are running on Kubernetes. My team is made up of approximately 50 developers all running macOS on company laptops. We are split up into teams that each work on specific parts of the project. Some are front end developers, some are backend developers. Currently we all develop on our laptops locally using either locally installed dependancies or via docker containers (mongoDb, PHP, RabbitMQ…) but we are limited to installing only the specific part of the project we are currently working on as it does not seem possible to run the whole project at the same time on each developers laptop, but this method is very clunky, and can vary from one developers machine to another. I like the whole idea of using docker-compose to have the whole project running locally, but in my experience as soon as you have a few docker volumes running, the MacBooks grind to a halt. Currently switching over to Linux laptops is not an option.

I am looking for advice on how other large scale companies deal with the problem. I would love to have a similar experience to the production systems, possibly running something like Minikube locally, but I don’t think the performance will be any better and it adds a lot of complexity for new developers. We have also thought about running our dev environments on linux VMs but that can quickly be expensive.

How do people in large companies / projects deal with this ?

Answer 1

TL;DR : you need a shared integration environment

---

There are a lot of moving parts here, so giving explicit advice on what to do might be quite impossible. When you're faced with a complex situation, the trick is to know how to think, rather than what to do.

This question will invite the criticism of generating opinionated answers, because there are many ways to provide a more stable development environment. A guiding principle may be however the idea of Dev / Prod Parity from the 12factor app which states:

Keep development, staging, and production as similar as possible

If the development teams on the various components follow independent release cycles (which they should, if the application is properly composed of microservices), then it would make sense to have a shared integration environment to develop against. This could be hosted remotely and provide a single environment for all developers.

Of course, this means you need to have access to that shared environment, and hence the network, so you will lose some offline capabilities. However, it seems like the only way to maintain offline development capability would be upgrading each individual member's hardware, so I'm guessing that you will be forced one way or another to do development connected to the network. If you're going to have network access, you might as well have access to a shared integration environment.

The tricky part of course is ensuring that this environment stays current and correct. This is where the practice of continuous delivery comes in. If you have, say, 4 teams working on 4 separate components of the application, then changes to each of those 4 components should be properly versioned and continuously delivered to the shared environment.

A good starting discussion for for developing robust APIs, I found is in the "APIs" edition of Increment.

Answer 2

The "silver bullet" really is multiple environments. DEV/TEST/UAT/STAGE... and all should be periodically synced with prod. Have you considered spinning up ad-hoc/on-demand development environments via CI/CD?

Another more immediate solution is this https://code.visualstudio.com/docs/remote/ssh

Since you're already using docker, you could build ec2 servers and run containers remotely on the remote development servers, then your dev's SSH onto the ec2 servers.

NOTE: On the remote development servers you can install the docker plugin (or any vscode plugin) to improve the development experience.