13

We currently have a "front-end" git repository that is all of our JavaScript client apps, and a "back-end" git repository containing all back end APIs written in .NET Core. There is also a few other random repositories built on differing technologies. This is all managed via Azure DevOps, which I believe will let us trigger builds on particular paths.

front-end repo
-- applications
----app1
----app2
-- common-library

back-end repo
-- apis
----api1
----api2
--common-library

some .net-core-product repo

We were considering putting everything in a single mono-repo because it is easier for the developers to manage with git clients such as Visual Studio. I cannot find an example of a multi-language monorepo - is there an example on github? What are the pros/cons?

1
  • if your devs cannot handle multiple repos you need better tooling to automate the builds etc ... keeping distinct repos encourages better loose coupling which can get lost going to mono ... remember any project today is already pulling in dozens if not hundreds of different libraries often directly from their own repos so handling multi repo is never going away and going mono can be self limiting Commented May 11, 2021 at 16:14

2 Answers 2

10

Monorepos have been getting a lot of attention in recent years since google popularized them. If you aren't on a similar scale as google it might not work out as well for you. I've been in a startup that moved to monorepos and we quickly found that:

  • most tools are built with one repo leading to one artifact that gets deployed. You will have to rework all of those tools to handle the monorepo. There are tools that are designed for monorepos.
  • a monorepo puts all of your eggs in one basket. If somebody steals a laptop would you rather them get half of your code or all of it? Keeping different things in their own repos reduces exposure.
  • your CI for code validation is now much more complicated. You need to support multiple languages in your code conformance tests. Developers will get upset when you check backend stuff and they've only made frontend changes. Now you're stuck trying to figure out how to disable some CI checks, but make sure they're enabled when needed.

All of this adds up to a load of work for release engineers. Do you have people focused on release engineering with free time to tackle multiple complicated projects? Google can make this work because they have hordes of engineers and they can dedicate ten folks to this sort of thing for months until it is smooth enough to develop on top of. If you are not at the same scale as google you might not be able to follow their example. The folks that have succeeded with this tend to be big organizations. You may not have found an open source example because none exists.

These points are also noted in this article:

The frank reality is that, at scale, how well an organization does with code sharing, collaboration, tight coupling, etc. is a direct result of engineering culture and leadership, and has nothing to do with whether a monorepo or a polyrepo is used. The two solutions end up looking identical to the developer. In the face of this, why use a monorepo in the first place? Please don’t!

So focus on improving your engineering culture and don't get worried about not following the latest tech trends.

2
  • 1
    Twitter is another company that uses a monorepo. Commented May 11, 2021 at 15:57
  • A proper workspace solution will have each project be its own repo, but the workspace can add or remove repos to the workspace so basically you get both the benefits of a polyrepo and monorepo. This is how Amazon built its internal Brazil workspace tool. You need to test how 2 modified repos would behave when shipped together? just add both to your workspace.
    – AjaxLeung
    Commented Jul 5, 2021 at 9:47
5

You can put two or more projects into the same repository without opting in to a "monorepo" where every project goes into the same repository by default.

If you have multiple products that need to be released in tandem putting them in the same repository allows you to release all of them at once with the same commit, "atomically". This is the principle consideration as far as I can tell.

Like in your example, if each release of the frontend requires an equivalent release in the backend. With separate repos you have to coordinate multiple pull requests between repos. This takes effort and that effort multiplies the more projects you have that depend on each other. As opposed to having a single PR that contains changes for both which are guaranteed to be released at the same time with no additional coordination effort.

An API and frontend might not be a good example since APIs are typically designed to be versioned separately from any client*. But say you have some integration tests for your UI. Both code bases are tightly coupled; a trivial change in your UI will likely require an equivalent change in your tests. Regardless of language, conceptually they are meant to "change together" and therefore, its far easier to manage them when they are in the same repository. Similar to unit tests, though those are usually the same language at least.

Another issue with multiple repos is that code reviews that require cross cutting changes have a little more friction because you have to update multiple branches in multiple repositories. Especially when doing multiple rounds of review.

Deployments are affected as well, they will have to be carefully coordinated. Merging a PR in one repo and triggering a deployment, might break the other deployments because the other PRs haven't been approved yet. Alternatively you will have to write all your projects to be backwards compatible always.

Automation across multiple repositories can alleviate some of these problems, but usually it still requires some form of manual syncing.

Realistically, either path has tradeoffs, making the answer to this question a deeply unsatisfying "it depends". My rule of thumb is that code that changes together, lives together in the same repository. Only when the burdens of keeping it in the same repository outweigh the burdens of keeping them separate do I make the split.

Regarding mixing languages

Git is language agnostic so it doesn't care. Your other tooling might. You will have to weigh that against the above benefit.

Some CI/CD tools need extra configuration when the root of your repository is not the root of your project. I would consider that tool poorly designed if it can't support that extra configuration and I have not yet run into such a tool**.

For language specific tools like linters, etc, as long as each project is in it's own folder, they're not going to be aware of each other. Running these tools at the CI/CD level now may require an extra cd <subdirectory> statement.

Your IDE might get confused when you open the project from the root. For example if you had a Python, C#, and Typescript in the same repo, one VSCode instance would have 3 "language servers" it would be communicating to, potentially 3 plugin UIs to hide and show depending on the file you're looking at, 3 places to pull debug configurations from and so on. So far in my experience, this only results in some minor configuration changes to workspace settings and otherwise everything works fine. Worst case I end up opening another instance VSCode per project which is what I'd be doing anyway if they were separate.

Teams play a role as well

This also touches on your team structure. If you have separate teams for each product, it might make sense to separate them because the cross team coordination needs to happen anyway because no one member commits in multiple repos.

I personally don't partition teams like this because I prefer teams that are able to write code in any part of the application system, front, back, side whatever. But I recognize this is not always desirable or possible.

Why does Google, Twitter, etc do it?

At Google scale, their projects' dependencies are probably developed by their own internal teams. I suspect this means that in any given piece of work, they're mostly making commits that affect many projects at once.

Since that was the default case for them they decided to put everything in the same repo. They incurred a significant cost in doing so but for them, it makes sense because cross cutting changes are the norm because of all these internally managed dependencies.

For typical projects, the dependencies are mostly projects managed by 3rd parties outside of the company. Think nuget and npm packages. And your other projects are largely independent from each other. So putting every project in a single repo by default doesn't make sense.

Overall, keep in mind, Google, Twitter, etc are all managing products they own and have total control over. Whereas your company may be developing multiple projects for different clients. In which case, you are facing challenges that Google does not despite how massive they are and Google has not considered those challenges in their solution.

Here is a self-proclaimed Google engineer discussing why they like the monorepo. It is a confirmation of the above.

Say I make a change to a commonly used library (let's say deprecating a function, and replacing it with another):

  • I can see literally every use of the old function.

  • I can run the tests for everyone who uses that function.

    • this is automated; the build/test tooling can figure out the transitive set of build/test targets that are affected by such a change.
  • I can (relatively) easily update every use of the deprecated call with the new hotness

  • I can do that all within the same commit (or set of commits, realistically)


None of this is impossible with multiple repos, it's just a lot more difficult to coordinate

Asides

*In the case of an API and a frontend that uses that API. I would typically consider them independent and version them separately. An API should be designed to service many clients and not just one frontend. Otherwise a server side UI framework like ASPnet or PHP would be the more appropriate choice imo. I recognize though that this is often not the case and we have frontends tightly coupled into backends designed just for them.

**SonarQube in the distant past had issues with scanning repositories with multiple languages. So there is some precedent for important tools not being able to handle mixed language projects. Though that issue may have been before the project really took off. Certainly it was before I had heard about it.

1
  • this is a great, unbiased answer
    – Thom
    Commented Jul 19, 2023 at 14:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.