15

Brooks's law: Adding manpower to a late software project makes it later.

In his book No Silver Bullet — Essence and Accidents of Software Engineering Frederick Brooks defines the concept of Mythical Man Month:

Brooks's assumption is that complex programming projects cannot be perfectly partitioned into discrete tasks that can be worked on without communication between the workers and without establishing a set of complex interrelationships between tasks and the workers performing them.

Since 1982 we have certainly moved forward and gathered some more experience in mitigating this problem. What are some of the solutions that you have successfully applied at your work to add resources to a project without creating more problems.

  • 5
    I'm voting to close this question as off-topic because it fits better in Software Engg. SE (softwareengineering.stackexchange.com), and also cause it's not strictly Devops-specific – Dawny33 Mar 3 '17 at 3:11
  • 2
    This is a strictly DevOps specific question. It relates directly to process around software delivery. Are you sure you actually understand what DevOps means? – Jiri Klouda Mar 3 '17 at 3:14
  • 3
    You keep saying DevOps, I don't think it means what you think it means. – Jiri Klouda Mar 3 '17 at 3:22
  • 3
    @Dawny33 : Please, read one of the foundational books of DevOps - The Phoenix Project. You will not find a single mention of AWS, Docker, Jenkins, or any other tools. The entire book is about process, organizational hierarchy and structure, the way to work in teams. DevOps is a way to bring the scientific ideas that improved manufacturing in Japan and later United States to the process of Software Development. Ideas based on work of Edward W. Deming and Eliyahu M. Goldratt. What you see as DevOps is just the surface, the tools, the results. The superfluous parts of it. – Jiri Klouda Mar 3 '17 at 4:10
  • 5
    @Dawny33 This question is not suitable for Software Engineering. Although this general topic is, the question is far too broad. It is seeking opinions rather than attempting to solve a problem. Please do not suggest other communities if you do not understand what types of questions they accept. If this question were to be posted on Software Engineering, it would be down voted, closed, and likely deleted very quickly. That leads to a poor user experience. – Thomas Owens Mar 3 '17 at 13:29
17

What is the MMM

First I want to explain the context for Brook's Law. What was the assumption that made him create it back in 1975?

A Man-month is a hypothetical unit of work representing the work done by one person in one month; Brooks' law says that it is impossible to measure useful work in man-months.

source: https://en.wikipedia.org/wiki/The_Mythical_Man-Month

Back in the day, complex programming projects would mean big monolith systems. And Brooks claims that these cannot be perfectly partitioned into discrete tasks that can be worked on without communication between developers and without establishing a set of complex interrelationships between tasks and the people performing them.

This is very much true in highly cohesive software monoliths. No matter how much decoupling is done, still the big monolith mandates time required for the new programmers to learn about the monolith. And increased communication overhead which will consume an ever increasing quantity of the time available.

But does it really have to be this way? Do we have to write monoliths and keep communication channels to n(n − 1) / 2 where n is the number of developers?

We know there are companies where thousands of developers are working on big projects ... and it does work. So there must be something that changed since 1975.


Possibility to mitigate the MMM

In 2015, PuppetLabs and IT Revolution published the results of the 2015 State of DevOps Report. In that report, they focussed on the distinction between high performing organizations vs. non-high performing.

The high-performing organizations show some unexpected properties. For example, they have the best project due-date performance in development. Best operational stability and reliability in operations. As well as the best security and compliance track record.

One of the surprising things highlighted in the report is the deployments per day metric. But not just deployments per day, they also measured deploy/day/developer and what is the effect of adding more developers in high-performing organizations vs. non-high performing.

This is the graph from that report -

Deployments per Day per Developer

While low-performing organizations align with Mythical Man Month assumptions. The high-performing organizations can scale the number of deploy/day/dev linearly with the number of developers.

An excellent presentation at DevOpsDays London 2016 by Gene Kim talks about these findings.


How to do it

First, how to become a high-performing organization? There are a couple of books that talk about this, not enough space in this answer so I'll just link to them.

For software and IT organizations, one of the critical factors for becoming a high-performing organization is: focus on quality and speed.

For example Ward Cunningham explains Technical Debt as all the things we allowed to be left unfixed. This is accepted by management because it always comes with a promise that it is going to be fixed when there is time.

There is never enough time, and technical debt just becomes worse and worse.

What are these things that cause technical debt to grow?

  • legacy code
  • manual configuration of environments
  • manual testing
  • manual deployments

Legacy code As defined in Working Effectively with Legacy Code by Michael Feathers is any code that doesn't have automated testing.

Any time shortcuts are used to get the code to production; the operations is burdened with the maintenance of this debt forever. Then the process of deployment becomes longer and longer.

Gene tells a story in his presentation about a company who has six-week long deployments. Involving tens of thousands of extremely error prone tedious steps, tying up 400 people, and they would do this four times every year.

One of the tenets of DevOps is that reliability comes from doing smaller deployments more frequently.


Example

These two presentations show all the things that Amazon did to decrease the time it takes them to deploy code to production.

According to Gene, the only thing that is changed across time in these high-performing organizations is the number of developers. So from the Amazon example, you could say that in four years they increased their deployments ten times just by adding more people.


This means that under certain conditions, with the right architecture, the right technical practices, the right cultural norms, developer productivity can scale as the number of developers is increased. And DevOps is definitely in the middle of all this.

4

What I have done (and this is only subjective) is as follows:

When a manager thinking about a due date wishes to add people into my team to cut the time needed and seems under MMM, I first discuss with him or her on why this could be bad. My favorite analogy for this is to remind them that if one woman can have a baby in nine months, nine women won't have one baby in one month, but they'll have nine babies in nine months. The time isn't cut, just the parallel processing is better.

When the decision is forced upon our team, we usually try to either further divide some tasks, and when this is not possible we usually rely on pair programming, where one programmer is responsible for typing, and the other dictates the code (and switching periodically).

The code writing itself is slower, but there's fewer typo/linting and bugs when testing because of the inevitable review done while writing. I feel the overall code quality is also a little better, but I've no metrics to support that claim.

  • 1
    I like the pair programming idea. That is actually something that could help. I might be able to explain why later based on the theory I have been working on. – Jiri Klouda Mar 4 '17 at 5:31
  • please do, waiting for it! – Peter Jun 8 '17 at 10:58
4

Speaking exclusively from a CI perspective, increasing the number of developers working on a project typically translates into more people working in the same branch.

Traditional CI systems have a scalability issue in this respect: the probability of breakages/regressions/blockages increases slowing down the integration speed and inviting smaller teams to break off and move on to child branches (i.e. further slowdowns). See How can continuous integration scale for very large projects/teams?. This plays right along the Mythical Man Month concept.

The solution I suggest in my answer to that question, a highly scalable CI system would enable migration towards true CI - single branch/trunk based integration for entire greater teams (even with huge sizes).

With everyone of the same page, using the same automated tools/processes and the vast majority of the QA verifications automated inside the CI system itself it becomes much easier to switch roles and focus between team members. The whole development proces becomes smoother, more predictable, more relaxed.

Bringing new people aboard in such an environment, getting them productive simply by offloading less difficult tasks from the more experienced team members which can then take on more difficult ones is thus easier.

All of these can be seen, I believe, as soothing the Mythical Man Month concept effects.

  • In high-performing organisations, adding more developers usually means creating more independent teams who write decoupled software. This allows more people to achieve more and faster. Having them all communicate via a single integration branch is an anti-pattern and most probably will slow things down considerably. – Evgeny Mar 4 '17 at 10:05
  • Having them all communicate via a single integration branch is an anti-pattern - why? If they're decoupled in the sense that they will no longer need to integrate their work then they will touch the branch in a non-overlapping/non-conflicting manner. If their work still needs to be integrated then going on additional branches will just delay and complicate the integration by deviating from the CI methodology and losing all its advantages. – Dan Cornilescu Mar 5 '17 at 6:44
  • I think we are not agreeing on the meaning of "branch". It is fine to have one large repository, with a single branch (git, or svn), and suffer the overhead of everyone working on the same one. It is also fine to have multiple repositories where each repository has a branch that tracks that specific decoupled service. It depends on the tool, the amount of overhead it adds to commit and checkout code. – Evgeny Mar 5 '17 at 7:05
  • Ah, sorry, yes - I'm so used to it and keep forgetting others aren't. By branch I'm referring to the high-level, generic representation of the SCM branch, it doesn't really matter which are the particularities of the actual underlying SCM system(s) as long as they're managed together in a monolithic manner. 1 big repo with a main branch or 10 side-by-side repos (git modules?) each with a main branch should be pretty much equivalent from the CI prospective. – Dan Cornilescu Mar 5 '17 at 7:42
  • Then my statement from the first comment holds true. Independence cannot be done in a tower of babel, when you have a monolith the overhead is very high for everyone - so everyone is burdened. It is much better to represent decoupled projects as decoupled small VCS systems to manage. If you remember far enough, some companies were using ClearCase and other VCS to manage ALL the company code - the overhead was horrible. – Evgeny Mar 5 '17 at 7:44

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.