From 6 months to 16 seconds? grokking DevOps speedup data from DevOps report by DORA and Puppet Labs

Question

Puppet Labs' annual DevOps reports are a very nice and representative source of information, AFAIK. Indeed, with Kim Gene on board they are THE industrial gurus I'd say.

Now I have a question to the following data in this year's report (page 21).

Can we multiply the data for 2016 and 2017?

For example, take the lead time for changes.

2555x440=1124000 # more than a milion times faster?

Does this mean then, if in 2015 it took 6 months (say 180 days real time i.e. 4320 hours) to deliver a bugfix, then

• in 2016 it took just under 2 hours,
• and in 2017, 16 seconds?

Is the warp speed delivery speed real now? In 2014, management was very sceptical about even 30x speedup. Now I'm sceptical.

Question: this must be absolute minimum values for very lightweight microservices. That is, as the bugfix commit arrives..

• you really count from this moment
• you have no compilation step
• the Docker image is very small
• you have really fast (or, fastest you can get) infrastructure.

Right?

Answer 1

I wouldn't exactly calculate it like that. It doesn't necessarily mean that the actual verification time for a changeset is decreasing that much.

The verification process itself can be (and often is these days) a processing pipeline in which the average/effective verification time per changeset can, indeed, decrease to insanely small amounts.

For example the processing pipeline can process multiple changesets simultaneouly. If you have, let's say, 60 proposed changes, each one of them touching a different file in a repository, you can combine them into an equivalent single uber-changeset touching 60 files. You could then process this uber-changeset through a verification step that takes, let's say, 1h. If the verification passes you obtain a speedup factor of x60 and an effective verification time per changeset of 1 min, even if the actual verification time was 1h. Granted, you'll lose some of the speedup when the verification fails and you need to perform bisections to identify the changeset at fault, but that's a different story.

Or you can have each of the changesets verified independently, through 60 1h verifications performed in parallel. Same effective verification time, but with a whole lot higher verification resource usage.

The point, I guess, is that there are ways of building processing pipelines which can greatly reduce the effective per-changeset verification times and still keep very large scale projects agile.

Another point to consider is the branching strategy. Those 6 months weren't the actual verification time. Most of that time was simply spent navigating through the branch spagetti and fighting the integration hell, just to propagate the fix into the proper shipping branch - effects of the waterfall development model which was considered the norm at the time.

Today CI/CD (the proper one, not the CI Theatre) and Trunk Based Development eliminate all that wasted time/effort and side-stepping, with the fix being virtually ready for delivery right from the moment it is integrated in the trunk - no unpredictable massive branch merging obstacles ever stand ahead of shipping.