Is scrum or kanban really useful for SRE teams?

Question

Agile practices like scrum and kanban were primarily designed for software development.

Interrupt and unplanned work is a significant component of what most SRE (Site Reliability Engineering) or DevOps teams do. While it is always useful to use a tracking system like Jira to manage work, do sprint or kanban really work for SRE teams?

The constraints I see are:

• The work is very dynamic in nature, with priorities changing on a daily basis. Because of this, the sprint duration of two weeks seems very aggressive and it adds unnecessary overhead.
• People being on call adds another dimension to the problem. Sometimes, more than one team member might get involved in on call / post-mortem tasks.
• The team doesn't have a single "product" and hence it doesn't yield itself to a common planning process
• Daily standup meetings may not make much sense because of the lack of overlap among tasks
• The team might be working on tasks related to more than one partner teams and hence spanning multiple Jira projects. Since a sprint or kanban board allows only one Jira project, it may not be able to fit in all the work.

From what I hear from many SREs that I have spoken to, sprint planning hasn't worked for them at all. I would like to hear from the community here what their experience with sprint and kanban is.

I asked this question on scrum.org as well:

Can scrum be used effectively by SRE teams?

And here is a blog post that raises concerns about Agile and SRE in general:

Agile is Lava

Answer 1

Is scrum or kanban really useful for SRE teams?

TLDR: No

More nuanced & Long-form answer:

Speaking from many experiences at various software companies:

Niether strict Agile nor strict Kanban has ever truly "worked" for any Operations team that I've seen. Whether the company leadership labeled them as the "DevOps Team", "SRE Team", the "Core Operations Team", the "Platform Team", or any other label for that type of Ops work, those strict type of methodologies were never a perfect fit for all the unplanned and ad-hoc work. No matter how hard the company leadership tried to shoehorn the team into some form of those methodologies.

The reasons were due to all of the bullet points you mentioned above:

• The work is very dynamic in nature, with priorities changing on a daily basis. Because of this, the sprint duration of two weeks seems very aggressive and it adds unnecessary overhead.

• People being on call adds another dimension to the problem. Sometimes, more than one team member might get involved in on call / post-mortem tasks.

These ones are precisely the issues why Agile sprints are not useful. The type of work is very dynamic in nature. Planned vs. Unplanned work constantly compete for the Ops/SRE/DevOps/Whatever-you-label-it Engineer's time, bumping and shifting priorities as the "sprint" goes on. Most Agile practices such as "planning poker", or adding "points" to stories/tasks, also fail due to the "Planning Fallacy". This is a built-in cognitive bias that all humans share. It means that we tend to underestimate the amount of time it will take us to complete a task. Supposedly the Agile methodology has this concept that is alleged to alleviate the issues caused by the Planning Fallacy and the "Mythical Man-Month": The "Story points" system.

Most planning-poker style meetings on Software & Ops teams either used a "T-shirt size" (Small, Medium, Large), or fibonnacci number point system, which had no basis or equivalence to "time", by design. This exercise was by definition intentionally useless to estimate time, although leadership at these companies expected it to do so thanks to their conflation of the Agile methodology's concept of "cadence" and the average number of points that were assigned to these tasks in a sprint. These average point estimates changed and evolved over time as team members, tasks, and projects all changed. Unfortunately, the planning fallacy also applies to non-time-based difficulty estimates, because most all humans are overly optimistic about how difficult an unknown problem or task might be. For unknown or unplanned tasks, the actual difficulty can wildly vary and is not actually plannable.

So, all of the above results in lots of carryover of tasks to the next "sprint", whatever time period is chosen for a sprint cadence. As things change (change is the only constant), the point system estimates change too. However, managers' and company leadership's expectations generally aren't as flexible. This sets the team up for failure, because it's inevitable that at some point things change enough to throw off the sprint estimates.

Reliability & Ops Engineering is not a "sprint", it's a marathon.

• The team doesn't have a single "product" and hence it doesn't yield itself to a common planning process

This is another aspect describing the same dynamicity of the type of work mentioned above.

• Daily standup meetings may not make much sense because of the lack of overlap among tasks. The team might be working on tasks related to more than one partner teams and hence spanning multiple Jira projects. Since a sprint or kanban board allows only one Jira project, it may not be able to fit in all the work.

This is yet another reason why most Agile methodology concepts do not fit this type of work. Working across teams & projects is very common for Ops teams. Overlap is rare when working across teams unless there is an underlying common infrastructure that these teams all depend on.

Every single company ended up doing some form of loose Kanban in the end. That is to say: A board was used to track tasks and these were organized by priority. The team was encouraged to limit tasks in progress so as to not become burned out, or overwhelmed by context switching. There was no usefulness of tracking cycle time, nor any of Toyota's Kanban metrics.

These were software companies, not car factories.

The usefulness of strict Kanban was very debatable, due to the fact that it was developed at a car factory. Factories have processes and machinery to fabricate the same type of parts repeatably. Software engineering and Operations are not like a car factory, even when you try to make things repeatable and deployable by putting them into containers. Containers just hide the internal complexity, they do not eliminate it. For these reasons, strict Kanban also is not useful for this type of work.

In Conclusion:

The SRE & DevOps methodologies themselves, and the type of work involved are at odds with the Agile methodology, which was developed with Software Engineering in mind, not Operations. I've observed many dysfunctional software companies where development teams can chuck things over the fence and call it "done", and then tell Ops to deploy them. They then expect Ops to keep all these plates spinning until the end of time. There is no end point unless a product or service is decommissioned. Additionally, those companies made the mistake of not building in reliability into the product from the beginning of the product lifecycle. This is a recipe for Operations Team burnout and overload when all these complex pieces of software start building up and breaking in complex ways. The company leadership did not scale the Ops/SRE/DevOps team appropriately, nor did they allow Ops Engineering or Developer Engineering time for reducing toil.

Answer 2

We don't use Agile for the DevOps group ourselves, but we do integrate with the normal Scrum Teams. When something is needed by the team from DevOps, such as optimizing the build server, the related team puts a PBI in their backlog with a 'DevOps' label. Our lead has a custom dashboard in Jira with all issues labeled 'DevOps'. They work with the Scrum Master to get prioritization and then one of the DevOps Engineers is tasked with being an ad-hoc member of that Scrum Team for the life of that issue. This helps us prioritize work based on our "customer's" priorities, ties our effort to the sprint, and allows us to get "credit" for the work we're doing.

Prior to this, we used Kanban on Jira just for a todo list. Unfortunately, we would sometimes have to stop working on something we had planned to work on because one of the teams needed something. Now, unless it's an emergency, they basically request DevOps resources (people/time) via their backlog and the Scrum Master communicating that need to our Lead.

Answer 3

Agile works very well for this type of chaotic environment. However, for the reasons you highlighted, pure textbook Scrum may not be a great fit. As a Scrum Master who does a fair amount of DevOps I've used Kanban boards in Jira to track work.

Kanban Advantages

• Visualization of the work the team is doing.
• Identifies bottle necks for the team.
• Helps keep the team focused on moving work along.
• Allows for work to be added at any point.
• Gives visibility to work being done for other teams that are relying on the SREs.
• Can be generic to cover multiple types of jobs.

While a traditional stand up may not be beneficial, do consider modifying it. A good stand up meeting highlights blockers that a team member may be facing that another team member knows how to solve.

Answer 4

IMO: Yes, Scrum can be used effectively by SRE teams. I have never heard or read that team members can only work on sprint workitems, so it is a misconception that you need to account for all of your time and effort in the scope of a sprint; Also, I feel there is a misconception that all of your work is applicable to sprints. So, convergently, you should manage some work with Scrum and some outside of it.

Scrum, at it's core, is a framework (with artifacts and events) that is flexibly and used for continuous improvement over the nature of work that is accepted by the team (or applicable to such work).

What you need to do is strike a balance between the time and effort you can provide for planned, accepted sprint work -- that you and your team can deliver on -- and the time and effort to tackle unplanned work outside of the sprint.

---

There is a measure of severity and priority to all workitems. If you're considering Scrum, you should accept work up to a particular severity. You should also consider work that can reduce the severity of upcoming work (if at all possible). I would also recommend you start your sprints with about half capacity (which may be difficult to determine on since relating time and effort in a capacity fashion in the beginning is) - go light. Your team's goal should always be to deliver what work your team accepts, so be picky and don't overextend.

---

I think the nature of your work is actually comparable to production bugfix work for development teams. Thus, you may want to consult production maintenance teams on their Scrum adoption and success, and Not necessarily constrain yourself to DevOps Engineers experience in program and project management.

Answer 5

Here is my observation and this is what I follow:

• Use a ticket for any piece of work that takes an hour or more of effort
• Keep the ticket updated as and when you make progress on the work
• Try to set a due date for each work item and try to finish them before that time
• Set priorities to indicate the urgency level of the ticket
• Organize work into Epics, Stories, and Tasks if you are using Jira. Use Stories for planned work and Tasks for unplanned work.
• Make sure only a small number of tickets are in progress at any point in time, to maintain a strong focus
• Keep updating ticket status (Open -> In Progress, In Progress -> Blocked/Waiting, In Progress -> Closed etc.)
• Use a dashboard to track your tickets
• Make sure that no tickets sit in the system for too long
• If possible, influence your team to follow the same process and have a common dashboard to track team progress

No need to worry about Sprint and Kanban. If you want to really be very formal, start using story points to track effort estimates, but don't waste team's time in task estimation meetings.