What is a reasonable level of coding expertise expected of an SRE?

Question

SREs and coding is a contentious topic. In most situations, SREs end up spending a lot of time on operational tasks, even while being efficiency and automation focused. This means that SREs don't spend as much time coding as a full time developer, which puts them in a very disadvantageous position as far as programming rigor is concerned.

Also, SREs are typically responsible for multiple services or applications, unlike developers who may be more focused on a small number of applications. Gaining deep knowledge of large codebases can be a daunting task especially if the focus is diffused by interrupt-driven nature of SRE work.

To be very good in coding and be able match a developer's level of expertise in programming, SREs need to spend more time coding on a continuous basis, which may not be possible.

Also, the question arises - if an engineer is a great developer, why would he or she opt for an SRE role? The pure development roles seem to have a better career path.

Also, the argument that SREs should spend most of their time coding and not in operations seems a little disconnected from reality. Not all operations is evil and not all code is an angel either. As someone said, "the best code is the one that was never written". Running operations efficiently and reliably is an important goal for SREs and coding can be helpful in achieving that goal, though it is not the only means to get there.

So, what is a reasonable expectation for an SRE when it comes to coding? Is it practical to have two cadres of SREs -- development focused ones or "SWE SREs" and operations focused ones or "Operation SREs"? Could they be a part of the same team? Is it practical to rotate people between these two roles?

I hope this question is not dismissed as opinion-based.

Answer 1

If an engineer is a great developer, why would he or she opt for an SRE role?

Here is a caricature of being a developer: Working in Sprints, getting a ticket with all of the BA work completed, spending 3-4 days on that problem, marking it done, getting the next ticket and so on. This is a really boring work environment. Yes there are better environments to work in, and not all development roles fall into the scrum mediocrity trap, but it happens frequently.

DevOps is new enough that it's not yet been "solved" as a problem. It certainly hasn't yet had all of the engineering challenge managed into separate roles, so it allows a certain type of developer a bit more freedom, working using multiple skills, in a fast paced environment. It's not for the faint hearted.

What is a reasonable expectation for an SRE when it comes to coding?

That they are a programmer that you would want on the development team. SRE is not a place for substandard developers. The development challenges in SRE are as great as that in the development space, but the space for errors is less.

Is it practical to have two cadres of SREs -- development focused ones or "SWE SREs" and operations focused ones or "Operation SREs"?

Not really. To be in SRE you must understand Dev and Ops. If thats not the case, then you are not doing SRE, you are doing something else. It doesn't mean it's wrong, or not working for you - it could be just what you need for your environment, and you have the people that make it work. It is possible to have a specialist who isn't really a developer on the team, IF they bring an important skill to the team that the developers are not able to cover (e.g. low level networking or linux kernel knowledge), but this is done rarely.

Could they be a part of the same team?

You can place Dev and Ops into the same team. It would make for some interesting management challenges. Please run the team in a more kanban style, ops doesn't work in sprints

Is it practical to rotate people between these two roles?

Most developers benefit from an exposure to the land of DevOps. However, if it is a true ops environment, that timescale is probably 2-3 days at most. Not sure most ops people would be able to assist meaningfully as a developer, but they would often make a good tester. Most ops people would not want to be in a testing role for long.

Answer 2

I think that if an SRE is spending “a lot of time on operational tasks” then they are missing an opportunity to improve their situation through code. Not all code is managed, but one of the driving tenets of the DevOps and SRE models is to approach operational problems as if they are software problems.

As far as rotating roles within and SRE team, you do what it takes to deliver value for your organization. I don’t think there’s a single right answer; I do think that the goal is to minimize manual intervention as much as possible, as well as doing everything you can to identify and resolve reliability issues. Whether you fulfill this as a traditional developer or from a zero-code approach doesn’t matter.

Answer 3

So, what is a reasonable expectation for an SRE when it comes to coding? Is it practical to have two cadres of SREs -- development focused ones or "SWE SREs" and operations focused ones or "Operation SREs"? Could they be a part of the same team? Is it practical to rotate people between these two roles?

class SRE implements DevOps as they say. There are other models.

If you classify SRE as Software Engineers tasked with operations and asked to deliver software engineering oriented solutions, then you will want to hire SWEs with operational knowledge, and ask them to build a team to inculculate that knowledge. They should be coding first, but oriented on systems and infrastructure. SRE is where generalists who deal with high levels of ambiguity and ignorance thrive; there's always something new.

Concretely: A SRE should be the kind of person who looks at Kubernetes and is happy to write code against the APIs directly, rather than routing through YAMLs as the interface.