34

The idea of having a DevOps Engineer has become quite popular recently, and it seems appealing to just have a person who can slot in and provide many of the benefits of DevOps, as described in the Puppet blog:

Organizations using DevOps practices are overwhelmingly high-functioning: They deploy code up to 30 times more frequently than their competitors, and 50 percent fewer of their deployments fail, according to our 2015 State of DevOps report.

However, I've noticed a lot of vocal opposition to the idea of a DevOps Engineer to try and make these improvements:

Even with broad agreement about core DevOps attributes, controversy surrounds the term “DevOps engineer.” Some say the term itself contradicts DevOps values. Jez Humble, the co-author of Continuous Delivery, points out that just calling someone a DevOps engineer can create a third silo in addition to dev and ops — "...clearly a poor (and ironic) way to try and solve these problems."

Why might it not be such a great idea for a business to hire a DevOps Engineer to try and 'implement DevOps', as opposed to the organisational change advocated by blogs like this? Will the benefits be negated by just having an isolated DevOps role?

Improve this question
1
  • You should do whatever is most effective for your business, team, and project. You should experiment to find out what is most effective. Agility is effecting change appropriate to your specific situation. As Kent Beck put it, "any decent answer to an interesting question begins, 'it depends...'"
    – Adrian
    Commented Apr 6, 2017 at 15:15

7 Answers 7

Reset to default
26

TL;DR: You should never try to hire a DevOps Team

There are essentially three most common roles to hire for:

  1. DevOps Architect / Evangelist
  2. DevOps Engineer
  3. CI/CD Engineer

These roles are distinct from your 6 essential software development roles that traditionally compose the software engineering organization:

  1. Product Management
  2. Software Development
  3. Tools Development
  4. Security and Compliance
  5. Quality and Testing
  6. System Operations (SRE)

Lets go through the three roles one by one and see how they fit

DevOps Architect or Evangelist

  • Why: If you are lost, slow, broken and don't know what to do.
  • When: At the start of the process in planing stages.
  • What: Management level role to guide all managers and leads in the entire Software Engineering org. This person will plan the entire transformation of your engineering organization to a highly functioning state.
  • Who: Consulting member well versed in theory, management practices, culture topics and operations who reports directly to VP of Software Engineering.

In some cases and for smaller and mid-sized companies you might start the process instead with hiring a consulting organization, like DORA.

DevOps Engineer

  • Why:
    1. To bridge the gaps between teams if they are organized along the functional roles mentioned above to ensure cross functional level cooperation.
    2. To embed with product oriented teams, which have each of the 6 traditional roles included in the team, to help bridge the knowledge gaps and to help with implementation and adoption of the novel practices and tools.
  • When: Once you've laid out your plans and the organizational transformation starts and the entire management team is on board.
  • What: Enable cross function cooperation, ensure that team boundaries are broken down, that local optimizations inside teams are not creating a barrier to high throughput of work throughout the value chains all the way from customer wishes to customer deliveries.
  • Who: Experienced engineer with skills both in software development and system operations. He should be skilled in the best practices, process and culture changes related to DevOps transformation.

CI/CD Engineer

  • Why: To help implement CI/CD pipelines, integrate your tool chain, bring in the tools that will enable better working of the company.
  • When: During the transition in larger organization, while the above roles have been already filled.
  • What: Engineer, which is essentially part of the tools team that will be able to setup CI/CD pipelines and start integrating internal systems in a way that will remove friction from the throughput of work.
  • Who: Engineer experienced with Tools, Integration process, Release Management and DevOps practices. Someone who understands they are replacing human gating in release process by Automation.
Improve this answer
7
  • 11
    I've a hard time linking your tl;dr to the rest of the answer where you give reasons to hire...
    – Tensibai
    Commented Apr 6, 2017 at 13:58
  • The rest of the answers explains how none of the roles related to DevOps are conductive to creating a team of those people. Don't hire new team, embed individuals into specific places in your organization based on needs.
    – Jiri Klouda
    Commented Apr 6, 2017 at 14:17
  • 5
    @JiriKlouda excellent answer, I almost 100% agree, short of the term "System Operations (SRE)" - System Operations != Site Reliability Engineer, the latter is Google's model for DevOps that still embodies the core principles of DevOps but retains some of the advantages of having a team of specialist operators.
    – Richard Slater
    Commented Apr 6, 2017 at 14:33
  • What I meant is operations team in any form, either traditional or SRE or any other form of infrastructure or platform management. And trust me, you can have Site Reliabikity teams without fully adopting DevOps :)
    – Jiri Klouda
    Commented Apr 6, 2017 at 14:46
  • 1
    Honestly there is simply not enough there. CI/CD engineer should know enough to design the pipelines. DevOps architect can do the high level work at the organizational level. I have separated the role from DevOps engineer because it has different characteristics. It is tools oriented engineering job, it can be easily part of a team (tools/integration/internal apps team). This would be what people mistake DevOps engineers for mostly. It is the evolution of the release engineers, but with automation. Instead of gating, they are now simply building and monitoring the automation.
    – Jiri Klouda
    Commented Apr 8, 2017 at 19:35
10

I'd argue Devops Engineer as described in your question link is mainly a sysadmin role. Quoting the skills here for background to this answer:

Your climbing gear.

  • Strong background in Linux/Unix Administration experience with automation/configuration management using either Puppet, Chef or an equivalent
  • Ability to use a wide variety of open source technologies and cloud services (experience with AWS is required)
  • Strong experience with SQL and MySQL (NoSQL experience is a plus, too, since we also use Redis)
  • A working understanding of code and script (PHP, Python, Perl and/or Ruby)
  • Knowledge of best practices and IT operations in an always-up, always-available service

In this sample job description DevOps Engineer is just a buzz word for a sysadmin comfortable with cloud based infrastructure, automation, able to read code to help in diagnostic, and aware of high availability practices and solutions.

This is loosely related to DevOps practices and silo breaking culture between dev and ops as seen in this question What is the difference between Sysadmin and DevOps Engineer?

It won't be a good idea because a sysadmin, how comfortable he/she can be with devops practice and culture, is not the right person to drive a company transformation. You won't be hiring this person with a culture change in mind but with a tool configuration view, which won't really help breaking the processes. This may also be badly received by his/her colleagues and you'll just bring resistance to the change if there's no culture change planned beforehand

For a successful pattern toward the gains of devops, @Jiri Klouda's answer gives a great overview on an acceptable DevOps Engineer role along with the step in the change it would bring value and help toward success.

Improve this answer
4
  • 1
    How would you suggest one distinguishes between a sysadmin who is comfortable reading code to diagnose issues, cloud infrastructure and automation, and traditional sysadmins who have lots of experience but none of those skills?
    – avi
    Commented Apr 6, 2017 at 16:28
  • @avi by their resume, mine for the example I'm more comfortable to compare with, has those while still being titled Net/Sysadmin . I have references to devops organization for some projects I've worked with. And I usually don't run for proposal using devops as a buzzword because of the caveats I mentionnes in this answer (been hit once during a contract )
    – Tensibai
    Commented Apr 6, 2017 at 16:33
  • @Avi if you mean in a job proposal, in the details of the proposal, qualifications required are far different from the ones in the question and those in Jiri's answer to keep the job titles accordingly IMHO
    – Tensibai
    Commented Apr 6, 2017 at 16:35
  • 1
    I'm inclined to say that a sysadmin who isn't comfortable with automation is just a poorly-skilled sysadmin, not a different job title. See also this essay of John Allspaw's.
    – Xiong Chiamiov
    Commented Apr 7, 2017 at 17:53
6

I realise this answer might not be a perfect fit for you, but here's what I did

I was the first developer working at a very busy ecommerce startup, with incredibly high traffic. I realise the company was still young, and that, for a while, I would be the only technical in-house resource.

Knowing this, I decided to structure my infrastructure in such a way, that I would have to do ZERO systems administration.

I decided to host in the cloud, because doing so relieved me of systems maintenance. I looked for a AWS engineer, with puppet experience. Together, we architected an autoscalable, infrastructure, written as code in cloudformation. All configuration files were versioned within puppet.

This allowed me, as a developer, to assume this devops role. I built code release tools, in python, fabric. I used the same script to bootstrap my application onto new servers that are autoscaled.

This worked very well, and today, 3 years later, I still don't do any system maintenance. We have a systems admin (the same AWS engineer) for 10 hrs a month, and I try to structure his sprints in such a way that I don't become an annoyance for him. In that way I respect his time, and manage his sprints in the best way I can.

If a system has a degraded performance I simply terminate it, and another one spins up in his place.

I hope this answer could somehow benefit you

Improve this answer
3
  • Very helpful, thank you. It's interesting to hear how you essentially became what others might call a 'DevOps Engineer' indirectly, and I think (from what the other answers have said) that your way is more in line with the DevOps philosophy of not having a completely separate 'DevOps' department. It sounds like it's worked well for you so far!
    – Aurora0001
    Commented Apr 8, 2017 at 8:33
  • So basically you manage everything yourself? What happens if you leave the company? Will the business be able to survive? What is the point of view of your management on this?
    – 030
    Commented Apr 26, 2018 at 22:19
  • The infrastructure manage itself. It's fully documented, and we use Terraform & Puppet to orchestrate infrastructure and manage server configurations. So in reality, any puppet / terraform engineer with AWS experience can plug right in. I am now a shareholder in the business, and our dev team has grown significantly. Thankfully everyone now knows how the infrastructure flows
    – user2965205
    Commented Apr 28, 2018 at 13:52
4

You shouldn't hire a DevOps engineer because DevOps encompasses such a wide variety of disciplines that one individual cannot possibly be an expert in all aspects of these disciplines. By hiring a jack of all trades, you would be hiring a master of none.

DevOps is necessarily a team based endeavor and you cannot possibly expect one individual to support the expectations of an entire team. Consider the scope of DevOps. One person cannot possibly:

  • Be a rockstar developer in [language]
  • Be a networking guru, knowing all requisite RFCs
  • Be an exert Systems Administrator
  • Be an expert QA tester
  • Be a Database Administrator
  • Specialize in storage and backup
  • Know Site Reliability Engineering
  • Potentially other disciplines as well

Some of the above even have sub disciplines, such as Windows Systems Admin vs. Linux/Unix System admin or maybe you use more than one coding language in your.

No one person can possibly be an expert in all this which means that if you are adverting for a DevOps engineer, when the weakest area on your DevOps team is Networking, you are not doing a very good job of advertising your need for a networking specialist for your DevOps team. While no individual should be pigeon-holed to a specific role in the DevOps team, you would do your team a disservice by pretending there are no specialists or subject matter experts (SMEs) within the scope of DevOps. Swinging the pendulum from one extreme to the other - from silo-ing to pretending like every role on your DevOps team is the same - can cause just as many problems.

While having team members cross-train in more than one discipline - particularly in the overlapping areas is good, expecting them to be able to be proficient in such a wide volume of knowledge simply isn't practice.

This means that anyone who tells you they know all aspects of DevOps is probably lying to you. Hire a specialist in the area you are weakest in who has worked on a DevOps team - not a "DevOps Engineer."

Improve this answer
3
  • So all those job descriptions are just fluff to see who applies? They seem to want everything in the world. I look at it and think, I know this, this, that, and not that, not that, not that.... Maybe all business put the world up in description and see what they get.
    – johnny
    Commented Apr 9, 2019 at 20:33
  • 1
    @johnny I have heard tales of businesses requiring 7 years of experience in technologies that were created only 5 years ago... yes, they are wish lists. Not hard requirements.
    – James Shewey
    Commented Apr 9, 2019 at 20:54
  • Thanks. Wish list is the phrase I was looking for.
    – johnny
    Commented Apr 10, 2019 at 18:57
3

I had this exact challenge when implementing at ASOS. The goal for us is to have teams who are self sufficient and so having a dedicated role can be an anti-pattern, however we live in the real world and for many developers thinking about good DevOps practices is not their primary thing so they need help getting there.

What we did was:

  • lose the term of DevOps engineers, DevOps is something we should all do, not our job title so we called them something different.

  • rolled them out to teams but only 1 per 3, this meant they could not be a doer but had to be seen as a competency to help teams get themselves better and solve their own problems (with guidance)

  • maintained a central function as well to act as competency centre and handle the enterprise considerations, anything that affects all teams

As we evolve we review this model but for us it is working effectively so far

Improve this answer
3

I don't think you will be able to get a definitive answer for this because it seems like a lot of things factor in.

  • How on board the company is with the DevOps practice
  • What sort of application or service the company provides
  • The structure of your company

I just got done interviewing for positions and ended up with a title of DevOps engineer, but some of what I am doing is Sys Admin work. That is just out of necessity because of the size of the company and the nature of what is being managed application wise. Some positions I interviewed for had a similar title and they were looking more for someone from the development side of things experience wise. They expected more writing of code and not a sys admin who does automation. SRE seems to be a title gaining popularity so that might be the way to go. I had myself as System Administrator and Automation Engineer as my last job because I was writing ansible and chef stacks part of the time. The company was following a pretty good devops model where everyone was on board and devs worked alongside ops but I think their future didn't have dedicated infrastructure people in mind.

Now that I am in this position I am trying to shoe horn in some automation and we have some people problems we will have to sort out. People come and go and some of the workflows were designed because someone else designed it that way and not because it fits how people work.

So basically I think you should figure out the job description and not worry so much about the title unless its tied to pay somehow or you think one or the other would attract the right kind of people.

Improve this answer
1

If you are concerned with the meaning of DevOps, and following a "one true path". You shouldn't hire a DevOps Engineer. You should hire an Automation Engineer, or A Deployment engineer, or Platform Architect, or a number of other roles that do what you need.

If being a true DevOps practitioner isn't important to you, then you can call it whatever you want.

Improve this answer
2
  • 1
    Please expand a little on your position, this answer is kind of too short for the matter in this question.
    – Tensibai
    Commented May 22, 2017 at 15:27
  • 1
    @Tensibai - My only point is that it's just a title. Calling someone a DevOps engineer doesn't matter if you're not truly following DevOps prinicples
    – Erik Funkenbusch
    Commented May 22, 2017 at 15:37

Not the answer you're looking for? Browse other questions tagged or ask your own question.