Ways to the DevOps hiring process optimization through CALMS paradigma?

Question

Much of DevOps recruitement happens to follow along the lines of keyword matching, which leads in my opinion to solely technology focus.

Now, DevOps is about so much more than just technology, and DevOps Engineer is not just a better system administrator with some coding skills.

Senior DevOps role/profile means to me also offering seniority in many other foundations and practices beyond infrastructure and software engineering skills like Lean, Measurement and being open and communicative (who asks DevOps hires for their communication skills, honestly?!)

So, can a job ad / interview be more efficient in some way - for example, by applying questioning CALMS categories as well? - Leading to questions like "now, how do you apply lean principles? How were cultural aspects addressed in your recent DevOps projects?"

Further elaboration:

• Culture (e.g. strategies for conflict management and attitude to failures, own and others')
• Automation (here you ask about Puppet/Docker etc skills)
• Lean (foundations of Lean? Waste types?)
• Measurement (ask for tools like JMeter but go also to things like sampling, data modeling..)
• Sharing (obviously knowledge management and according tools)

UPDATE - so why wouldn't employers/recruiters structure the interview by CALMS as shown below (additionally, the "automation" section could be formulated along the DevOps toolchain model (document link, readonly)?

Side note - so culture for example is actually not just a soft skill anymore, for DevOps it is one of the core skills - like all the others in this domain.

Answer 1

This is a brilliant idea, also because of Daniel Kahneman who showed that if you split a single score into 5 weighed scores and add numerical criteria and bounds to those, you will significantly reduce bias. You could design not just the resume scoring, but the entire hiring process, with phone screens, onsite interviews, everything in this way. It would significantly reduce the inherent bias of the interviewers. We have actually started to do something similar for all hiring.

Obviously, inside each area, your should add weight to what is important to the company for the position, but you are hiring a well-rounded engineer and you want someone who will propose major changes to how your organization operates, you are not simply hiring someone for specific skills to work in a limited area. Many people simply see this role as a higher paid Release and Build Engineer and if that is the case, that is what you should hire and advertise for.

For a DevOps hire, I would suggest replacing the Lean with Learning. It is originally CAMS and even though some extend it to CALMS to include Lean, that is somewhat restricting as DevOps is based on so much more than just Lean. It is also Deming's ideas about Special and Common Cause Variation and System Thinking, Nash's Equilibrium (if each optimizes for themselves, the result could be suboptimal, compared to when everyone includes the interest of the group), Shewhart's Statistical Process Control, Goldratt's Theory of Constraints, Taleb's Anti-Fragility and so many more.

This would also allow you to include participation in conferences in Learning and presentations at conferences or meetups as Sharing. In a position where you are not always a part of a team or your company might not be big enough to have your peers as your co-workers, it is even so much more important to establish and maintain out of workplace relationships and learning opportunities. We generally grouped those two under Culture.

I would personally put under Culture the soft skills required to be effective in improving processes at your organization. CMMI, Kanban, Work in Progress limits, Agile practices, etc.

JIRA seems more like Sharing tool and Git is more closely related to Automation.

Answer 2

EDIT

I believe that this depends from organisation to organisation and what a DevOps / Senior DevOps is expected to do, therefore, your first sentence is 100% accurate. Because, a DevOps should be able to use a set of tools that the company uses and also improve or bring new set of tools which enables the company and its developers to work faster and waste less.

In my opinion a DevOps should have strong SysAdmin skills and obviously coding skills as Puppet, Chef, Python, Bash will be used extensively as well as some knowledge of the code that goes on the servers at least to be able to do minor debugging on why the application doesn't behave as expected from one environment to the other.

Now, as a Senior DevOps, CALM may be applied however, the Lean and Measurement principles might / might not apply. For example we are developing applications using Chef / Puppet / Ansible to automate the mundane things and keep everything in sync which obviously saves time and produces less waste.

Regarding measurement, I am not sure if that's applicable in most of the cases. However, the other CALM principles are part of a DevOps position.

Having good communication skills is also important as a DevOps, but more important as a Senior DevOps because you will have to not only deal with your team and share knowledge and with the developers as you are there to support them, but also you might have to create reports and keep presentations in front of the management.

I like the spreadsheet you added, and is good to have a point system, however, some of the companies are also adding more skills / technologies in a job advert than required.

Also, after a telephone interview (if there's one) I'd find it useful that in an interview you will be given some problems to resolve or at least to show your debugging process and how you will behave in given situations. Personally, I dislike written tests as I believe there are 'n' ways of solving a problem, and also, sometimes Google is your friend, as you are not expected to know everything by heart.

As a DevOps / Senior DevOps I believe there's a line between the applications that are used and knowledge. You could be amazing at using these new / old tools or write code, but when it comes to debugging or just understanding what the problem with a server, Jenkins job might be you are not able to do so.

Finally, the spreadsheet presented I think is a way of assessing a DevOps knowledge also for a Senior position I might add there some Interpersonal and Management skills to make it complete.

And when it comes to the selection process you could have a look at the spreadsheet and pick the person with a score that you believe is the right one for your organisation as well as keeping in mind his(er) behaviour during the interview and the way (s)he presented / answered those questions.