Does QA in DevOps roles means CI/CD or more?

Question

One of views on DevOps is to say that it combines fileds of Development, Operations and Quality Assurance.

Now many DevOps roles have focus either

• on development with the left shift in terms of operations awareness (up to a No-Ops full stack, you build it - you run it (c) Besos, Amazon's CTO),
• or a sysadmin who is expected to write CaSC (Configurarion as Code), IaC (infrastructure as code) and automation codes to reduce work required in operations.

In both cases you see lots of tools typical for these roles with and without DevOps context.

What does QA DevOps roles do today as a formal commitment? Test automation always has been there, for example. Is QA just meant here to support CI/CD infrastructure and teams working on it, or something else?

Answer 1

First, DevOps is a journey, not a destination. Not every organization approaches similar problems with the same solution; while automation is a component of a DevOps platform, it's not the single defining characteristic.

The role and responsibilities of QA professionals haven't really changed; they're still responsible for validating that code is ready for release. What HAS changed is the focus on both automation and smaller releases. Much like operational roles, QA staff should now have the opportunity to focus on adding additional automatic quality checks into the development life cycle.

In a shop with a mature CI/CD pipeline, QA becomes more about designing great tests, and letting the software do the actual heavy lifting. As Tensibai commented above, "some QA tests can't be automated" (yet), so there will still need to be some engineering responsibilities, but the goal should be to utilize automation in every aspect of delivery in order to facilitate adding additional quality.

Answer 2

This is an open-ended question, which is why the following answer is an opinion.

The focus of QA (quality assurance) in the DevOps field is to monitor the deployment metrics to ensure that faults are caught before they can manifest. The most direct metrics of application health include application availability and errors. These can be directly extracted from error logs and automated monitoring of served web pages (that status code 200 or the login form are always returned).

The next level allows you to anticipate errors, bottlenecks and resource usage. This is very deployment specific. In some cases you may have a micro service that receives a high load or in others a single page may make several database queries. Monitoring your servers' CPU load and memory usage can also be used as metrics. These may act as key indicators to your applications' health.

Other aspects delve into feature and device usage which enable you to make informed decisions about depreciating these in order to limit your application's complexity. Since complexity is directly linked to bugs and development velocity, it makes sense to limit it where possible.

Sources: Telerik and Stackify

Answer 3

The key aspect in a Dev(Sec)Ops-oriented organisation is the inter-field communication and collaboration. From this perspective, the QA role can include activities such as:

• continuous improvement of automated QA coverage/effectiveness, reliability, performance (which ideally should target CI/CD, but not always is/can be)
• direct, active/constant involvement with Development:
  • for TDD-based development
  • for reproducing difficult customer-found defects, developing strategies for verifying their fixes and plugging the holes that allowed such bugs to reach the customer
• feature, roadmap, resource planning, in which all fields should be involved

For organisations building software delivered for deployment and operation by 3rd party customers (like IT infrastructure equipment or consumer products, for example) the QA role could bring their expertise (ideally alongside Development) in customer-facing interactions (maybe just in an assisting role) to improve the understanding of how the customer is using the product and the challenges they face and bring that feedback into the aspects of the product lifecycle under their influence.

For organisations which also deploy and operate the software being produced the QA role may extend to/blend into the Security/Operations field while closing the loop back towards Development and itself, for example:

• ensuring the product quality matches expectations
• managing SLAs
• constantly aiming for reducing deltas between the production environment and the Development/QA one(s), regardless of them being inside of the CI/CD pipeline or not