Is local / staging / production still industry standard?

Question

I've inherited a project that uses local / staging / production.

It is a simple project that lets users create a login and download customized PDF documents. We keep the document templates and the per-user-customisations in S3.

But I've heard that githubflow allows direct deploy-to-production--upon--merge-PR-to-master.

The project comprises a React front-end and a Django backend. The backend is hosted on AWS and administered through Kubernetes. There is also a database and S3 bin on AWS.

The infrastructure was never completed, so I'm in the process of automating deploy-to-{staging/production}, and wondering if I'm being a dinosaur.

My thought is that if deploy-to-production automatically backs up the database and S3, this could give a faster/more-efficient workflow.

Answer 1

When you say "Industry Standard" that's very much going to depend on what industry.

You can drop code into production however you want (legally restricted industries obviously being the exemption to this), but the question you should probably be examining is if you should be doing that.

I've recently worked on products which range from having incredibly test coverage, huge amounts of infrastructure set up to perform every imaginable regression test against any integration point that exists, and even though it was not the done thing with that particular product, after passing all of the automated testing required for a branch to be mergable, I probably would have been pretty comfortable launching straight to production. On the other end of the spectrum I've also worked on projects which arguably were developed in production given that the production instances were so dissimilar to what was actually checked in to source control or running in any test environment and, of course, no test cases to be seen.

The standard approach to making production deployments as safe as possible is to try and ensure that the thing you are planning to deploy is as rigorously tested as possible, and to do that most of the testing will need to be automated, otherwise you'll just end up spending your life testing.

Additionally, most modern systems use standard patterns to make a production roll back an extemely frictionless task, if not to automate it entirely. If you are deploying canaries of your software, or you are running a blue/green deployment and can have built in mechanisms to roll back if certain success conditions aren't met, then it further reduces the risk of a production change breaking something.

Essentially, I would question anyone telling you how to manage and derisk your production releases unless they are just as intimate with the software as you are.

Answer 2

In the software industry, it is popular to have staging environments but in order to understand why it is important to understand the reasons why these environments exist in the first place.

The production environment is where the company's software meets the company's users. Most companies want their users to be happy, and to receive high-quality software to use that meets their expectations.

On the other hand, a development environment is where developers are creating the software. In most cases, the quality of this software when first created is far from good. Even great developers make mistakes from time to time, and they need some safety net to catch these problems so they can fix them.

Improving software quality is an iterative process. Developers are tasked with constantly creating more and more changes, it is their job. These changes need to become high-quality, and the only way of doing that is to test the changes somewhere until all the wrinkles are polished. This is where the QA, Staging, User Acceptance environments come in. These environments are where software can be checked and any problems caught before it is being put in front of customers.

All of the above describes how having multiple environments is used to reduce the risk of bad code in production, while at the same time allowing developers to have a place where they can make mistakes safely and fix these.

Many companies have additional rules that make it practical to have multiple environments. For example, having a tradition to release software on a "special date" for everyone to use as Apple does. They still have beta testers and other testing happening before this release - which can be considered a staging or user-acceptance environment.

In some industries which are heavily regulated, there are restrictions on who can have access to the "production environment". In many cases only designated people are allowed to touch these environments because of security and legal concerns. Where in such a company there are hundreds of other developers who are not designated, they need someplace to integrated and test their code - the staging environment is less strict regarding who is allowed to change things in the environment.