Your question doesn't seem to make any assumption about the platform/OS it is about. Which is why it may make sense to add an answer about how this is typically done/addressed in a mainframe environment, where the "engineers" (as in your question title) are actually groups of people were dozens (possibly hundreds) of people are involved. And this byMy answer is based on using ChangeMan ZMF (disclosure: the SCM product where I'm most familiar with (not sure if it's needed to disclose the product name).
### 1. Architecture
- All code (and related artifacts like executables, etc) is stored in files which, all together is what we call the library structure.
- For each environment on each (possibly remote) target system, there is a server (a "started task" in mainframe speak), which takes care of ALL (repeat: ALL) updates to anything in the library structure. There are a few exceptions (like security people, or space management team), but apart from that, nobody (repeat: nobody) has authorization to apply updates to any file within that library structure. In other words: the server gets exclusive update authority to the entire library structure. Attention: OPS-people will go bonkers if you walk in to limit their access (at first they are going to resistsresist ...), so make sure you're covered by upper management (CxO) to impose those access rules ...
- The actual software changes my consist of a single component (a tiny code fix in the middle of the night ...), or it may also be hundreds or thousands of sources, executables, or whatever other artifacts (during a release weekend). To make them manageable, things that should be moved (more or less) together, at the same time, are bundled together in what's called a software change package.
- Only the server will execute the required (and preconfigured) steps.
- The server will only do a specific step (= update something somewhere in the library structure), after the required approvals (from human beings) have been gathered to perform such step.
- The approvals can only be given by users who have a rolerole that allows them (= permission) to issue such approvals.
### 2. Roles and permissions
The server will ensure that the user trying to make something happen(like 'approve something') will only be able to do so, if the user's permissions are appropriate. That part is easy. But you don't want to use the SCM system to administer all those permissions for all the users involved, that's what belongs in your security system(not the SCM system!), so that you can adapt your workflow(in your SCM system) to go check those permissions whenever appropriate. The steps below provide some more details on that.
Step 1: Configure the permissions (in the security system)
- The roles
Define security entities in your security system, with well defined names for those entities. A few samples (add as many similar ones to fit your own needs):
PrmUnit
, used for getting permission to request a Promote to say Unit-testing.PrmQA
, used for getting permission to request a Promote to say Qa-testing (let's assume this is the highest level of testing).PrdEnduser
, used by end-users involved in some level of testing, to indicate that they are satisfied by the results produced by some kind of testing. And because of that, those end-users agree with the change moving forward in the library structure.PrdRelmgnt
, used by release managers to authorize an Activation in production (= the last/highest level in the library structure).
Define groups of users in your security system. A few samples (add as many similar ones to fit your own needs):
GrpDevs
, which (say) corresponds to your developers (probably more then just 1).GrpEnduser
, which (say) corresponds to your end-users (at least 1, preferably with more similar users).GrpRelMgnt
, which (say) corresponds to your release managers (at least 1, preferably a few more users).
Grant permissions, also using your security system, to allow access to selected "security entities" for selected "groups of users". To continue the example above, here is what seems appropriate (adapt to fit your own needs):
- Group
GrpDevs
gets access to (only!) security entityPrmUnit
. - Group
GrpEnduser
gets access to (only!) security entityPrdEnduser
. - Group
GrpRelMgnt
gets access to (both!) security entityPrmQA
andPrdRelmgnt
.
- Group
Step 2: Configure the workflow (in the SCM system)
After the permissions are configured in your security system (as in Step 1), all that's left to do in your SCM system is to configure how the various steps in the lifecyle match with the related security entities in your security system. That is, only those users who have the appropriate access to the required security entity, are allowed to request the server to perform the corresponding step in the workflow.
Here are some examples of how you'd configure your SCM system to make some magic happen:
- If a user has access to
PrmUnit
, then such user is allowed to request a Promote to Unit-testing. Obviously, the users in groupGrpDevs
are the users authorized for this (permissionsnote: not, e.g, the users in groupGrpRelMgnt
) required. - If a user has access to
PrmQA
, then such user is allowed to request a Promote to QA-testing. Obviously, the users in groupGrpRelMgnt
are the users authorized for this (note: not, e.g, the various steps may varyusers in groupGrpDevs
, but these permissions are definedor in the security system being usedgroupGrpEnduser
). - If a user has access to
PrdEnduser
, andthen such user is allowed to authorize the server will ensure thatchange moving forward in the user tryinglibrary structure (which is typically a prereq for users in groupGrpRelMgnt
to approve something will onlyeven be able to do soreview a change). Obviously, if the user's permissionsusers in groupGrpEnduser
are appropriatethe (only) users authorized for this. - If a user has access to
PrdRelmgnt
, then such user is allowed to authorize an Activation in production (= the last/highest level in the library structure).
To complete the picture (workflow) as explained above, the server creates an audit trail (logging) of anything that's happening in the system. So that at any point in time, it is always possible to answer questions like "What happened when and why, and who authorized it?".
### 3. Expect the unexpected, and be ready for it
The above is just a blueprint, which hopefully helps to understand how in the end it is the server that takes care of the segregation of duties ... provided you have the CxO cover you to impose some access rules that not everybody will like. But
To complete the picture as explained above, those measures help quite a lot to keep a bank orthe server creates an airline goingaudit trail .(logging) of anything that's happening in the system. So that at any point in time, it is always possible to answer questions like
What happened when and why, and which authorized user actually approved it ... upfront?
But, probably the toughest part is to have adequate reporting tools available (and know how to use them). AndAt least to (easily) satisfy requests from IT auditors love these kinds(their questions can be very challenging). But also to point to relevant log records in your SCM system to answer all sorts of solutions"What happened"-questions in crisis situations where (part of) production is down.
PS: I leave it to everybody's own judgement if my answer is yes or no DevOps-compliant.