Skip to main content
added 4237 characters in body
Source Link
Pierre.Vriens
  • 7.2k
  • 14
  • 37
  • 84

Your question doesn't seem to make any assumption about the platform/OS it is about. Which is why it may make sense to add an answer about how this is typically done/addressed in a mainframe environment, where the "engineers" (as in your question title) are actually groups of people were dozens (possibly hundreds) of people are involved. And this byMy answer is based on using ChangeMan ZMF (disclosure: the SCM product where I'm most familiar with (not sure if it's needed to disclose the product name).


### 1. Architecture
  • All code (and related artifacts like executables, etc) is stored in files which, all together is what we call the library structure.
  • For each environment on each (possibly remote) target system, there is a server (a "started task" in mainframe speak), which takes care of ALL (repeat: ALL) updates to anything in the library structure. There are a few exceptions (like security people, or space management team), but apart from that, nobody (repeat: nobody) has authorization to apply updates to any file within that library structure. In other words: the server gets exclusive update authority to the entire library structure. Attention: OPS-people will go bonkers if you walk in to limit their access (at first they are going to resistsresist ...), so make sure you're covered by upper management (CxO) to impose those access rules ...
  • The actual software changes my consist of a single component (a tiny code fix in the middle of the night ...), or it may also be hundreds or thousands of sources, executables, or whatever other artifacts (during a release weekend). To make them manageable, things that should be moved (more or less) together, at the same time, are bundled together in what's called a software change package.
  • Only the server will execute the required (and preconfigured) steps.
  • The server will only do a specific step (= update something somewhere in the library structure), after the required approvals (from human beings) have been gathered to perform such step.
  • The approvals can only be given by users who have a rolerole that allows them (= permission) to issue such approvals.

### 2. Roles and permissions
The server will ensure that the user trying to make something happen(like 'approve something') will only be able to do so, if the user's permissions are appropriate. That part is easy. But you don't want to use the SCM system to administer all those permissions for all the users involved, that's what belongs in your security system(not the SCM system!), so that you can adapt your workflow(in your SCM system) to go check those permissions whenever appropriate. The steps below provide some more details on that.

Step 1: Configure the permissions (in the security system)

  • The roles

    Define security entities in your security system, with well defined names for those entities. A few samples (add as many similar ones to fit your own needs):

    • PrmUnit, used for getting permission to request a Promote to say Unit-testing.
    • PrmQA, used for getting permission to request a Promote to say Qa-testing (let's assume this is the highest level of testing).
    • PrdEnduser, used by end-users involved in some level of testing, to indicate that they are satisfied by the results produced by some kind of testing. And because of that, those end-users agree with the change moving forward in the library structure.
    • PrdRelmgnt, used by release managers to authorize an Activation in production (= the last/highest level in the library structure).
  • Define groups of users in your security system. A few samples (add as many similar ones to fit your own needs):

    • GrpDevs, which (say) corresponds to your developers (probably more then just 1).
    • GrpEnduser, which (say) corresponds to your end-users (at least 1, preferably with more similar users).
    • GrpRelMgnt, which (say) corresponds to your release managers (at least 1, preferably a few more users).
  • Grant permissions, also using your security system, to allow access to selected "security entities" for selected "groups of users". To continue the example above, here is what seems appropriate (adapt to fit your own needs):

    • Group GrpDevs gets access to (only!) security entity PrmUnit.
    • Group GrpEnduser gets access to (only!) security entity PrdEnduser.
    • Group GrpRelMgnt gets access to (both!) security entity PrmQA and PrdRelmgnt.

Step 2: Configure the workflow (in the SCM system)

After the permissions are configured in your security system (as in Step 1), all that's left to do in your SCM system is to configure how the various steps in the lifecyle match with the related security entities in your security system. That is, only those users who have the appropriate access to the required security entity, are allowed to request the server to perform the corresponding step in the workflow.

Here are some examples of how you'd configure your SCM system to make some magic happen:

  • If a user has access to PrmUnit, then such user is allowed to request a Promote to Unit-testing. Obviously, the users in group GrpDevs are the users authorized for this (permissionsnote: not, e.g, the users in group GrpRelMgnt) required.
  • If a user has access to PrmQA, then such user is allowed to request a Promote to QA-testing. Obviously, the users in group GrpRelMgnt are the users authorized for this (note: not, e.g, the various steps may varyusers in group GrpDevs, but these permissions are definedor in the security system being usedgroup GrpEnduser).
  • If a user has access to PrdEnduser, andthen such user is allowed to authorize the server will ensure thatchange moving forward in the user tryinglibrary structure (which is typically a prereq for users in group GrpRelMgnt to approve something will onlyeven be able to do soreview a change). Obviously, if the user's permissionsusers in group GrpEnduser are appropriatethe (only) users authorized for this.
  • If a user has access to PrdRelmgnt, then such user is allowed to authorize an Activation in production (= the last/highest level in the library structure).

To complete the picture (workflow) as explained above, the server creates an audit trail (logging) of anything that's happening in the system. So that at any point in time, it is always possible to answer questions like "What happened when and why, and who authorized it?".


### 3. Expect the unexpected, and be ready for it

The above is just a blueprint, which hopefully helps to understand how in the end it is the server that takes care of the segregation of duties ... provided you have the CxO cover you to impose some access rules that not everybody will like. But

To complete the picture as explained above, those measures help quite a lot to keep a bank orthe server creates an airline goingaudit trail .(logging) of anything that's happening in the system. So that at any point in time, it is always possible to answer questions like

What happened when and why, and which authorized user actually approved it ... upfront?

But, probably the toughest part is to have adequate reporting tools available (and know how to use them). AndAt least to (easily) satisfy requests from IT auditors love these kinds(their questions can be very challenging). But also to point to relevant log records in your SCM system to answer all sorts of solutions"What happened"-questions in crisis situations where (part of) production is down.


PS: I leave it to everybody's own judgement if my answer is yes or no DevOps-compliant.

Your question doesn't seem to make any assumption about the platform/OS it is about. Which is why it may make sense to add an answer about how this is typically done/addressed in a mainframe environment, where the "engineers" (as in your question title) are actually groups of people were dozens (possibly hundreds) of people are involved. And this by using ChangeMan ZMF (disclosure: the SCM product where I'm most familiar with).

  • All code (and related artifacts like executables, etc) is stored in files which, all together is what we call the library structure.
  • For each environment on each (possibly remote) target system, there is a server (a "started task"), which takes care of ALL (repeat: ALL) updates to anything in the library structure. There are a few exceptions (like security people, or space management team), but apart from that, nobody (repeat: nobody) has authorization to apply updates to any file within that library structure. In other words: the server gets exclusive update authority to the entire library structure. Attention: OPS-people will go bonkers if you walk in to limit their access (at first they are going to resists ...), so make sure you're covered by upper management (CxO) to impose those access rules ...
  • The actual software changes my consist of a single component, or it may also be hundreds or thousands of sources, executables, or whatever other artifacts. To make them manageable, things that should be moved (more or less) together, at the same time, are bundled together in what's called a software change package.
  • Only the server will execute the required (and preconfigured) steps.
  • The server will only do a specific step, after the required approvals (from human beings) have been gathered.
  • The approvals can only be given by users who have a role that allows them to issue such approvals.
  • The roles (permissions) required for the various steps may vary, but these permissions are defined in the security system being used, and the server will ensure that the user trying to approve something will only be able to do so, if the user's permissions are appropriate.

To complete the picture (workflow) as explained above, the server creates an audit trail (logging) of anything that's happening in the system. So that at any point in time, it is always possible to answer questions like "What happened when and why, and who authorized it?".

The above is just a blueprint, which hopefully helps to understand how in the end it is the server that takes care of the segregation of duties ... provided you have the CxO cover you to impose some access rules that not everybody will like. But, those measures help quite a lot to keep a bank or an airline going ... And IT auditors love these kinds of solutions.

Your question doesn't seem to make any assumption about the platform/OS it is about. Which is why it may make sense to add an answer about how this is typically done/addressed in a mainframe environment, where the "engineers" (as in your question title) are actually groups of people were dozens (possibly hundreds) of people are involved. My answer is based on using the SCM product where I'm most familiar with (not sure if it's needed to disclose the product name).


### 1. Architecture
  • All code (and related artifacts like executables, etc) is stored in files which, all together is what we call the library structure.
  • For each environment on each (possibly remote) target system, there is a server (a "started task" in mainframe speak), which takes care of ALL (repeat: ALL) updates to anything in the library structure. There are a few exceptions (like security people, or space management team), but apart from that, nobody (repeat: nobody) has authorization to apply updates to any file within that library structure. In other words: the server gets exclusive update authority to the entire library structure. Attention: OPS-people will go bonkers if you walk in to limit their access (at first they are going to resist ...), so make sure you're covered by upper management (CxO) to impose those access rules ...
  • The actual software changes my consist of a single component (a tiny code fix in the middle of the night ...), or it may also be hundreds or thousands of sources, executables, or whatever other artifacts (during a release weekend). To make them manageable, things that should be moved (more or less) together, at the same time, are bundled together in what's called a software change package.
  • Only the server will execute the required (and preconfigured) steps.
  • The server will only do a specific step (= update something somewhere in the library structure), after the required approvals (from human beings) have been gathered to perform such step.
  • The approvals can only be given by users who have a role that allows them (= permission) to issue such approvals.

### 2. Roles and permissions
The server will ensure that the user trying to make something happen(like 'approve something') will only be able to do so, if the user's permissions are appropriate. That part is easy. But you don't want to use the SCM system to administer all those permissions for all the users involved, that's what belongs in your security system(not the SCM system!), so that you can adapt your workflow(in your SCM system) to go check those permissions whenever appropriate. The steps below provide some more details on that.

Step 1: Configure the permissions (in the security system)

  • Define security entities in your security system, with well defined names for those entities. A few samples (add as many similar ones to fit your own needs):

    • PrmUnit, used for getting permission to request a Promote to say Unit-testing.
    • PrmQA, used for getting permission to request a Promote to say Qa-testing (let's assume this is the highest level of testing).
    • PrdEnduser, used by end-users involved in some level of testing, to indicate that they are satisfied by the results produced by some kind of testing. And because of that, those end-users agree with the change moving forward in the library structure.
    • PrdRelmgnt, used by release managers to authorize an Activation in production (= the last/highest level in the library structure).
  • Define groups of users in your security system. A few samples (add as many similar ones to fit your own needs):

    • GrpDevs, which (say) corresponds to your developers (probably more then just 1).
    • GrpEnduser, which (say) corresponds to your end-users (at least 1, preferably with more similar users).
    • GrpRelMgnt, which (say) corresponds to your release managers (at least 1, preferably a few more users).
  • Grant permissions, also using your security system, to allow access to selected "security entities" for selected "groups of users". To continue the example above, here is what seems appropriate (adapt to fit your own needs):

    • Group GrpDevs gets access to (only!) security entity PrmUnit.
    • Group GrpEnduser gets access to (only!) security entity PrdEnduser.
    • Group GrpRelMgnt gets access to (both!) security entity PrmQA and PrdRelmgnt.

Step 2: Configure the workflow (in the SCM system)

After the permissions are configured in your security system (as in Step 1), all that's left to do in your SCM system is to configure how the various steps in the lifecyle match with the related security entities in your security system. That is, only those users who have the appropriate access to the required security entity, are allowed to request the server to perform the corresponding step in the workflow.

Here are some examples of how you'd configure your SCM system to make some magic happen:

  • If a user has access to PrmUnit, then such user is allowed to request a Promote to Unit-testing. Obviously, the users in group GrpDevs are the users authorized for this (note: not, e.g, the users in group GrpRelMgnt).
  • If a user has access to PrmQA, then such user is allowed to request a Promote to QA-testing. Obviously, the users in group GrpRelMgnt are the users authorized for this (note: not, e.g, the users in group GrpDevs, or in group GrpEnduser).
  • If a user has access to PrdEnduser, then such user is allowed to authorize the change moving forward in the library structure (which is typically a prereq for users in group GrpRelMgnt to even be able to review a change). Obviously, the users in group GrpEnduser are the (only) users authorized for this.
  • If a user has access to PrdRelmgnt, then such user is allowed to authorize an Activation in production (= the last/highest level in the library structure).

### 3. Expect the unexpected, and be ready for it

The above is just a blueprint, which hopefully helps to understand how in the end it is the server that takes care of the segregation of duties ... provided you have the CxO cover you to impose some access rules that not everybody will like.

To complete the picture as explained above, the server creates an audit trail (logging) of anything that's happening in the system. So that at any point in time, it is always possible to answer questions like

What happened when and why, and which authorized user actually approved it ... upfront?

But, probably the toughest part is to have adequate reporting tools available (and know how to use them). At least to (easily) satisfy requests from IT auditors (their questions can be very challenging). But also to point to relevant log records in your SCM system to answer all sorts of "What happened"-questions in crisis situations where (part of) production is down.


PS: I leave it to everybody's own judgement if my answer is yes or no DevOps-compliant.
Source Link
Pierre.Vriens
  • 7.2k
  • 14
  • 37
  • 84

Your question doesn't seem to make any assumption about the platform/OS it is about. Which is why it may make sense to add an answer about how this is typically done/addressed in a mainframe environment, where the "engineers" (as in your question title) are actually groups of people were dozens (possibly hundreds) of people are involved. And this by using ChangeMan ZMF (disclosure: the SCM product where I'm most familiar with).

Here are the highlights of how I'd answer your question:

  • All code (and related artifacts like executables, etc) is stored in files which, all together is what we call the library structure.
  • For each environment on each (possibly remote) target system, there is a server (a "started task"), which takes care of ALL (repeat: ALL) updates to anything in the library structure. There are a few exceptions (like security people, or space management team), but apart from that, nobody (repeat: nobody) has authorization to apply updates to any file within that library structure. In other words: the server gets exclusive update authority to the entire library structure. Attention: OPS-people will go bonkers if you walk in to limit their access (at first they are going to resists ...), so make sure you're covered by upper management (CxO) to impose those access rules ...
  • The actual software changes my consist of a single component, or it may also be hundreds or thousands of sources, executables, or whatever other artifacts. To make them manageable, things that should be moved (more or less) together, at the same time, are bundled together in what's called a software change package.

With the above in place, any kind of update to be applied by the server to the library structure, will only be possible via a well defined workflow, which we call the lifecycle of a software change package (SDLC if you prefer). To actually execute the various steps in that workflow, this is what it takes to make it happen:

  • Only the server will execute the required (and preconfigured) steps.
  • The server will only do a specific step, after the required approvals (from human beings) have been gathered.
  • The approvals can only be given by users who have a role that allows them to issue such approvals.
  • The roles (permissions) required for the various steps may vary, but these permissions are defined in the security system being used, and the server will ensure that the user trying to approve something will only be able to do so, if the user's permissions are appropriate.

To complete the picture (workflow) as explained above, the server creates an audit trail (logging) of anything that's happening in the system. So that at any point in time, it is always possible to answer questions like "What happened when and why, and who authorized it?".

The above is just a blueprint, which hopefully helps to understand how in the end it is the server that takes care of the segregation of duties ... provided you have the CxO cover you to impose some access rules that not everybody will like. But, those measures help quite a lot to keep a bank or an airline going ... And IT auditors love these kinds of solutions.