How can DevOps help to improve Software Escrow procedures?

Question

Consider a software vendor and a licensed customer of some software of this vendor, wheres the software being licensed is either used on premise (at the customer's location), or in the format of SaaS solution (hosted by the vendor). However the customer only gets access to what's needed to use/run the software (executables and similar things), so not the source components and anything related to that to create the executables.

To protect the customer's business continuity in scenarios where something may go wrong with the vendor (eg: bankruptcy), both parties may agree to some sort of Software Escrow (SE ... 'also') agreement (also called Source code escrow). With such agreement both parties agree to get a 3rd party involved (= a "Software Escrow agent"), trusted by both parties. These are the highlights of such SE-agreement (= specs of the actual SE-process):

• ALL pieces of software components (artifacts related to the licensed software) get deposited by the software vendor, at some agreed location related to the SE-agent. Such deposits include the executables, but also the source components and anything related to that to create the executables (even documentation, instructions, etc to create the executables).
• Since the software vendor may create multiple releases for the duration of the software license, and the customer has the right to receive such new releases (as per the license agreement), part of such SE-agreement is that "with every major new release" (whatever "major" may mean ...), the deposit delivered to the SE-agent will also be updated/refreshed.
• If specific conditions are satisfied (e.g: bankruptcy of the vendor), then the Software Escrow agent will deliver to the licensed customer, upon request of such customer, of copy of everything that was deposited, so that customer will be able to continue using the software, and where needed even adapt the source code to continue to use it for the customer's business.

A common practice for such SE-agent to get involved, is some sort of a legal person/entity, such as a lawyer. But to actually "process the SE-deposits" (by the SE-agents), all sorts of release management and/or software delivery tasks need to be performed by somebody or something (the poor SE-agent) who probably doesn't know at all what the licensed software is supposed to do ... fun guaranteed!

My question:

How can DevOps help to improve Software Escrow procedures as described above? Like what kind of toolchain-tools would you recommend to be used for the fulfillment of which part of the SE-agreement? And where appropriate using which (preferably open source) software solutions for it?

Notes:

1. To not further complicate things, just assume that it is agreed between all parties involved, that the SE-agent does NOT have to do any type of "verifications" about the deposits being done. That is: whatever is deposited is assumed to be complete, up-to-date, documented, etc.

2. About "major new release": assume there are between 1 to 3 every year, which means that the licensed customer only expects to be able to get access (via the SE-agent) to those releases. Even though if there have been intermediate deliveries (like fixes or beta versions) to the licensed customer, those types of deliveries are considered out-of-scope. Even if it was only because:

   • the SE-agent charges "for each deposit to be processed by the SE-agent".
   • the licensed customer rarely changes releases, and is only interested in being able to use the SE-agreement if things go wrong, for the very release they are running at the moment things go wrong.

Answer 1

A very interesting question. On the assumption that the goal of a Software Escrow process is to allow for a 3^rd Party to take over or nominate an additional party to fulfil the responsibility of the software vendor, I would suggest the following elements of a DevOps operating model that would support software escrow:

1. Infrastructure-as-code - effectively documenting through an executable specification of dependant infrastructure, stored and versioned in source control provides the environments that the source code is developed in. Unlike static documentation in text files because it is executed on a regular basis by the software vendor to build their own environments it does not go out of date of suffer from "bit rot". This will always hold the most value when the entire development pipeline is built and maintained in source control applying Infrastructure-as-code principals.

2. Continuous Integration - the goal of continuous integration is to execute a set of steps that integrate the solution on a regular basis, ideally upon each change. Typically this means upon check-in and push to a central repository a set of tests are executed that validates the process. From a software escrow perspective I would expect this to also push a working version to a secondary "backup" repository owned and operated by the 3^rd parties. It is important to note that this does need to be both legally and financially "unlinked" from the software vendor.

3. Continuous Deployment - the goal of continuous deployment is to deliver software in a working state on a frequent basis. In this sense the 3^rd party is just another deployment target to deploy the outputs to, albeit probably not actively spinning up infrastructure on the 3^rd parties fabric.

Some other considerations to bring into the equation are:

• the move from static documentation to infrastructure as code significantly reduces the toil involved in updating documentation that describes how to install, configure and recover software.
• don't forget secrets management, such as X.509 Certificates, symmetric keys, passwords and licence keys, these could be stored in source control however this does have it's own drawbacks.

From a tools perspective this will depend very much upon the development environment, I don't believe there are any tools specific to software escrow however:

• .NET Development

  • GitHub or BitBucket for source control
  • AppVeyor, TeamCity or Jenkins for the build and test elements of the pipeline.
  • Ansible, Puppet or Chef for the configuration management elements of the pipeline.
  • ProGet for the package management elements of the pipeline
  • Octopus Deploy for the deployment aspects of the pipeline

• Java Development, and to some extend Open Source platforms

  • CircleCI, TravisCI, TeamCity or Jenkins for the build and test elements of the pipeline.
  • Ansible, Puppet or Chef for the configuration management elements of the pipeline.
  • JFrog Artifactory or Sonartype Nexus for the package management elements of the pipeline
  • Capistrano for the deployment aspects of the pipeline

At the end of the day if you are able to employ Infrastructure-as-code, Continuous Integration and Continuous Deployment principles to a software package it can be used to fulfil your obligations under a Software Escrow contract.

Answer 2

But to actually "process the SE-deposits" (by the SE-agents), all sorts of release management and/or software delivery tasks need to be performed by somebody or something (the poor SE-agent) who probably doesn't know at all what the licensed software is supposed to do ... fun guaranteed!

I wouldn't do business with an SE provider which would allow such unfortunate situation to exist - they don't know what they're doing.

If the SE deposits are to be processed in any manner by the SE agent the exact and complete processing procedure needs to be fully documented in the agreements so that it is actually usable.

That should also include the exact environment specification (or procedure to reproduce it) and the toolchain used for such processing. In other words the choice is not really open unilaterally to the SE agent. Except maybe for the actual storage strategy (personally I'd include that choice in the agreement specification as well, even if the choice is done unilaterally by the SE agent).

A good SE agreement would also ensure that each and every SE deposit by the vendor goes through that processing and the customer always gets and qualifies/signs off the result of that particular processing, not an alternate result coming directly from the vendor - to validate that the procedure itself remains up2date and effective for each and every SE deposit.

Otherwise the ability to reproduce the "SE withdrawals" at a later time (if/when needed) is questionable, which practically voids the whole SE story.