I have used tar, dump, Amanda, Legato (EMC) Networker and other backup software for making backups of systems. However, I don't know the best tools for making backups in a CI/CD (Continuous Integration/Continuous Deployment) environment and doing "rolling recoveries" of systems "as you go" which is the case in a DevOps oriented system.
Many of these backup utilities are not necessarily best suited for a CI/CD environment because of the continuous changes taking place on both the development and production environment.
I'm looking at programs like Borg and Git based backups like Bup These programs would allow for many incremental backups and finer granularity of backups and work well with CD tools like Ansible. Even Ansible has a couple of way to make backups. However, there is no clean way to recover the most recent backup in an automated way.
Relying on some form of caching or mirroring of your content on the cluster and hope it's not corrupted is not a way to go :(
The things that would be backed up would be the config files of the servers, the database and uploaded content as these will almost always be unique and necessary. The containers, VMs, even the webapps would NOT be backed up as these should all be uniform and constantly updated under the CI/CD principles. Currently I use a combination of scp and tar into an archive directory sorted by date and machine. I like to find something better if possible.
I would like to know what are the best backup and recovery tools used in the CI/CD environment? and an example of how you would configure it?
I don't expect a "magic bullet" just some possible solutions.