It is a fascinating question for which real answers may not actually exist; I appreciate that while you tried to keep the question contextualized on the VCS, it naturally scaled by itself up to infrastructure design and implementation planning.
Though, it seems many of us are working of this kind of transitioning, which can be exciting, but at the same time ...
I can answer bout docker, one of the best practice to use docker is to keep the docker file and the compose files in the same repository of the project, so wherever you clone the project you can build the docker image, and it is good to keep multiple versions of docker compose files for example (prod, staging, dev) so you can build the image and run the ...
No, it is not, the chef-repo is the legacy way back to chef 10.
Knife allows working on multiples directory with the -o option. Berks work from current directory.
You can also work around the default .chef directory by using the KNIFE_HOME environment variable too.
The current recomendations is indeed a repo per cookbook (or alongside another app)
Each tool's code goes into its own repo. for e.g
Jenkins Groovy template into a Jenkins repo
Ansible YAML playbooks in its own repo (with the roles , tasks , inventory sub directories
Cloudformation / Terrform templates in its own repo
Docker files in its own
5.. And So on
This would help you scale better in terms of process orchestration and maintaining ...
The current organisation of the code and configuration you describe is structured by the technical solutions involved. This is a bad design that will add a lot of overhead in our maintenance activities and will add a lot of traps in our way as well. Instead, that organisation should be structured around the artefacts we are deploying.
The reason for this ...
The layout of your repository depends in many ways upon the context you are developing the automation in. If, for example, you are building out the infrastructure for a product as part of a product team, then it would make sense to tie the infrastructure to the product - i.e. keep the infrastructure in the same repository as the software source code.
I do not believe that there is a way to set this as a global policy. However, there is a simple command you can run that will "prune" or remove your unused images.
You will want to use docker image prune --filter [options] to achieve this. Check out the official documentation for information on how to best filter. As a best practice you should be tagging ...
I don't really understand this one:
However, this seems clunky, as I have to create the logic to read this
file in every script that is made.
You can source all variables from a bash file in one line (source file-name.sh). If they are "exported", you can pick them up in applications in any language you like (Java, python, or whatever), ...
Making a distinction between Ops, Dev and DevOps promotes isolation and enforces a "throw it over the wall" mindset. To increase cooperation between teams one should put everything in a repository that is required to build and deploy the project.
Having said that, the answer to the question:
How to structure DevOps related code and configs in a code ...
Docker images are after all VM templates, i.e. they have to be more or less self-contained: you get the image, you run the environment with all the dependencies.
To approach the challenge operationally, Docker supports reuse of environments through the FROM statement i.e. you could maintain a base image and on top a much smaller image with the app itself.
It is wise to think about package management early on.
There are multiple options in terms of package management. The 2 largest are jFrog Artifactory and Sonatype Nexus. Both offer support for multiple formats such as maven, NuGet, Docker, npm, ...
If you only have standard requirements, both of these will suffice. If you have specific requirements, it's ...
So I ended up using this simple shell script
docker images -q --filter "before=ecr.us-east-1.amazonaws.com/myapp:02e56bac4ee3f27a4d5670a6ac0d578e3dd5b2e5" | xargs docker rmi --force
First you filter your images to list only those created before the latest one (you either know your recent tag or just use latest one)
The you just remove those images with ...
It turns out that /etc/ssl/cert.pem is the bundle scanned by apk to verify a repo behind HTTPS, so the following works:
cat CA.crt >> /etc/ssl/cert.pem
sed -i 's,http://dl-cdn.alpinelinux.org/alpine,https://nexus-endpoint.company.com/repository/alpine-proxy,' /etc/apk/repositories
If the repo is on AWS Codecommit, you should be able to access it via a role setup for each type of instance.
See under IAM role here https://docs.aws.amazon.com/codecommit/latest/userguide/auth-and-access-control.html
Another option is to use saltstack's product: salt
This Configuration Management System (CMS) can protect sensitive data in three ways.
1) Pillars; these are securely stored and encrypted key:value or key:list dictionaries stored on your primary computer(s) that you manage the entire cluster from.
This will allow you to create salt scripts that can have ...
You could place your private repo in the same VPC as the instance and then pull code from there. Just make sure the repo cannot be accessed from outside the VPC.
Ssh into the new instance and SCP the binary over.
Use docker and deploy a private container registry (e.g. Amazon ECR). Build the image on your machine / build ...
Most repository managers have some way of implementing this.
GitHub (and GitHub Enterprise) has the idea of statuses on a commit; you'd set up a webhook that pings your tool's webserver when a commit is pushed, and then it would use the status API to set whether or not it passes the version check.
Bitbucket, when self-hosted, supports plugins to integrate ...