Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
7

Vagrant is for setting up test environments right? Ansible/Chef etc. seem good for servers you own, but what about cloud? Docker... great for standardizing app, so why use Ansible/Chef? Kubernentes.. good for deploying to multiple environments and scaling Just a personal note: the way you formulate these things makes me assume that you are diving freshly ...


7

Sit and watch the pipeline run? No, that is not how you work efficiently. Developers push their commits to the source control repository and then the CI/CD pipeline is triggered. Developers may post a well-written pull request anytime they want. There is usually a visual mark representing an "on-going build"/"failed build"/"successful build". Typically a ...


6

There are lots of bad examples on the Internet. Doing a git pull to distribute code is fabulous for development, but prone to all sorts of issues in practice. It should not be used for production deploys without careful consideration of the negative consequences. What happens when somebody checks out a different branch? You'll be sad when you keep git ...


5

I am not familiar with Python development nor DigitalOcean, so I'll just offer a few pointers: The goal is to automate. Everything. How you achieve that is really up to you, and creating your own tooling is not far-fetched, many do it that way. One concrete and pretty low(ish) hanging fruit is to get a git post-receive hook running which deploys and ...


4

DevOps includes automation but that's only part of it. DevOps is a cultural change to break down the silos between the different parts of the organization to provide a complete value stream. Providing a culture where business, development, quality assurance, infrastructure, security, ops, etc all work together to provide value to who ever is the end user. ...


4

I would recommend taking a look at DVC - an open source version control system for data science projects. One of the basic things that it perfectly handles is managing data files (along with code) - inputs, outputs (models), intermediate results. Semantically it's similar to git-lfs but unlike git-lfs it is capable of managing files like 100GB and what is ...


4

Assuming there are no changes in Master that are not in your release branch and you don't rebuild after you merge the code then you could deploy first and then merge to master. If either of those are not true, then merging first would be more common. The process depends on how you handle your branching strategy. If other things could be merged into ...


4

The best stack depends on what environments and platforms your working with, how much standardization you already have, and if your working with open source or not. If you have the option of using containers for everything then you can simplify the stack and reduce the tool set used. The other important note is you are deploying application artifacts and ...


3

This sounds like a problem for git cherry-pick. If you can separate the user story code from the fix code, by committing them separately, you can cherry pick the relevant commit to merge back into the develop branch. See https://stackoverflow.com/questions/9339429/what-does-cherry-picking-a-commit-with-git-mean


3

Thanks for the great question. Nothing is really trivial the first time you do it and we all were new to something once. My first recommendation is to revisit docker. Try some different guides and tutorials. It's really simple. You have a docker file that gets "built", literally just commands you want ran on the "container" or "image". You push that image ...


3

Prior to DevOps, an Agile team's definition of "done" typically meant is was "done" for the sprint (and accepted by the product-owner at or before the end of the sprint). In reality, there might be subsequent levels of testing/delivery/release-engineering before it was actually released & deployed. From a DevOps perspective, the common saying is "Done ...


3

You need to narrow down your scope and deal with tooling for monorepos. There are various awesome-monorepo lists on Github (this is a recent one) to actually find out, what kind of tooling fits your use case best (what programming languages are you using?). If you have completely isolated projects in your repository you can cobble up some bash scripting ...


3

Try LFTP that has commands to mirror or parallel copy. LFTP can run over many protocols. The likely problem you have is that you are transfering a lot of data from a build service to a single vm over the Internet. Enterprises use the same tools and protocols but they pay for more bandwidth else ensure that their builds and deploy happens within the same ...


3

You can use "git flow" or more simple "gitlab flow". Our small team merge into master on release or sprint ends.


2

From your comment I was hoping for unit tests of target code, some analysis à la valgrind all the way to CD I had an answer to the Arduino StackExchange question that was mentioned, which addresses some of this. But since writing that answer I've added some memory-checking features to arduino_ci [of which, full disclosure, I am the author]. Although it ...


2

Auto Scaling groups have a useful feature for this, named lifecycle hooks. Worflow taken from the documentation above: As you can notice there's a Scale in step, triggering a Terminating:Wait for the autoscaling group and notifying the instance to be terminated, the instance has now to do it's work and once done signal it is ok to be terminated. If the ...


2

The simplest solution is to add an entry for libraries.example.com to the /etc/hosts file with the IP you want resolved while keeping the dig line from @tensibai's comment and flagging whether it disagrees with the hosts entry (and noting where the wrong answer came from). After CI is working reliably (even if based on an ugly hack), you can query (using ...


2

Jenkins Job and Jenkins Pipeline are basically the same. In a pipeline you define the steps of your job as groovy code (actually it is CPS https://github.com/jenkinsci/workflow-cps-plugin, but that should in general just be a custom groovy interpreter). The point, that is making pipelines "better", form my perspective, is, that you can add those in so-...


2

I have a similar setup. For feature/* and develop, I use something like 18.08-SNAPSHOT in my pom. When we're ready to start release hardening, I'll cut a release branch (release/18.08) and change the version in the pom/s to 18.08. I'll also bump the version on develop to 18.09-SNAPSHOT, for example. From there, bugfix and finalise the release, merge to ...


2

I think your research is leading you in the right direction, but I cannot see the value of Docker in here. I found managing LAMP environments with Ansible completely unproblematic and the scripts I have used for 16.04 only needed a single line changes for upgrade to 18.04. I have also used them on a local development environment and there is no need to have ...


2

In this scenario, the cpu cycles required to read from the database and render to PHP will all be handled by a single core, ie they will be handled one after the other. This will be about the slowest way to deliver website traffic out of your server. Each time a request is made, a certain amount of memory is occupied by PHP. You can test the actual amount, ...


2

It’s not clear exactly what level of trust you have towards contributors. You could update your question to explain a bit more about whether people are all on one team or might be strangers. One feature that is likely to help is “branch protection settings” on this protected branches article. You can state that only code reviewed PRs are allowed into a ...


2

You could use the same tools used in Trunk Based Development (TBD) to carry work-in-progress in the trunk (the only non-release TBD branch with a longer lifespan): feature-toggles/feature-flags branch by abstraction: "Branch by Abstraction" is a technique [1] for making a large-scale change to a software system in gradual way that allows you to ...


2

There are some source control systems that bill themselves as artifact management platforms as well (for example Perforce/Helix) and they try to support the mode of work. But git is especially bad at that. You would have to do a lot of work to make git support both development of code and management of artifacts efficiently. I'd suggest to not do that and ...


1

In your deployment project configuration for each environment you could add a task prior to the actual deployment which would take care of setting up the right configuration file. It could be a script simply copying/replacing the configuration file just as you currently do manually. The script could be invoked with some pre-configured argument to identify ...


1

As @Harith mentioned, the use of secrets simplifies deployments in various environments using environmental variables. To load in a secret directly use: $ kubectl create secret generic < NAME OF SECRET > --from-literal=< KEY >=abc123!:? NOTE! This will put the secret in the Default namespace, add a namespace flag for specific resources e.g.: $ ...


1

We currently use the Jenkins as our CI/CD server and we use docker in every stage of our CI/CD. Every time a change is pushed to a SCM, it is automatically built and deployed to production(cloud). The Jenkins library feature helps to keep all generic steps in a centralized library and reused by every pipeline. You can draw some inspiration from SAP S/...


1

could you please redirect me to some article, blog, white paper or suggest me a book where I could learn and get a feel of what fully implemented CI/CD looks like. Sure, look how Google does it. I haven't seen a fully implemented DevOps lifecycle ever. There seems to be a misunderstanding here. "DevOps" is a lot of things; but a "full" CI/CD (meaning ...


1

If you start a container with an environment variable, that variable is part of the container definition. As a pro, when the server is rebooted, the same variable will be there when the container restarts. But as a con, anyone with access to the docker API can view that secret in clear text, and some apps log their environment when debugging. With swarm ...


1

For a specific GitLab runner tied to a project, the token can be found in Step 3; in Project -> Settings -> CI/CD -> Runners in the Setup a specific Runner manually section. You will also be able to find the URL which you need to use to register the runner in Step 2.


Only top voted, non community-wiki answers of a minimum length are eligible