Hot answers tagged

25

You can use the Ansible wait_for module which checks a specific TCP port is open. Since in this case, all ports should be open already, we can use a minimal no. of retries, just enough to cover network issues: - name: Check all port numbers are accessible from current host wait_for: host: mywebserver.com port: "{{ item }}" state: started ...


19

There is currently no full solution to this integrated into Terraform, but there are some building blocks that could be useful to assist in writing tests in a separate programming language. Terraform produces state files in JSON format that can, in principle, be used by external programs to extract certain data about what Terraform created. While this ...


18

Some options out there.. Testing tools: Sorted by github stars Serverspec - Ruby, most popular tool out there, built on ruby's rspec Goss - YAML, simple, <10MB self-contained binary, extremely fast, can generate tests from system state Inspec - Ruby, think of it as an improved serverspec, almost same syntax, made by the chef guys. Built to be easier to ...


13

The two tools I've seen for this are InSpec and ServerSpec. Serverspec is a Ruby-based tool that builds on RSpec. InSpec is inspired by RSpec and ServerSpec. I've used ServerSpec. It's cool, but maybe not 100% stable. I've had problems with testing for specific versions of software on Ubuntu. I've read the InSpec docs but haven't dug in deep. It does ...


10

I'll give my experience on this one, mostly because it showcases why some answers are not always applicable. Some context to start: We have 7 environments to host roughly 80 applications, most of them rely on each others through webservices or shared tables on db2-iSeries. For good or bad, the iSeries are our DB system of reference. This last point ...


10

When using configuration management tools, such as Ansible, the tool itself would be responsible preventing configuration drift. Once you used Ansible to set a certain configuration, repetitive execution of Ansible will ensure your configuration is as you defined it to be. This also requires your Ansible code to be written in a manner which is idempotent. ...


10

As an update to this question, there is now Kitchen-Terraform which allows the testing of Terraform Configuration files without breaking production environments. The repository also includes a few examples for different Terraform providers.


9

IMHO that depends on what the role of the tester was before such transformation. BTW, I believe my answer applies to the DevOps transformation in general, not only to the You build it, you run it paradigm. If the role was a drone job - mindlessly executing tests - it's bound to go away, automation will eat such jobs. If the role included writing the test ...


8

Sounds like you're talking about a test environment which is constantly re-used without being reliably re-initialized for every test execution. This makes such test an unreliable one. Similar, from the reliability perspective, with manual testing, if you want. IMHO you shouldn't be using such testing inside your CI/CD qualification purposes as that will ...


8

We recently open sourced Terratest, our swiss army knife for testing infrastructure code. Today, you're probably testing all your infrastructure code manually by deploying, validating, and undeploying. Terratest helps you automate this process: Write tests in Go. Use helpers in Terratest to execute your real IaC tools (e.g., Terraform, Packer, etc.) to ...


8

I'm posting this here not because I endorse these solutions (in fact, I've never tried them), but just because they are a potential answer to your question: You can start with JenkinsPipelineUnit, a unit testing framework for Pipeline scripts. There is also a project called jenkinsfile-runner which executes your Jenkinsfile in a transient, headless Jenkins ...


8

I can see some options: Use Vagrant to create your VMs; it separates the process of creating the VM (including the base OS) and the actual provisioning. It also has some options to run certain provisioning steps at certain circumstances only. Use Ansible, Puppet or something like that to switch to a provisioning mode where you do not do the same stuff ...


7

NOTE: It's probably not worth reading too much into the outward comprehension of how good-or-not StackExchange is at managing their disaster recovery scenarios. I suspect they are following much of the best practice below and simply testing scenarios to validate their configuration. Depending on the environment you operate within: A disaster recovery plan ...


7

The usual approach is to create different environments: DEV - this is the place where dev team mess the things. Here are create all changes tunings, deploy new version and so on. Here is the place where CI is integrated fully. PREPROD/QA - this is the place "play" QA/test/validation team do tests. This environment usually freeze during the tests. ...


7

You could look at tools such as Postman which focuses on testing REST APIs with JavaScript - it has some nice features but you lose the use of Python. Instead, I'd suggest looking at REST-related plugins for pytest, a Python test framework that simplifies your test code, while still running tests written using unittest. For example, writing parameterised ...


6

I will make my answer, based on the knowledge for security testing, but IMHO this can be generalized. Black box testing - when the tester know nothing about the system, components, liaisons, connections, etc. This can be helpful more like UI/UX testing, functional testing. Example: you do not work for Microsoft and also you do not have the source code and ...


6

A Sandbox could be part of the solution To bring material to the subject you can check the questions with tag sandbox on MetaSE, there are sanboxes for: Q/A formatting. Comments formatting. Chat. There's a feature request about "Can we get a Stack Exchange sandbox?" which has no status (neither declined, pending, review nor in-progress meta-tag). So I ...


6

1) How can I run several different branches on the staging server? Docker 2) How would I set up the DB evolution system to make sure it always has appropriate DB for each branch? This depends on how much you expect your DB to scale. You can get pretty crazy with methods to clone database data but typically you will want a master copy that you do not ...


6

In short, I see two categories of tests for your infrastructure: 1) does it have everything you need to run your application and 2) does it not have any superfluous stuff. First and foremost, you can treat the test suite of your actual software as a kind of "meta test" for your infrastructure. As long as you create the infrastructure from scratch for each ...


6

First of all, git is certainly not to blame for merges taking a long time. Clean merging is the hallmark of git, unless it is used in some broken way. So I encourage you to look at what exactly seems to be the underlying problem. If your devs regularly make sweeping changes to the same files, then it's likely more a problem of cohesion/coupling than of the ...


6

Since "deployment" to the IoT device, especially in bulk, will not be done using the CI system. Then the purpose of the CI system is mostly to make sure that it will work okay. This means that you mostly want to run some tests using the CI. Automated testing is definitely possible, both for unit tests and for integration tests. The most important thing for ...


5

On Aws-Side there is https://github.com/k1LoW/awspec - it should be possible, to feed in terraform.state and test, wheter terraform applied correct. But I think, beyond testing on low level the tool, you used, it's probably a better idea, to think about how to test whole infrastructures. We're discussing around this idea here: https://github.com/...


5

Test Kitchen has a kitchen-ansible provisioner plugin for testing of Ansible code. It isn't as deep as the Chef integration but it does get the job done for most cases. There is also the more recent Molecule project which is a dedicated Ansible testing system.


5

http://www.vagrantup.com You could use vagrant to deploy VMs on the local laptop. You could also check whether it is possible to split the script in smaller parts so it will not take four hours to test it.


4

TeamCity has a Shared Resources build feature which allows you to define a resource which multiple Build Definitions depend upon it. Build Definitions can either require a Read Lock or a Write Lock, you can also define if these locks are exclusive or allow a degree of parallelism. If we make the following assumptions about a shared environment named ...


4

In addition to all the other options mentioned, I would like to mention that InSpec 2.0 added support for cloud provider APIs. Basically, you can continue writing the IaC with Terraform, then write compliancy checks with InSpec for your cloud resources. In addition, InSpec supports writing tests for individual machines if you ever need it. Here is an ...


4

"You build it, you run it" This quote aim at giving emphase on the break between silo teams, one the principle of devops is to avoid silo achieving one task. While the idea here focus on the build and run phases, the important idea is to bring a whole team together, from architecture to exploitation roles. A "Devops Team" would be made of all roles taking ...


4

If testing locally isn't an option, then the most straight forward approach would be to use disk volume snapshots/backups to your advantage. These will still cost $$$, but will save you time in the long run. You should then separate your bash script into different working segments/scripts that can be tested individually. Once your server is provisioned, run ...


4

There is however an issue of merge conflicts still, some of which taking up to 2 or 3 hours to resolve. The team are questioning if the tool (Git) is to blame whereas under analysis. What is causing the merge hassle? Why does it take up to two or three hours to resolve merge conflicts? What workflow do the developers use? In summary, what is the cause ...


3

Working off what @chupasaurus said, here's what I came up with: node('master') { dir("../builds/${BUILD_NUMBER}/") { sh "cp -r cucumber-html-reports $WORKSPACE" } archive "cucumber-html-reports/*" } Obviously all this does is archive the report for that build, but you can easily extend this to copy it somewhere else where it can be ...


Only top voted, non community-wiki answers of a minimum length are eligible