34
votes
Accepted
Ansible: Other option available for telnet check of open ports?
You can use the Ansible wait_for module which checks a specific TCP port is open.
Since in this case, all ports should be open already, we can use a minimal no. of retries, just enough to cover ...
- 872
24
votes
Accepted
How to test provisioning and configuration in Ansible setup?
Some options out there..
Testing tools: Sorted by github stars
Serverspec - Ruby, most popular tool out there, built on ruby's rspec
Goss - YAML, simple, <10MB self-contained binary, extremely ...
- 471
24
votes
Accepted
How to test a Terraform Configuration?
There is currently no full solution to this integrated into Terraform, but there are some building blocks that could be useful to assist in writing tests in a separate programming language.
Terraform ...
- 2,171
16
votes
How to test a Terraform Configuration?
We recently open sourced Terratest, our swiss army knife for testing infrastructure code.
Today, you're probably testing all your infrastructure code manually by deploying, validating, and ...
- 291
13
votes
How to test a Terraform Configuration?
As an update to this question, there is now Kitchen-Terraform which allows the testing of Terraform Configuration files without breaking production environments. The repository also includes a few ...
- 3,218
12
votes
How to test provisioning and configuration in Ansible setup?
The two tools I've seen for this are InSpec and ServerSpec. Serverspec is a Ruby-based tool that builds on RSpec. InSpec is inspired by RSpec and ServerSpec.
I've used ServerSpec. It's cool, but ...
- 4,068
11
votes
How to test provisioning and configuration in Ansible setup?
When using configuration management tools, such as Ansible, the tool itself would be responsible preventing configuration drift. Once you used Ansible to set a certain configuration, repetitive ...
- 8,963
10
votes
Accepted
What is the future role of tester?
IMHO that depends on what the role of the tester was before such transformation. BTW, I believe my answer applies to the DevOps transformation in general, not only to the You build it, you run it ...
- 6,730
10
votes
Accepted
How to avoid continuous integration-caused instabilities in test environments?
I'll give my experience on this one, mostly because it showcases why some answers are not always applicable.
Some context to start:
We have 7 environments to host roughly 80 applications, most of ...
- 11.3k
10
votes
Accepted
Testing Jenkins jobs and pipelines
I'm posting this here not because I endorse these solutions (in fact, I've never tried them), but just because they are a potential answer to your question:
You can start with JenkinsPipelineUnit, a ...
- 2,932
10
votes
How to test VM provisioning script without provisioning
I can see some options:
Use Vagrant to create your VMs; it separates the process of creating the VM (including the base OS) and the actual provisioning. It also has some options to run certain ...
- 4,866
9
votes
A systematic way to test RESTful APIs with curl?
You could look at tools such as Postman which focuses on testing REST APIs with JavaScript - it has some nice features but you lose the use of Python.
Instead, I'd suggest looking at REST-related ...
- 872
8
votes
How to avoid continuous integration-caused instabilities in test environments?
Sounds like you're talking about a test environment which is constantly re-used without being reliably re-initialized for every test execution. This makes such test an unreliable one. Similar, from ...
- 6,730
7
votes
Accepted
How to improve DRP-testing of SE sites?
NOTE: It's probably not worth reading too much into the outward comprehension of how good-or-not StackExchange is at managing their disaster recovery scenarios. I suspect they are following much of ...
- 11.6k
7
votes
How to test a Terraform Configuration?
In addition to all the other options mentioned, I would like to mention that InSpec 2.0 added support for cloud provider APIs. Basically, you can continue writing the IaC with Terraform, then write ...
- 323
7
votes
How to avoid continuous integration-caused instabilities in test environments?
The usual approach is to create different environments:
DEV - this is the place where dev team mess the things. Here are create all changes tunings, deploy new version and so on. Here is the place ...
- 431
7
votes
How to create staging servers for multiple git branches?
1) How can I run several different branches on the staging server?
Docker
2) How would I set up the DB evolution system to make sure it always has appropriate DB for each branch?
This depends on ...
- 3,724
6
votes
How to test provisioning and configuration in Ansible setup?
Test Kitchen has a kitchen-ansible provisioner plugin for testing of Ansible code. It isn't as deep as the Chef integration but it does get the job done for most cases. There is also the more recent ...
- 1,197
6
votes
How to avoid a regular user can only test in production?
A Sandbox could be part of the solution
To bring material to the subject you can check the questions with tag sandbox on MetaSE, there are sanboxes for:
Q/A formatting.
Comments formatting.
Chat.
...
- 11.3k
6
votes
Accepted
What is "black box" testing?
I will make my answer, based on the knowledge for security testing, but IMHO this can be generalized.
Black box testing - when the tester know nothing about the system, components, liaisons, ...
- 431
6
votes
Accepted
Infrastructure as code and TDD
In short, I see two categories of tests for your infrastructure: 1) does it have everything you need to run your application and 2) does it not have any superfluous stuff.
First and foremost, you can ...
- 4,866
6
votes
How to test VM provisioning script without provisioning
http://www.vagrantup.com
You could use vagrant to deploy VMs on the local laptop.
You could also check whether it is possible to split the script in smaller parts so it will not take four hours to ...
- 13.1k
6
votes
Continuous Integration and Isolated Feature Testing
First of all, git is certainly not to blame for merges taking a long time. Clean merging is the hallmark of git, unless it is used in some broken way. So I encourage you to look at what exactly seems ...
- 4,866
6
votes
Accepted
Best practices for CI for embedded platforms (ESP32/Arduino)
Since "deployment" to the IoT device, especially in bulk, will not be done using the CI system. Then the purpose of the CI system is mostly to make sure that it will work okay.
This means that you ...
- 8,963
5
votes
How to test a Terraform Configuration?
On Aws-Side there is https://github.com/k1LoW/awspec - it should be possible, to feed in terraform.state and test, wheter terraform applied correct.
But I think, beyond testing on low level the tool, ...
- 141
5
votes
How to test VM provisioning script without provisioning
If testing locally isn't an option, then the most straight forward approach would be to use disk volume snapshots/backups to your advantage. These will still cost $$$, but will save you time in the ...
- 3,218
4
votes
How to implement a frozen test environment?
TeamCity has a Shared Resources build feature which allows you to define a resource which multiple Build Definitions depend upon it. Build Definitions can either require a Read Lock or a Write Lock, ...
- 11.6k
4
votes
What is the future role of tester?
"You build it, you run it"
This quote aim at giving emphase on the break between silo teams, one the principle of devops is to avoid silo achieving one task.
While the idea here focus on the build ...
- 11.3k
4
votes
Accepted
macOS testing on Desktop: Docker containers vs. Homebrew installation
One of the advantages of using Docker would be that you can easily run tests with different versions of the (DB) apps, which could be quite difficult with Homebrew (some apps don't easily support ...
- 6,730
4
votes
macOS testing on Desktop: Docker containers vs. Homebrew installation
One of the cons of Homebrew for the use of testing your DB stuff is how tightly coupled the DB is to your local environment.
You'll inevitably end up needing to manage the details of what services ...
- 51
Only top scored, non community-wiki answers of a minimum length are eligible