Hot answers tagged

31 votes
Accepted

Ansible: Other option available for telnet check of open ports?

You can use the Ansible wait_for module which checks a specific TCP port is open. Since in this case, all ports should be open already, we can use a minimal no. of retries, just enough to cover ...
user avatar
  • 812
24 votes
Accepted

How to test provisioning and configuration in Ansible setup?

Some options out there.. Testing tools: Sorted by github stars Serverspec - Ruby, most popular tool out there, built on ruby's rspec Goss - YAML, simple, <10MB self-contained binary, extremely ...
user avatar
24 votes
Accepted

How to test a Terraform Configuration?

There is currently no full solution to this integrated into Terraform, but there are some building blocks that could be useful to assist in writing tests in a separate programming language. Terraform ...
user avatar
15 votes

How to test a Terraform Configuration?

We recently open sourced Terratest, our swiss army knife for testing infrastructure code. Today, you're probably testing all your infrastructure code manually by deploying, validating, and ...
user avatar
13 votes

How to test a Terraform Configuration?

As an update to this question, there is now Kitchen-Terraform which allows the testing of Terraform Configuration files without breaking production environments. The repository also includes a few ...
user avatar
  • 2,993
12 votes

How to test provisioning and configuration in Ansible setup?

The two tools I've seen for this are InSpec and ServerSpec. Serverspec is a Ruby-based tool that builds on RSpec. InSpec is inspired by RSpec and ServerSpec. I've used ServerSpec. It's cool, but ...
user avatar
  • 3,973
11 votes

How to test provisioning and configuration in Ansible setup?

When using configuration management tools, such as Ansible, the tool itself would be responsible preventing configuration drift. Once you used Ansible to set a certain configuration, repetitive ...
user avatar
10 votes
Accepted

How to avoid continuous integration-caused instabilities in test environments?

I'll give my experience on this one, mostly because it showcases why some answers are not always applicable. Some context to start: We have 7 environments to host roughly 80 applications, most of ...
user avatar
  • 11.1k
10 votes
Accepted

Testing Jenkins jobs and pipelines

I'm posting this here not because I endorse these solutions (in fact, I've never tried them), but just because they are a potential answer to your question: You can start with JenkinsPipelineUnit, a ...
user avatar
  • 2,743
10 votes

How to test VM provisioning script without provisioning

I can see some options: Use Vagrant to create your VMs; it separates the process of creating the VM (including the base OS) and the actual provisioning. It also has some options to run certain ...
user avatar
  • 4,706
9 votes
Accepted

What is the future role of tester?

IMHO that depends on what the role of the tester was before such transformation. BTW, I believe my answer applies to the DevOps transformation in general, not only to the You build it, you run it ...
user avatar
8 votes

How to avoid continuous integration-caused instabilities in test environments?

Sounds like you're talking about a test environment which is constantly re-used without being reliably re-initialized for every test execution. This makes such test an unreliable one. Similar, from ...
user avatar
8 votes

A systematic way to test RESTful APIs with curl?

You could look at tools such as Postman which focuses on testing REST APIs with JavaScript - it has some nice features but you lose the use of Python. Instead, I'd suggest looking at REST-related ...
user avatar
  • 812
7 votes
Accepted

How to improve DRP-testing of SE sites?

NOTE: It's probably not worth reading too much into the outward comprehension of how good-or-not StackExchange is at managing their disaster recovery scenarios. I suspect they are following much of ...
user avatar
7 votes

How to test a Terraform Configuration?

In addition to all the other options mentioned, I would like to mention that InSpec 2.0 added support for cloud provider APIs. Basically, you can continue writing the IaC with Terraform, then write ...
user avatar
7 votes

How to avoid continuous integration-caused instabilities in test environments?

The usual approach is to create different environments: DEV - this is the place where dev team mess the things. Here are create all changes tunings, deploy new version and so on. Here is the place ...
user avatar
7 votes

How to create staging servers for multiple git branches?

1) How can I run several different branches on the staging server? Docker 2) How would I set up the DB evolution system to make sure it always has appropriate DB for each branch? This depends on ...
user avatar
  • 3,674
6 votes

How to test provisioning and configuration in Ansible setup?

Test Kitchen has a kitchen-ansible provisioner plugin for testing of Ansible code. It isn't as deep as the Chef integration but it does get the job done for most cases. There is also the more recent ...
user avatar
  • 1,197
6 votes

How to avoid a regular user can only test in production?

A Sandbox could be part of the solution To bring material to the subject you can check the questions with tag sandbox on MetaSE, there are sanboxes for: Q/A formatting. Comments formatting. Chat. ...
user avatar
  • 11.1k
6 votes
Accepted

What is "black box" testing?

I will make my answer, based on the knowledge for security testing, but IMHO this can be generalized. Black box testing - when the tester know nothing about the system, components, liaisons, ...
user avatar
6 votes
Accepted

Infrastructure as code and TDD

In short, I see two categories of tests for your infrastructure: 1) does it have everything you need to run your application and 2) does it not have any superfluous stuff. First and foremost, you can ...
user avatar
  • 4,706
6 votes

How to test VM provisioning script without provisioning

http://www.vagrantup.com You could use vagrant to deploy VMs on the local laptop. You could also check whether it is possible to split the script in smaller parts so it will not take four hours to ...
user avatar
  • 12.6k
6 votes

Continuous Integration and Isolated Feature Testing

First of all, git is certainly not to blame for merges taking a long time. Clean merging is the hallmark of git, unless it is used in some broken way. So I encourage you to look at what exactly seems ...
user avatar
  • 4,706
6 votes
Accepted

Best practices for CI for embedded platforms (ESP32/Arduino)

Since "deployment" to the IoT device, especially in bulk, will not be done using the CI system. Then the purpose of the CI system is mostly to make sure that it will work okay. This means that you ...
user avatar
5 votes

How to test a Terraform Configuration?

On Aws-Side there is https://github.com/k1LoW/awspec - it should be possible, to feed in terraform.state and test, wheter terraform applied correct. But I think, beyond testing on low level the tool, ...
user avatar
  • 141
5 votes

How to test VM provisioning script without provisioning

If testing locally isn't an option, then the most straight forward approach would be to use disk volume snapshots/backups to your advantage. These will still cost $$$, but will save you time in the ...
user avatar
  • 2,993
4 votes

How to implement a frozen test environment?

TeamCity has a Shared Resources build feature which allows you to define a resource which multiple Build Definitions depend upon it. Build Definitions can either require a Read Lock or a Write Lock, ...
user avatar
4 votes

What is the future role of tester?

"You build it, you run it" This quote aim at giving emphase on the break between silo teams, one the principle of devops is to avoid silo achieving one task. While the idea here focus on the build ...
user avatar
  • 11.1k
4 votes
Accepted

macOS testing on Desktop: Docker containers vs. Homebrew installation

One of the advantages of using Docker would be that you can easily run tests with different versions of the (DB) apps, which could be quite difficult with Homebrew (some apps don't easily support ...
user avatar
4 votes

Continuous Integration and Isolated Feature Testing

There is however an issue of merge conflicts still, some of which taking up to 2 or 3 hours to resolve. The team are questioning if the tool (Git) is to blame whereas under analysis. What is ...
user avatar
  • 12.6k

Only top scored, non community-wiki answers of a minimum length are eligible