Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [testing]

Software testing is any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results. Use this tag on questions that talk about the testing of systems or software that would be on-topic for this site.

2
votes
0answers
30 views

How to ensure testing execution order with GitLab CI?

I have about half a dozen of integration tests (handled with Robot Framework) to run in a GitLab CI pipeline. Once upon a time, we did not care about the execution order of these tests. The ...
2
votes
1answer
33 views

Jmeter simulating traffic and avoiding 429

I am using Jmeter to investigate the performance and capacity needs of a few web services. Jmeter sends traffic to the web-facing url (served by Nginx), with a rising rate, to see where the service ...
0
votes
0answers
10 views

Mocking CloudStack API

Is it anyhow possible to have a mock interface for the Apache Cloud Stack API? For example, in this bug report for Python Cloud Stack client I test an SSH key upload to CloudStack; a mock could help ...
0
votes
0answers
30 views

Comparative analysis on when to run the smoke test (and if it should be blocking the build)

Currently we are a team of 12 developers in two locations. In the past 3 quarters (3 releases) we always had an overload of bug-fixing just before the release: memory leaks functionality that worked ...
0
votes
0answers
22 views

Data Cleanup in Integration tests

How are you sanitizing a cloud RDBMS resource for Integration Testing in your pipelines? We are running in to a problem where the integration tests drop and recreate a database between each test, so ...
0
votes
0answers
197 views

How to lock resources when a Jenkins agent has multiple executors

At work we are transitioning from Bamboo to Jenkins. Currently we have multiple agents (Windows and macOS) each having a number of mobile devices attached (Android and iOS). Historically, Bamboo ...
0
votes
0answers
16 views

How did you change the culture of “Oh UI tests, Dev responsibility” to “Oh UI test, fix myself and create PR”?

At the moment Selenium is used to test the UI. Selenium itself has some steep learning curve as one has to be familiar with programming and these tests a quite fragile especially when run in headless ...
2
votes
1answer
69 views

What is a good storage for storing performance results and visualizing in grafana

Intro I am a tester and need to analyze the results of automated tests (response times and etc.) Current situation 1) I run performance tests using locust.io 2) During test execution, response ...
1
vote
0answers
62 views

Tools for provisioning windows testing environment on Hyper-V

I'm looking for tools that would help our team create and manage environments for our internal testing and debugging. Each environment consists of single windows server and multiple windows clients. ...
0
votes
2answers
39 views

Code coverage versus functional coverage

Hope this question is not to theoretical or open... We are working on a big Java project where we use JUnit tests. Our Jenkins helps us to monitor the code coverage and shows us the progress we make ...
3
votes
1answer
35 views

Tool preferences for rapidly spinning up and spinning down k8s clusters

My team hosts several completely different projects on a production k8s cluster with a given configuration. We use AWS (and are starting to use Google Cloud). We have a QA cluster as well, that "sort ...
2
votes
0answers
49 views

What integration test framework has the best docker support?

We need integration testing framework - pass a url pass requests to that url and get responses and evaluate repsonses. Requirements: 1. We want a tool that can easily implement into a CICD pipeline ...
2
votes
1answer
272 views

How do I make read only mount in Docker container writable?

When developing code on a very large repository, I would like to run multiple builds/tests in parallel in containers. I need to get my repository or some other large directory into the container. If I ...
0
votes
0answers
388 views

Build Breaker plugin on SonarQube version 7.0?

It seems build breaker plugin is not supported in SonarQube versionn 7.0. Ref: https://github.com/SonarQubeCommunity/sonar-build-breaker So What are the alternatives I can use for this plugin ...
0
votes
1answer
575 views

Automated Integration Testing of an API in Azure Functions using VSTS

I am building a HTTP API as an Azure Function App in C# and I would like to run automated integration tests (calls to the API) as part of the CI/CD. Is this possible to do using VSTS? I have ...
0
votes
0answers
19 views

Reference DevOps process implementation in Maven or Gradle containing working examples for acceptance testing

Back in the Java world.. Maven and Gradle are quite advanced tools offering sophisticated configuration capabilities. For example, you might want to have a fast-fail build stage in your CI to find ...
1
vote
1answer
724 views

Validating custom Jenkinsfile

Currently we're using Jenkins shared libraries which we're planning to use with custom Jenkinsfiles. Idea here is that developers should be able to manage those custom Jenkinsfiles without any ...
5
votes
2answers
185 views

Best practices for CI for embedded platforms (ESP32/Arduino)

Development processes for embedded software always lags behind that for web development. When looking at unit testing, automated builds and code analysis there are many limitations to popular tools. ...
1
vote
1answer
48 views

Chef (well, Test Kitchen) hanging when using local cookbook

I am new to Chef, trying to put together a POC for my company. So I've got a project that has two cookbooks under the cookbook folder. One cookbook is a "base" cookbook called 'hydra-base', the other (...
4
votes
2answers
88 views

Continuous Integration and Isolated Feature Testing

I've been given the task at work to move everyone from using TFS Source Control to Git as the Development Team has been lead to believe that Git may well be the silver bullet that will fix alot of the ...
3
votes
1answer
422 views

Is it possible to run every testing.xml file with different jenkins job?

I've around 100 test method in my testing.xml file. and that testing.xml configured in pom.xml file so I'm able to run my all those testcase as maven test and I'm getting the proper reports and all. ...
10
votes
3answers
843 views

How to test VM provisioning script without provisioning

Currently I'm on a state that testing costs me money and a lot of time... Background: I'm deploying VMs at softlayer and using a post-deployment script(bash) that will install every software that I ...
11
votes
4answers
379 views

Infrastructure as code and TDD

Infrastructure as code tells us to use tools that automate your builds. Great. Tools like ansible, chef, puppet, salt stack and others push us towards writing how infrastructure looks like, while ...
4
votes
1answer
39 views

Automated Testing Jobs

We have multiple environments close to 7 for an application that has UI and also integration server that accepts SOAP xmls. Its hard to maintain the environment integrity when many people working, ...
1
vote
1answer
65 views

How to setup testkitchen with ansible-provisioning from localhost for different vagrant-vms?

As pointed out here, testkitchen is in principable usable with ansible if the right plugins are installed. Now I've done all the installing and am at the point, where I wonder, what I missunderstood: ...
13
votes
1answer
2k views

A systematic way to test RESTful APIs with curl?

I have noticed during integration testing that I actually work on recurring use cases with restful APIs (or in general HTTP interfaces) I check here and there with bash+cURL. It starts looking quite ...
6
votes
1answer
87 views

How to test database applications in QA testing environments?

Process Our current process is: Build process produces a windows setup (msi). This setup consists of a web site deployment and also database scripts - which can create a new database or upgrade ...
-4
votes
1answer
43 views

How to implement Devops environment test section? [closed]

In our university, our instructor gave Devops environment improvement as a project. I am in the test section of this project and I have a few questions that I can not understand. Our instructor said ...
3
votes
2answers
317 views

Best tool for Docker Container test automation?

In my project i am building about 200 containers I want to check in a smoke test if the containers are still running after a rebuild in pipeline. I want to bring up the container with certain ...
2
votes
1answer
61 views

What are best practices for running Pipeline tests into a CI framework?

According to this github repo it is possible to test the Jenkinsfile, but this is run by Jenkins and if there are issues it will not run. Should one run such unit test in another CI? What are best ...
7
votes
1answer
528 views

Testing Jenkins jobs and pipelines

Currently, we have a fair number of Jenkins jobs and pipelines for builds, testing, deployments and other automated activities. Every time we change or add a new job, we only test it manually - e. g. ...
12
votes
4answers
7k views

Ansible: Other option available for telnet check of open ports?

I'm new to Ansible. Here's my task ... I have 400+ hosts, and I need to verify if 5 different ports are open from their end to our web server. Individually, I could log in and run: telnet ...
6
votes
3answers
1k views

How to create staging servers for multiple git branches?

I have to create a new staging process for our development and testing. At any given time, there are only around 4 git branches actively being developed and tested. Within each git branch, there may ...
3
votes
2answers
678 views

macOS testing on Desktop: Docker containers vs. Homebrew installation

Often I test some stuff on my MacBook. Mostly database and data store related stuff (MongoDB, MariaDB, Postgres). I install the software with Homebrew and test. I wonder if I should switch to docker? ...
7
votes
2answers
541 views

What are the pros and cons of running application testing during vs. after a docker build?

Our dockerfile creates the application environment (e.g. env variables, install libraries with apt-get etc.). It also pulls and copies the python application code from its git repo. However, we are ...
1
vote
1answer
53 views

Installing phantomjs karma launcher in Ubuntu-Docker context

Karma is a nodejs tool to automate browser testing. It seems that it depends on nodejs plugin fsevents. But, fsevents is for processing Mac OS X events, therefore I get the error "not supported on ...
7
votes
3answers
780 views

What is the future role of tester?

If the new paradigm is "You build it, you run it" (Werner Voegels, Amazon CTO) which obviously puts much more responsibility - and pressure - on software engineers, what does this change introduce to ...
4
votes
2answers
3k views

How to export Cucumber Reports via Jenkins Pipeline

I'm using the Cucumber Reports Plugin for Jenkins and I want to be able to export the HTML reports so they can be posted on an external portal. I can see that there is a folder named cucumber-html-...
0
votes
2answers
37 views

System test frameworks for scale testing on physical and virtual systems

I'm searching for a framework for running system tests. The requirement are: Able to manage provisioning to multiple physical machine Able to manage provisioning to multiple virtual environments (AWS,...
6
votes
1answer
160 views

How to improve DRP-testing of SE sites?

Have a look at what's described in the question about "Brief outage planned for Wed, May 3, 2017 at 0:00 UTC, 8pm US/Eastern (like a fire drill for computers)", which is about testing of the existing "...
26
votes
5answers
7k views

How to test a Terraform Configuration?

If you had a Terraform configuration that had a moderate degree of complexity, how would you write tests around the configuration that could be executed as part of a Continuous Integration / ...
7
votes
2answers
419 views

How to test a cluster of microservices?

I took a look into Server Spec and it says it clearly: Remark: serverspec test suites are meant to be run against a single machine (or docker container). In other words, you should not try to ...
8
votes
3answers
125 views

How to implement a frozen test environment?

Here is a partial quote from an answer to the question about "How to avoid continuous integration-caused instabilities in test environments?": This environment usually freezes during the tests. My ...
20
votes
5answers
417 views

How to avoid continuous integration-caused instabilities in test environments?

Assume you're using continuous integration processes which frequently update some target environments, so that every time there are some changes "you" can test your changes right away. That's part of ...
6
votes
1answer
78 views

How to avoid a regular user can only test in production?

When configuring test environments, the issue often comes up where I ask the question to the customer like in my question title here: How to avoid a regular user can only test in production? There ...
30
votes
5answers
3k views

How to test provisioning and configuration in Ansible setup?

Looking at trying to build some resilience into our Ansible setup which deals with provisioning and configuration. I understand a few methods of testing on the configuration side of things but I'm ...
8
votes
2answers
221 views

What is “black box” testing? [closed]

What is "black box" testing and how it is different from ordinary testing? For example black box tests prior to deploying to production. Note: "black box" testing is a common requirement when ...