Skip to main content

Questions tagged [testing]

Software testing is any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results. Use this tag on questions that talk about the testing of systems or software that would be on-topic for this site.

Filter by
Sorted by
Tagged with
0 votes
1 answer
70 views

Docker gets stuck when running 16 tests

I'm running my dotnet 7.0 tests using "testcontainers" on windows machine. The tests are parallelized integration tests which use SQL Server inside a Linux docker container. Each test ...
Bizhan's user avatar
  • 101
0 votes
1 answer
41 views

QA instability due to lack of stable UAT/staging environment

Our testing teams are facing extensive downtime due to testing in a shared QA environment. When our front end teams perform end to end testing, their test (QA) environment points to the back end ...
Half_Duplex's user avatar
1 vote
1 answer
98 views

User-level secrets in GitHub Actions

I would like to create a GitHub workflow for running tests against a REST API. The API needs a key that is always tied to a user, there is no global "testing" key. Is there a way for each ...
Parzival's user avatar
0 votes
1 answer
305 views

Deploy A War/Ear To Container Marked build As failure When Deploying To Tomcat 9 Server

When I was Deploying A Sample Application On Tomcat9 I Faced This Issue. This is the Tomact9 Users File where I added credentials do I need to add anything more, Is There Any Configuration Which I ...
Prathamesh Shukla's user avatar
1 vote
1 answer
265 views

How does one perform systems testing against multiple interdependent machines?

I'm looking for a platform that allows me to run tests against multiple, closely-coupled systems. We deploy different products to Red Hat family machines - think Rocky Linux, CentOS, Fedora, RHEL, etc....
vpseg's user avatar
  • 11
0 votes
2 answers
2k views

TeamCity run step in docker

I'm trying to set up TeamCity to run tests on a .NET project. I have installed TeamCity and its agent (with access to docker) using docker compose : teamcity: image: jetbrains/teamcity-server ...
Ombrelin's user avatar
  • 101
1 vote
1 answer
799 views

How can you do pre-merge testing in trunk based development?

Currently, we use a simple branching strategy, with a single main branch and one layer of feature branches; no branches on branches on branches. We also have many pipelines that run against each PR to ...
James Geddes's user avatar
5 votes
2 answers
912 views

How to test the CI/CD pipeline itself?

I have gitlab CI/CD defined through .gitlab-ci.yml. It runs different jobs if the pipeline is triggered manually, from a merge request or from a scheduled pipeline. It accepts a bunch of environment ...
John Doe's user avatar
0 votes
1 answer
269 views

Snapshot or Saving Prometheus Data for Performance testing and comparison

I have an application and system that exposes its resource metrics to Prometheus and Grafana that I want to do performance testing on. I want to be able to conduct a test, capture the metrics on CPU, ...
DogEatDog's user avatar
  • 191
6 votes
1 answer
236 views

What are the metrics to consider when doing API performance testing?

I want to establish the benchmark behavior of my API through performance testing. The purpose is to ensure that the product meet or exceed requirements such as: Throughput Response time Stability ...
storm's user avatar
  • 1,759
3 votes
3 answers
218 views

Should I shut down the staging environment when the deployment test ends?

I have a small application deployed on an EC2 instance. I need a staging environment so I've thought of two choices: Deploy the same app again on the same instance, and that would be a staging ...
mr L's user avatar
  • 131
1 vote
1 answer
56 views

Sorting out automated testing in CI pipeline and getting conflicting information related to building, unit testing and Docker images

I seem to get a unique response for every person that I ask, or every article that I read, which is just confusing me even more. Basically, I'm trying to setup automated testing in the CI pipeline ...
cheslijones's user avatar
2 votes
1 answer
108 views

What is a good strategy to reliably and automatically perform remote system tests of a web service?

In a company I work for, we have deployed many services: a major part of them are small and rather rarely used, but an outage would be a problem nonetheless. At some point I realized that manual ...
d33tah's user avatar
  • 71
1 vote
1 answer
40 views

What is it called to deploy major changes for limited number of customers first?

Is there a common name for this deployment strategy which is to avoid deploying major updates across all the customers straight away but rather limit the initial number to see their feedback, find any ...
Greendrake's user avatar
3 votes
0 answers
46 views

Is there an all-in-one control center software that can be used to manage build and test execution and to visualize test results?

If there are no constraints with regards to other components used in the system at all, do you know a software that can be used as a sort of "cockpit" / control center to visualize test ...
Hunter2's user avatar
  • 31
1 vote
1 answer
48 views

Testing docker image builds when low level libraries change

Background In our organization we have many small repos for shared libraries used among our applications. Particularly, we have many Python libraries and Python applications. We have a structures like:...
flakes's user avatar
  • 121
2 votes
1 answer
98 views

How to test production Ansible execution without having to provision target

We use ansible for our day to day operation and so far so good. We have been looking for a way to run local test etc just like chef guys and found molecule that we use with testinfra/vagrant. While I ...
black sensei's user avatar
3 votes
1 answer
438 views

Trunk Based Development - release branch testing

We are currently using Git flow branching strategy and looking to go to a trunk based strategy - although we are quite a ways away from getting there I am trying to get all my ducks in a row, ...
VKR's user avatar
  • 71
2 votes
1 answer
49 views

A name for testing software migrated to a new environment against original application

This is more of an english language question - we have various types of tests - unit tests, A/B tests, etc... Suppose I need to migrate code to a different language, machine, or any other different ...
kabanus's user avatar
  • 123
0 votes
1 answer
453 views

Safari test automation using Robot Framework via Jenkins

I spent about one week trying to make my Robot Framework tests work for Safari browser in MacOS, when launched by Jenkins via ssh. The main issue was that Robot couldn't open a browser when executed ...
Vladimir T's user avatar
1 vote
1 answer
277 views

Azure DevOps Server: Why does a Visual Studio Test step take so long to run unit tests?

In an Azure DevOps Server build system installed on premises, I have a Build Pipeline with a Visual Studio Test step. The step is configured to discover and run XUnit unit tests in a DLL. The step is ...
urig's user avatar
  • 121
1 vote
1 answer
101 views

Is Cypress ok for acceptance tests on APIs in a CI/CD pipeline?

I have a small team but a gigantic application. We don't have PHP unit testing, so we will be using Cypress as a baseline for quality. Everything will go through a CI/CD pipeline and use regression ...
Daniel Vianna's user avatar
4 votes
1 answer
1k views

How to share variables across instances?

I'm writing TestInfra test scripts for mongo cluster instances using Molecule Ansible. I want to share variables across all the instances. I created a python class, and initialise and update the ...
Abel's user avatar
  • 141
0 votes
1 answer
668 views

Running Protractor tests in docker through teamcity

I have a project for which I am using Teamcity to automate deployment, and I would also like to automate testing. It is an angular frontend, and we have some protractor tests. I would like to run ...
Miguel Mesquita Alfaiate's user avatar
2 votes
2 answers
163 views

How can I test native macOS UI software in a container before deployment?

As part of the CI setup, I'd like to run (automated) tests on native macOS GUI software in a 'safe environment' (i.e. without messing with the host software installation too much). Virtual machines (...
Frerich Raabe's user avatar
6 votes
0 answers
89 views

Blackbox testing vs openbox testing applications in a containerized environment [closed]

We use containers to deploy our Node.js applications. But currently, the development and a good share of testing is done by the developers on their own host system and thus using different versions of ...
Sylvain Leroux's user avatar
2 votes
0 answers
570 views

How to ensure testing execution order with GitLab CI? [closed]

I have about half a dozen of integration tests (handled with Robot Framework) to run in a GitLab CI pipeline. Once upon a time, we did not care about the execution order of these tests. The ...
avazula's user avatar
  • 949
2 votes
0 answers
91 views

How to deploy large web app in container minimal data transfer? [closed]

In current deployment of a web app we are using svn to deploy a specific revision from a repository. The amount of data transferred is just the diff between the currently deployed revision and the ...
Jiri Klouda's user avatar
  • 5,807
1 vote
1 answer
1k views

Jmeter simulating traffic and avoiding 429

I am using Jmeter to investigate the performance and capacity needs of a few web services. Jmeter sends traffic to the web-facing url (served by Nginx), with a rising rate, to see where the service ...
Bruce Becker's user avatar
  • 3,613
3 votes
1 answer
637 views

What is a good storage for storing performance results and visualizing in grafana

Intro I am a tester and need to analyze the results of automated tests (response times and etc.) Current situation 1) I run performance tests using locust.io 2) During test execution, response ...
GrimSmiler's user avatar
1 vote
2 answers
78 views

Code coverage versus functional coverage

We are working on a big Java project where we use JUnit tests. Our Jenkins helps us to monitor the code coverage and shows us the progress we make with this. But there seems to be something lacking ...
Frank's user avatar
  • 111
3 votes
1 answer
63 views

Tool preferences for rapidly spinning up and spinning down k8s clusters

My team hosts several completely different projects on a production k8s cluster with a given configuration. We use AWS (and are starting to use Google Cloud). We have a QA cluster as well, that "sort ...
swagrov's user avatar
  • 131
2 votes
0 answers
79 views

What integration test framework has the best docker support? [closed]

We need integration testing framework - pass a url pass requests to that url and get responses and evaluate repsonses. Requirements: 1. We want a tool that can easily implement into a CICD pipeline ...
DemiSheep's user avatar
  • 163
4 votes
1 answer
3k views

How do I make read only mount in Docker container writable?

When developing code on a very large repository, I would like to run multiple builds/tests in parallel in containers. I need to get my repository or some other large directory into the container. If I ...
Jiri Klouda's user avatar
  • 5,807
0 votes
1 answer
880 views

Automated Integration Testing of an API in Azure Functions using VSTS

I am building a HTTP API as an Azure Function App in C# and I would like to run automated integration tests (calls to the API) as part of the CI/CD. Is this possible to do using VSTS? I have ...
Victor's user avatar
  • 103
2 votes
1 answer
2k views

Validating custom Jenkinsfile

Currently we're using Jenkins shared libraries which we're planning to use with custom Jenkinsfiles. Idea here is that developers should be able to manage those custom Jenkinsfiles without any ...
zrks's user avatar
  • 21
5 votes
2 answers
1k views

Best practices for CI for embedded platforms (ESP32/Arduino)

Development processes for embedded software always lags behind that for web development. When looking at unit testing, automated builds and code analysis there are many limitations to popular tools. ...
Moritz's user avatar
  • 1,217
1 vote
1 answer
231 views

Chef (well, Test Kitchen) hanging when using local cookbook

I am new to Chef, trying to put together a POC for my company. So I've got a project that has two cookbooks under the cookbook folder. One cookbook is a "base" cookbook called 'hydra-base', the other (...
Kevin Milner's user avatar
4 votes
2 answers
288 views

Continuous Integration and Isolated Feature Testing

I've been given the task at work to move everyone from using TFS Source Control to Git as the Development Team has been lead to believe that Git may well be the silver bullet that will fix alot of the ...
Stu1986C's user avatar
  • 141
3 votes
1 answer
2k views

Is it possible to run every testing.xml file with different jenkins job?

I've around 100 test method in my testing.xml file. and that testing.xml configured in pom.xml file so I'm able to run my all those testcase as maven test and I'm getting the proper reports and all. ...
NarendraR's user avatar
  • 233
10 votes
3 answers
1k views

How to test VM provisioning script without provisioning

Currently I'm on a state that testing costs me money and a lot of time... Background: I'm deploying VMs at softlayer and using a post-deployment script(bash) that will install every software that I ...
Lucas Mattos's user avatar
11 votes
4 answers
942 views

Infrastructure as code and TDD

Infrastructure as code tells us to use tools that automate your builds. Great. Tools like ansible, chef, puppet, salt stack and others push us towards writing how infrastructure looks like, while ...
JackLeo's user avatar
  • 213
4 votes
1 answer
48 views

Automated Testing Jobs

We have multiple environments close to 7 for an application that has UI and also integration server that accepts SOAP xmls. Its hard to maintain the environment integrity when many people working, ...
Tech Dude's user avatar
1 vote
1 answer
129 views

How to setup testkitchen with ansible-provisioning from localhost for different vagrant-vms?

As pointed out here, testkitchen is in principable usable with ansible if the right plugins are installed. Now I've done all the installing and am at the point, where I wonder, what I missunderstood: ...
achnichtsowichtig's user avatar
17 votes
1 answer
6k views

A systematic way to test RESTful APIs with curl?

I have noticed during integration testing that I actually work on recurring use cases with restful APIs (or in general HTTP interfaces) I check here and there with bash+cURL. It starts looking quite ...
Ta Mu's user avatar
  • 6,782
6 votes
1 answer
143 views

How to test database applications in QA testing environments?

Process Our current process is: Build process produces a windows setup (msi). This setup consists of a web site deployment and also database scripts - which can create a new database or upgrade ...
user5078's user avatar
-4 votes
1 answer
52 views

How to implement Devops environment test section? [closed]

In our university, our instructor gave Devops environment improvement as a project. I am in the test section of this project and I have a few questions that I can not understand. Our instructor said ...
jundo's user avatar
  • 1
3 votes
2 answers
738 views

Best tool for Docker Container test automation?

In my project i am building about 200 containers I want to check in a smoke test if the containers are still running after a rebuild in pipeline. I want to bring up the container with certain ...
Mandragor's user avatar
  • 133
2 votes
1 answer
82 views

What are best practices for running Pipeline tests into a CI framework?

According to this github repo it is possible to test the Jenkinsfile, but this is run by Jenkins and if there are issues it will not run. Should one run such unit test in another CI? What are best ...
030's user avatar
  • 13.2k
14 votes
1 answer
3k views

Testing Jenkins jobs and pipelines

Currently, we have a fair number of Jenkins jobs and pipelines for builds, testing, deployments and other automated activities. Every time we change or add a new job, we only test it manually - e. g. ...
alecxe's user avatar
  • 849