So we have a TeamCity with numerous JUnit 4 test suites builds configured.
The problem: Tests tab is only populated after build is complete. If build crashes, tests tab is not populated, even with tests which completed before crash.
The reason for that is Surefire reporting the following to teamcity:
##teamcity[importData tc:tags='tc:internal' type='surefire' path='.../target/surefire-reports/TEST-*.xml' whenNoDataPublished='nothing' logAsInternal='true']
So TeamCity cannot know the state of tests until suite is done and XML file is written.
Now, TeamCity supports real-time tests reporting with messages such as
##teamcity[testFailed name='MyTest.test1' message='failure message' details='message and stack trace']
However, Surefire (the framework responsible for running JUnit tests in Maven) does not output such messages by default.
So it seems that I have three choices if I want tests to be reported in real-time:
- Configure a RunListener for JUnit/Surefire to output these messages for TeamCity. Obvious problem here is that I am reinventing the wheel here. I'm also not sure that my reporting of tests will closely match Surefire's reporting of tests in XML file, which may lead to glitches and duplication of tests.
- Change my tests to emit these messages. Easily doable since I have a common superclass for almost all tests, but reinventing the wheel is still there, as well as mismatch with XML.
- Find a way to either configure Surefire to output these messages, or make TeamCity talk to SureFire to enable their output. I recognize that Surefire is hit-or-miss thing, but this functionality is so popular I expect it has some solution.
Please help me choosing between these options. I have tried googling, the results are very poor.