Как заставить сборку Jenkins завершиться неудачно, когда модульные тесты Maven не пройдены?

Question

Я использую Jenkins, Maven 3.1 и Java 1.6. У меня в Дженкинсе настроена следующая работа в Maven со следующими целями и вариантами...

clean install -U -P cloudbees -P qa

ниже моя pom.xml верная конфигурация...

<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.17</version>
        <configuration>
                <reuseForks>true</reuseForks>
                <argLine>-Xmx2048m -XX:MaxPermSize=512M </argLine>
                <skipTests>false</skipTests>
        </configuration>
</plugin>

Однако, когда мои модульные тесты терпят неудачу, вывод консоли Jenkins по-прежнему говорит "BUILD SUCCESS", и сборка помечается как "нестабильная", а не полная ошибка. Как мне сконфигурировать вещи в Jenkins (или Maven, если это так), чтобы моя сборка не удалась (не стала нестабильной или прошла), если какой-либо из модульных тестов не пройден?

Вот что говорит консольный вывод

17:08:04   MyProjectOrganizationControllerTest.testRecoverFromError » IllegalState Failed to...
17:08:04   MyProjectOrganizationControllerTest.testVerifyDistrictListPopulated » IllegalState
17:08:04   MyProjectOrganizationControllerTest.testUpdateSchool » IllegalState Failed to loa...
17:08:04   MyProjectOrganizationControllerTest.testDeleteSchool » IllegalState Failed to loa...
17:08:04   MyProjectOrganizationControllerTest.testVerifyOrgListPopulatedPrivateSchoolOrgType » IllegalState
17:08:04   MyProjectOrganizationControllerTest.testSubmitMultipleParams » IllegalState Faile...
17:08:04 
17:08:04 Tests run: 155, Failures: 0, Errors: 154, Skipped: 1
17:08:04 
17:08:04 [ERROR] There are test failures.
17:08:04 
17:08:04 Please refer to /scratch/jenkins/workspace/MyProject/MyProject/target/surefire-reports for the individual test results.
17:08:04 [JENKINS] Recording test results
17:08:07 log4j:WARN No appenders could be found for logger (org.apache.commons.beanutils.converters.BooleanConverter).
17:08:07 log4j:WARN Please initialize the log4j system properly.
17:08:14 [INFO] 
17:08:14 [INFO] --- maven-war-plugin:2.4:war (default-war) @ MyProject ---
17:08:15 [INFO] Packaging webapp
17:08:15 [INFO] Assembling webapp [MyProject] in [/scratch/jenkins/workspace/MyProject/MyProject/target/MyProject]
17:08:15 [INFO] Processing war project
17:08:15 [INFO] Copying webapp resources [/scratch/jenkins/workspace/MyProject/MyProject/src/main/webapp]
17:08:15 [INFO] Webapp assembled in [662 msecs]
17:08:15 [INFO] Building war: /scratch/jenkins/workspace/MyProject/MyProject/target/MyProject.war
17:08:20 [INFO] 
17:08:20 [INFO] --- maven-failsafe-plugin:2.17:integration-test (default) @ MyProject ---
17:08:20 [JENKINS] Recording test results
17:08:25 [INFO] 
17:08:25 [INFO] --- maven-failsafe-plugin:2.17:verify (default) @ MyProject ---
17:08:25 [INFO] Failsafe report directory: /scratch/jenkins/workspace/MyProject/MyProject/target/failsafe-reports
17:08:25 [JENKINS] Recording test results[INFO] 
17:08:25 [INFO] --- maven-install-plugin:2.4:install (default-install) @ MyProject ---
17:08:25 
17:08:25 [INFO] Installing /scratch/jenkins/workspace/MyProject/MyProject/target/MyProject.war to /home/jenkins/.m2/repository/org/mainco/subco/MyProject/76.0.0-SNAPSHOT/MyProject-76.0.0-SNAPSHOT.war
17:08:25 [INFO] Installing /scratch/jenkins/workspace/MyProject/MyProject/pom.xml to /home/jenkins/.m2/repository/org/mainco/subco/MyProject/76.0.0-SNAPSHOT/MyProject-76.0.0-SNAPSHOT.pom
17:08:26 Started calculate disk usage of build
17:08:26 Finished Calculation of disk usage of build in 0 seconds
17:08:26 Started calculate disk usage of workspace
17:08:26 Finished Calculation of disk usage of workspace in 0 seconds
17:08:26 [INFO] ------------------------------------------------------------------------
17:08:26 [INFO] BUILD SUCCESS
17:08:26 [INFO] ------------------------------------------------------------------------
17:08:26 [INFO] Total time: 11:00.616s
17:08:26 [INFO] Finished at: Mon Feb 23 17:08:26 UTC 2015
17:08:27 [INFO] Final Memory: 90M/414M
17:08:27 [INFO] ------------------------------------------------------------------------
17:08:27 Waiting for Jenkins to finish collecting data
17:08:28 [JENKINS] Archiving /scratch/jenkins/workspace/MyProject/MyProject/pom.xml to org.mainco.subco/MyProject/76.0.0-SNAPSHOT/MyProject-76.0.0-SNAPSHOT.pom
17:08:28 [JENKINS] Archiving /scratch/jenkins/workspace/MyProject/MyProject/target/MyProject.war to org.mainco.subco/MyProject/76.0.0-  SNAPSHOT/MyProject-76.0.0-SNAPSHOT.war
17:08:31 channel stopped
17:08:31 Started calculate disk usage of build
17:08:31 Finished Calculation of disk usage of build in 0 seconds
17:08:31 Started calculate disk usage of workspace
17:08:31 Finished Calculation of disk usage of workspace in 0 seconds
17:08:31 Finished: UNSTABLE

Answer 1

Можете добавить -Dmaven.test.failure.ignore=false к MAVEN_OPTS если вы нажмете кнопку "Дополнительно" в разделе "Построение" вашей работы Jenkins.

См. Maven Surefire Plugin - surefire: варианты тестирования для справки.

Answer 2

Используйте плагин Text Finder. Настройте его для поиска There are test failures и понизить сборку до FAILED

Answer 3

Еще один полезный хак - использование groovy post-build для проверки и установки результатов теста.

например, этот groovy получает результат компоновки, добавляет полезные вещи для описания компоновки и устанавливает результат компоновки в UNSTABLE в случае, если нет проходов или неудач, но все тесты пропущены.

def currentBuild = Thread.currentThread().executable
// must be run groovy post-build action AFTER harvest junit xml  if you use junit xml test results
testResult1 = currentBuild.testResultAction

currentBuild.setDescription(currentBuild.getDescription() + "\n pass:"+testResult1.result.passCount.toString()+", fail:"+testResult1.result.failCount.toString()+", skip:"+testResult1.result.skipCount.toString())

// if no pass, no fail all skip then set result to unstable
if (testResult1.result.passCount == 0 && testResult1.result.failCount == 0 && testResult1.result.skipCount > 0) {
   currentBuild.result = hudson.model.Result.UNSTABLE
}

currentBuild.setDescription(currentBuild.getDescription() + "\n" + currentBuild.result.toString())

def ws = manager.build.workspace.getRemote()
myFile = new File(ws + "/VERSION.txt")
desc = myFile.readLines()
currentBuild.setDescription(currentBuild.getDescription() + "\n" + desc)