3

I have several tests that should be executed regardless of each other's success and I want Jenkins/Hudson to display red light if at least one of those tests failed. My current (simplified for clarity) configuration is as following:

ci.sh:

...
ant
...

build.xml:

...        
<target name="AllTests">
    <antcall target="TestA"/>
    <antcall target="TestB"/>
    <antcall target="TestC"/>
</target>

<target name="TestA">
    ...
    <exec executable="..." failonerror="false"/>
    ...
</target>

<target name="TestB">
    ...
    <exec executable="..." failonerror="false"/>
    ...
</target>

<target name="TestC">
    ...
    <exec executable="..." failonerror="false"/>
    ...
</target>
...

How can I make all the tests to execute anyways, but ant/Jenkins should fail if at least one of the three failed?

6 Answers 6

2

I would set a variable in your ant script, and have the ant script exit with that variable if the test fails. Jenkins jobs fail, if the exit code is anything but zero. Like below:

#!/bin/bash
zero="0"

*** run your ant script ***

if [$? -gt $zero]; # check to see if exit code of ant script is great than zero
then
    exit(5) # non-zero exit means failure.
else
    echo Success
    exit(0)
fi
5
  • I've just found a viable workaround, but your answer seems to be more correct and generic. Would it be possible to ask for a code snippet, and I'd accept your answer; mine will be there just as a supplementary? Jul 14, 2011 at 7:46
  • I am not familiar with ant scripts so I will write in bash; it will get the point across. Jul 14, 2011 at 13:59
  • Thanks, but we've missed the point here. It's about how ant should support it. Three targets, one fails, all the task fails but all the three targets should be executed. Thanks anyways. Jul 14, 2011 at 15:13
  • you said you wanted to set build to red if any of the builds failed. I told how to? you alternately did that by limiting it to one failing thread as supposed to how many were running before. My solution does answer you question. It will help others. If it didn't help you its all good. :) Jul 14, 2011 at 19:55
  • Sure it will help others and many thanks for that. I am just trying to follow the mainstream of ant as a primary option. Jul 17, 2011 at 7:29
1

I have found parallel task with thread count set to "1" as a viable workaround. This isn't perfect, but the change in build.xml is minimal:

build.xml:

...        
<target name="AllTests">
    <parallel threadCount="1" timeout="900000">
        <antcall target="TestA"/>
        <antcall target="TestB"/>
        <antcall target="TestC"/>
    </parallel>
</target>

<target name="TestA">
    ...
    <exec executable="..." failonerror="false"/>
    ...
</target>

<target name="TestB">
    ...
    <exec executable="..." failonerror="false"/>
    ...
</target>

<target name="TestC">
    ...
    <exec executable="..." failonerror="false"/>
    ...
</target>
...
0

Huh? AFAIK, ant per default does not stop on test failures or test errors. And Hudson/Jenkins can be told to collect JUnit test reports afterwards and alter the build status depending on the test results... (there is at least one plugin for Jenkins that allows you to set watermarks)

See http://ant.apache.org/manual/Tasks/junit.html - haltonfailure and haltonerror are off by default.

0

This is sequential ant solution, but it changes your build.xml a little more. It uses a temporary file to flag that one test failed.

<?xml version="1.0" encoding="UTF-8"?>
<project name="project" default="AllTests">
    <property name="test-failed.file.path" location="test.failed" />

    <macrodef name="checkTestResult">
        <sequential>
            <condition property="allOK">
                <equals arg1="${test.result}" arg2="0" />
            </condition>
            <antcall target="-create-test-failed-file" />
        </sequential>
    </macrodef>

    <target name="-create-test-failed-file" unless="${allOK}">
        <touch file="${test-failed.file.path}" />
    </target>

    <target name="AllTests">
        <delete file="${test-failed.file.path}" />

        <antcall target="TestA" />
        <antcall target="TestB" />
        <antcall target="TestC" />

        <available file="${test-failed.file.path}" property="oneTestFailed" />
        <delete file="${test-failed.file.path}" />
        <fail if="${oneTestFailed}" message="At least one test failed" />
    </target>

    <target name="TestA">
        <exec executable="hostname"
              failonerror="false"
              resultproperty="test.result" />
        <checkTestResult />
    </target>

    <target name="TestB">
        <!-- This one fails -->
        <exec executable="hostname"
              failonerror="false"
              resultproperty="test.result">
            <arg value="--NoSuchOption" />
        </exec>
        <checkTestResult />
    </target>

    <target name="TestC">
        <exec executable="hostname"
              failonerror="false"
              resultproperty="test.result" />
        <checkTestResult />
    </target>
</project>
0

You can install Log Parser Plugin to fail the build on certain pattern in the console log.

So all the scripts may run, but the build may fail at the end when errors/warnings appear in the log section (which you can configure to be printed).

The rules can be configured in the Parsing Rules File, like:

# match line starting with 'error ', case-insensitive
error /(?i)^error /

# list of warnings here...
warning /[Ww]arning/
warning /WARNING/

In case you've 3 scripts and Jenkins stops build on the 1st one, you can add #!/bin/sh -x to the Execute shell to not stop on the first error and use above mentioned Log Parser plugin to fail the build at the end.

See also: How/When does Execute Shell mark a build as failure in Jenkins?

0

Just adding my point: If you want to manually fail your ANT task and want your jenkins job to fail.

Then you can use something like this

<fail status="-1" message="" />

jenkins jobs fails based on the errorcode returned by Ant task.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.