4

How do I instruct JUnit (via Ant) to capture sdout and stderr per test in a test suite? Is this even possible?

Lets say I have a simple Ant task that runs a suite of verbose JUnit tests.

<target name="integration-tests" depends="init">
    <property name="test.reports" value="${build.dir}/test-reports" />
    <mkdir dir="${test.reports}" />
    <junit showoutput="true" printsummary="yes" fork="yes">
        <formatter type="xml" />
        <classpath>
            <pathelement path="${run.test.classpath}" />
        </classpath>
        <test name="org.example.IntegrationTestsTestSuite" todir="${test.reports}" />
    </junit>
</target>

When a CI (in my case Jenkins) invokes this target, JUnit generates a neat XML file for the CI to process further (publish). Unfortunately all stdout and stderr is merged into values of two XML elements.

<?xml version="1.0" encoding="UTF-8" ?>
<testsuite errors="0" failures="0" hostname="foo-host" name="org.example.IntegrationTestsTestSuite" skipped="0" tests="3" time="0.115" timestamp="2018-06-06T11:45:45">
  <properties>
    <!-- all properties used by Ant build script here -->
  </properties>
  <testcase classname="org.example.integration.tests.CommonStuff" name="testSomeFooDoingAThing" time="0.088" />
  <testcase classname="org.example.integration.tests.CommonStuff" name="testOtherFooDoingAThing" time="0.023" />
  <testcase classname="org.example.integration.tests.CommonStuff" name="testAnotherFooDoingAThing" time="0.004" />
  <system-out><![CDATA[]]></system-out>
  <system-err><![CDATA[]]></system-err>
</testsuite>

If something goes wrong with one test, the entire verbose output is displayed by the CI at once even the one of successful tests (when viewing a single failed test), defeating the intended purpose of such output.

The same problem exists when tests are run inside our IDE (Netbeans). But I remember from other projects that at least IntelliJ was able to show such verbose output per test (not sure if it applies to test suites though). Is IntelliJ doing something else to post/pre-process the output or is this a JUnit feature?

2
  • IntelliJ captures std(out/err) streams and splits them into separate text fields, it barely helps you. Take a look at JUnit's '@Rule' and '@BeforeClass' annotations. Jun 6, 2018 at 12:47
  • @MikhailKholodkov, what you are saying here is that if such a thing is needed, it needs to be handled by the tests themselves. Maybe use a TestWatcher, make it save std to a file, then link this file to a specific test case somehow? Or maybe even generate a test report programmatically.
    – predi
    Jun 6, 2018 at 13:07

1 Answer 1

1

There doesn't seem to be a built-in way for JUnit to do this via Ant. I ended up implementing this functionality on my own.

As per @Mikhail's suggestion I took a look at @Rule annotation, which lead me to JUnit's TestWatcher class. It may be used to handle test reports programmatically (it is an observer of a single test method's status events like "starting", "failed", "finished", etc.).

Unfortunately, a @Rule annotation cannot be applied to an entire test suite easily as it is intended to be used where test methods appear. I had to implement a custom Suite and Runner and fake presence of @Rule in each class containing test cases (via RunRules inside the Runner implementation).

After that I simply took a look at how a JUnit XML report is structured, collected all required test statistics and output with a TestWatcher (simply calls System.setOut() and System.setErr() when a test method starts, then restores those after test finishes) and wrote the report file myself.

The Ant task no longer needs the <formatter> element, since it just needs to invoke a Suite definition that now handles this implicitly. I did however use <sysproperty> to pass a directory name to the custom Suite, so it knows where to store the resulting file via System.getProperty(String).

This is completely doable using JUnit version 4.12.

2
  • 2
    You wouldn't happen to have code snippets or a repo somewhere for this, would you?
    – wufufufu
    Sep 26, 2018 at 14:55
  • @wufufufu, unfortunately no, but if you follow above links, you get all the required snippets, except the XML writing part.
    – predi
    Sep 27, 2018 at 8:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.