2

I've been creating unit tests using JUnit in Eclipse and the basic principles of JUnit are clear to me. The tests run as they should and now it's time to find a good way to report on the failed tests. Most examples I come accross on the web are using some sort of built system like Ant, Maven or Hudson to run the tests at the same time as when they're building the project, but don't need this integration in the build process. The tests should be able to run independent from a build system. In the end these systems just take the info they get from JUnit and put it in a nice HTML layout.

I learned that it's possible to read out the JUnit information by creating a custom listener via the RunListener class of JUnitCore and handle the failures as they happen using the "testFailure" method. When creating my tests, I always provided a custom message for the assertion errors. My problem is that this message does not show up in the failure trace of JUnit. Instead, the failure trace would show "java.lang.Exception: could not invoke *". When I look at the console, I see the same stack trace but below also folows a line that is preceded by "Caused by: *" and here I do see my custom error message.

Caused by: java.lang.AssertionError: <my custom error message>
    at *

As I'm just getting started with JUnit and I do not know if this is normal behaviour of JUnit or is there something wrong in the way I implemented my tests and should I just see my custom messages in the failure trace? I also noticed that when I catch the assertion error in the method where I perform the test, I CAN access the message using "error.getMessage()". So if necessary, this would be a solution and would also allow me to include additional information in the report that is accessible at the moment I catch the error (i.e. url of webpage, etc.) but putting a try-catch block around every assertion would defeat its purpose in my opinion.

As I said I'm new to unit testing but also java development in general, so it's very likely that I'm making some stupid mistakes. Any pointers are very welcome!

Update 2: Below an anonymized version of the stack trace on the console. Maybe it helps to clarify my question.

    java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at xxxxx.xxxxxxxxxxxx.xxxxxxxxx(xxxxxxxxxxx.java:103)
        at xxxxx.xxxxxxxxxxxxxx.xxxxxx(xxxxxxxxxxx.java:62)
        at junit.framework.TestCase.runBare(TestCase.java:134)
        at junit.framework.TestResult$1.protect(TestResult.java:110)
        at junit.framework.TestResult.runProtected(TestResult.java:128)
        at junit.framework.TestResult.run(TestResult.java:113)
        at junit.framework.TestCase.run(TestCase.java:124)
        at xxxxx.xxxxxxxxxxxx.xxxxx(xxxxxxxxxxxx.java:90)
        at junit.framework.TestSuite.runTest(TestSuite.java:243)
        at junit.framework.TestSuite.run(TestSuite.java:238)
        at xxxxx.xxxxxxxxxxxxxxxxxxxx.xxxx(xxxxxxxxxxxxxxxx.java:87)
        at junit.framework.TestSuite.runTest(TestSuite.java:243)
        at junit.framework.TestSuite.run(TestSuite.java:238)
        at xxxxx.xxxxxxxxxxxxxxxxxxxxxx.xxxxx(xxxxxxxxxxxxxx.java:87)
        at junit.framework.TestSuite.runTest(TestSuite.java:243)
        at junit.framework.TestSuite.run(TestSuite.java:238)
        at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
        at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
        at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
        at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
    Caused by: java.lang.AssertionError: <xx my custom message xx>.
        at xxx.xxxxxxxx.xxxxxxxxxxxxxx(xxxxxxxx.java:100)
        ... 27 more

Below, an anonymized version of the stack trace reported by JUnit java.lang.Exception: could not invoke action on current page

java.lang.Exception: could not invoke action <xx action name xxx> on current page class xxxx.xxxxxxx: [Ljava.lang.StackTraceElement;@71d382ab
at xxxxx.xxxxxxxxxxxxxxxxx.xxxxx(xxxxxxxxxxxxxxxxxxxx.java:65)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at xxxxx.xxxxxxxxxxxxxxxxx.xxxxx(xxxxxxxxxxxxxxxxxxxx.java:90)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at xxxxx.xxxxxxxxxxxxxxxxx.xxxxx(xxxxxxxxxxxxxxxxxxxx.java:87)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at xxxxx.xxxxxxxxxxxxxxxxx.xxxxx(xxxxxxxxxxxxxxxxxxxx.java:87)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:24)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
at xxxxx.<xx class that initiates the RunListener xx>.main(<xx class that initiates the RunListener xx>.java:11)

1 Answer 1

1

The tests should be able to run independent from a build system.

The test will be independent even if you use a build system. Its purpose is to make your life easier in the long run so I suggest you consider using one. I suggest maven which I didn't like at first because it tells you how to layout things like directories, but now I can see this makes your life easier.

When I look at the console, I see the same stack trace but below also folows a line that is preceded by "Caused by: *" and here I do see my custom error message.

Most tools remove this so you only see what you need. e.g. in most IDEs you can click on the stack trace and navigate to the line of code which cause the problem. If you use maven, the IDE will be in sync with your maven configuration so it can find the code responsible.

I also noticed that when I catch the assertion error in the method where I perform the test, I CAN access the message using "error.getMessage()".

You don't need to do any of this if you use maven and/or an IDE.

it's very likely that I'm making some stupid mistakes.

Its doesn't appear to be the case. I would suggest you use the tools most developers use (like and IDE and maven) and most of these issue are handled for you. (Also many other issues you might never know it did for you ;)

7
  • Thanks for your quick reply @peter. I do see the advantages of using a build system, but this is a one-off assignment for which getting familiar with these build systems would cost me too much time. Since the JUnit tests run dependently and I should be able access all information of the failures/errors through the listener I created, does anyone know if there is a way to configure the listener to show me the custom messages and not remove them from the stack trace?
    – pascie
    Jul 31, 2012 at 8:26
  • I thought you stated are in your stack trace, just not at the top. Try using e.getCause().getMessage() instead. Its never a good time to learn a build system, it just something you have to make time for ;) Jul 31, 2012 at 8:40
  • The "caused by" part is indeed shown in my error stack in the console window, but the Failure Trace provided by JUnit seems to chop it off. And the failure.getMassage() method returns the description of the exception in the first line of the stack trace. I tried to use failure.getCause().getMessage()in the testFailure() method of my listener and when I run it nothing inside de testFailure() method is executed anymore. @Peter Thanks for working towards a solution here with me!
    – pascie
    Jul 31, 2012 at 9:48
  • I believe JUnit does work as expected, but I should investigate why it reports that it fails to invoke the method instead of the underlying cause, which is the failure of my assertion test.
    – pascie
    Jul 31, 2012 at 10:00
  • 1
    Ok, I finally cracked it. In one of the parent classes that eventually called the method in which the assertion test was executed, a try-catch block was used to catch InvocationTargetExceptions. This was hiding my more detailed errors trown by JUnit!
    – pascie
    Jul 31, 2012 at 11:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.