1

I'm trying to run all tests in the project using testng.xml I have 80 classes with ~ 140 tests but every time I run test, testng runs only 83/140 tests. I tried to find pattern in tests that are not being executed but could not find any.

All tests/class files are under 'scripts' package and all class files have:

package scripts;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;

All tests have @Test annotation (i.e. there are 140 @Test tags) @Test(priority=1,testName="",description="")

I am NOT using enabled="false" in @Test annotation

My testng.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Regression">
    <test name="Regression">
        <packages>
            <package name="scripts"/>
        </packages>
    </test>
</suite>

Note: here around 57 tests are skipped for some reason. e.g. one of them is addMsd

  1. For some classes, if class has 3 tests, only 2 are executed
  2. For some classes, whole class file (having single or multiple tests) is skipped

If I add an additional parameter and add a test (that is usually skipped with above XML) as follows then it is executed successfully:

`<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Regression">
    <test name="Regression">
        <packages>
            <package name="scripts"/>
        </packages>
    </test>
    <test name="test">
        <classes>
            <class name="scripts.CalculateMSD">
                <methods><include name="addMsd"/></methods>
            </class>
        </classes>
    </test>
</suite>`

Here, addMsd will be executed successfully (which would be skipped in first xml)

I have tried running tests using IntelliJ testng plugin (right click > run) and maven surefire plugin and observed same behavior in both approaches.

I can't figure out what could be wrong with XML or the project setting. Any pointers will help!

2
  • did you check the console ? does it contain any error ? Feb 19, 2019 at 8:32
  • There is no error on console. But what I noticed is that most of the tests that run successfully are priority=1 (or priority=2) and ALL the tests that are skipped are priority = 2. If I remove priorities for all tests, then all tests are executed !! I suspect it has something to do with the priorities (e.g. if priority 1 test fails in a class then don't execute priority 2 test?).... Does testng has any logic for skipping tests based on priority?
    – Yogesh
    Feb 20, 2019 at 0:46

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.