If you are finding it difficult to decide which testing framework to use for your project, this post would help you make the right decision based on the features available in the two frameworks. I have also listed the most common pain points that developers face while writing test cases.

 

FEATURES

Fixtures

 

Features TestNG JUnit
Run test method @Test @Test
Run method before first method in class @BeforeClass @BeforeClass
Run method after last method in class @AfterClass @AfterClass
Run method before each method in class @BeforeMethod @Before
Run method after each method in class @AfterMethod @After
Ignore a test method @Ignore @Ignore
Expected exception @Test(expected=Exception.class) @Test(expected=Exception.class)
Timeout @Test(timeout=100) @Test(timeout=100)
Run method before first method in suite @BeforeSuite You have to create a suite class and add @BeforeClass to it
Run method after last method in suite @AfterSuite You have to create a suite class and add @AfterClass to it
Run method before first method in group @BeforeGroup JUnit has similar concept of Category which can be included or excluded in a suite
Run method after last method in group @AfterGroup JUnit has similar concept of Category which can be included or excluded in a suite

Ordering

Ordering is needed when some test cases are supposed to be run before/after other test cases. For example, running loginTest before updateNameTest or updateEmailTest might make more sense as the latter are more likely to succeed if loginTest succeeds before them. In TestNG, it is easily achievable by explicitly adding dependency compared to JUnit where you can run test methods based on alphabetical order.

 

Features TestNG JUnit
Ordering @Test(dependsOnMethods={“login”}) @FixMethodOrder(MethodSorters.NAME_ASCENDING)

Suite test

You might want to create a suite of unit tests and another suite of integration tests and have them run separately in some scenarios. Both TestNG and JUnit have support for creating a suite, but in TestNG you can easily do that in a simple xml file, while in JUnit you have to create a class and add @RunWith, @SuiteClasses etc do the same thing. Suites also allow you to run specific types of test methods, TestNG calls them as group and JUnit calls them as category.

 

Features TestNG JUnit
Creating a
suite
<suite name=”unit tests” ><test name=”manager tests” ><groups><run>

<include name=”slow”/>

</run>

</groups>

<classes>

<class name=”com.manager”  />

<classes>

</test>

</suite>

@RunWith(Suite.class)@Suite.SuiteClasses({AdminManager.class,UserManager.class})

@IncludeCategory(SlowTest.class)

public class UnitTestSuite {

// the class remains empty,

// used only as a holder for the above
annotations

// add @BeforeClass or @AfterClass to
run methods before/after first/last
method in this suite

}

Data providers and parameterised tests

You might want to run test cases with dynamic data, i.e. data that is generated by a method and passed to the test method. JUnit provides @Parameters that can be added to a static method that contains data, this method is called while instantiating the test class. The return value is passed to the test class as an argument, that means the test class constructor has to accept arguments. JUnit also provides @Parameter that can be added to any member field to set the data directly to the field instead of constructor. TestNG provides better features, as it allows you to add parameters directly at the test method and also at the xml configuration file. It also allows you to assign a method as data provider which any test method can use to get data dynamically.

 

Features TestNG JUnit
Data provider //This method will provide data to any
test method that declares that its Data Provider//is named “test1″@DataProvider(name = “test1″)public Object[][] createData1() {return new Object[][] {

{ “Cedric”, new Integer(36) },

{ “Anne”, new Integer(37)},

};

}
//This test method declares that its data
should be supplied by the Data Provider

//named “test1″

@Test(dataProvider = “test1″)

public void verifyData1(String n1, Integer n2) {

System.out.println(n1 + ” ” + n2);

}

@RunWith(Parameterized.class)public class FibonacciTest {@Parameterspublic static Collection<Object[]> data() {return Arrays.asList(new Object[][] {

{ 0, 0 }, { 1, 1 }, { 2, 1 }, { 3, 2 }

});

}
@Parameter // first data value (0) is default

public /* NOT private */ int fInput;
@Parameter(value = 1)

public /* NOT private */ int fExpected;
@Test

public void test() {

assertEquals(fExpected, Fibonacci.compute(fInput));

}

}

DEVELOPER PAIN POINTS

Parallelism

A lot of developers want to run test cases in parallel to reduce the overall execution or build time.  Some developers want to run suites separately in parallel. Both TestNG and JUnit have support to run test cases in parallel. Both satisfy the needs.

 

@BeforeClass method must be static

In JUnit, @BeforeClass can only be applied to static method that executes before first test method in the class. This is a problem for many developers. The reason is that they can’t initialize any instance variables of the test class, because the method can only access static variables (static methods can only access static variables). One work-around is to make the variables also static and initialize them inside the @BeforeClass method. But this may lead to concurrency issues.  TestNG solves this issue by not putting any such condition on the method. In TestNG, you can apply @BeforeClass to non-static methods as well.

 

Dependency tests and ordering

As mentioned above in this post, there are some scenarios where developers want to execute test cases in specific order due to dependencies among test cases. JUnit does not provide a flexible solution to this problem. You can only execute test cases in alphabetical order of the test case names. TestNG, however, solves this problem in a better way by allowing you to specifically set the dependencies for any test case. For example, you can set “test1″ and “test2″ as dependency for “test3″ this way TestNG would run “test1″ and “test2″ before running “test3″.

 

Grouping and creating suite

As mentioned above in this post, both TestNG and JUnit have support for creating a group or suite. The difference is how easy it is to create them. In TestNG it can be done at a single place, i.e. the TestNG configuration file testng.xml. In JUnit, you have to create a class for doing the same thing.

 

CONCLUSION

If you aren’t already using any framework, you can go with TestNG as its easy to configure and maintain. However, if you are already using JUnit, I would suggest you to upgrade to latest version of JUnit that has more features for grouping, parallelism, assertions etc.