TestNG Vs JUnit – What’s the Difference?
TestNG and JUnit is are open-source testing frameworks for Java. Comparing TestNG Vs JUnit, TestNG annotations are easier to use and understand than JUnit. TestNG allows us to create parallel tests, whereas JUnit does not support running parallel tests.
You can execute your existing JUnit test cases using TestNG. TestNG can automatically recognize and run JUnit tests, so that you can use TestNG as a runner for all your existing tests and write new tests using TestNG.
In this post, we'll discuss and compare these frameworks by covering their features and common use cases.
TestNG is an open-source testing framework where NG stands for 'Next Generation'. It is architected to simplify a broad range of testing needs starting from unit testing to integrated system testing
Features of TestNG testing framework:
· Supports annotations.
· TestNG uses more Java and Object Oriented features.
· Separates compile-time test code from run-time configuration/data info.
· Flexible runtime configuration and data driven testing
· It processes Unit tests with passing parameters into the test methods.
· Making use of Threads enhances testing performance better as execution time is reduced.
· Supports Dependent test methods, parallel testing, ordering and grouping of tests, load testing, and partial failure.
· In Built HTML reporter
· XML based test configuration
JUnit is an open-source testing framework. JUnit empowers Test Driven Development (TDD), where tests are written before writing actual code.
JUnit follows the approach of “testing first then coding”. This emphasizes setting up test data and testing the small piece of code first and then implementing the same. This increases productivity and helps in improving code stability. JUnit finds the bug early in code compared to other test frameworks.
Features of JUnit testing framework:
· There is a humongous list of Annotations to identify, execute, and support many features for the test methods and assertions for verifying the expected results.
· It provides Test Runner for executing the tests.
· JUnit provides a basic built-in template so that you may write small, simple test cases in no time.
· JUnit tests help you to write independent modules, thereby bettering the coverage of the test and the quality of the application.
· Junit provides robust assertions to test expected outcome
· Until the test execution is sailing through smoothly, you may relax watching at the green-colored test progress bar that shows while execution is in progress whereas it alerts you in ‘red’ as soon as the test fails a verification checkpoint.
· Test suites can be created in order to put a sequence or related set of test cases together.
· JUnit is non-supportive of parallel test execution
· JUnit aids the development of test-driven testing in a big way
· Junit tests can be integrated into CI/CD Pipeline
Feature Comparison chart for TestNG and JUnit
Following table compares JUnit vs TestNG on different features. The table excludes very specific or common features that are present in both these frameworks, such as testing exceptions, timeout settings, lifecycle callback methods etc.
Both JUnit 5 and TestNG are annotation based. They are similar in nature and behavior.
Ease of Use
Running tests need a certain dependency in JUnit
Writing tests and configuring them is easy in TestNG than JUnit
TestNG is better in terms of ease of use.
Supported on major IDEs such as Eclipse and IntelliJ IDEA.
Supported on major IDEs such as Eclipse and IntelliJ IDEA.
Both of them are similar and provides easy integration with major IDEs.
Supports multiple ways to provide test data, such as methods, Enum, CSV, CSV Files etc.
Supports data provider methods and from test suite xml file.
JUnit is better for injecting test methods input data
In JUnit, Test Suite is run using @RunWith and @Suite annotations.
TestNG uses an XML file to run Suite test.
JUnit needs external plugin maven-surefire-report-plugin to generate HTML reports.
TestNG automatically creates HTML reports for the test run.
TestNG HTML reports look outdated but it’s simple to use.
Running from Java Main Method
We can use JUnit 5 launcher API to run tests from java main method.
We can use TestNG run() method to execute tests from the java main method.
Both of them supports execution of test cases from java main method.
JUnit provides enough assertion methods to compare expected and actual test results.
TestNG provides enough assertion methods to compare expected and actual test results.
Both of them are similar in terms of Assertions support.
JUnit supports assumptions to skip tests based on certain conditions.
TestNG doesn’t support assumptions.
JUnit is better if you want to skip tests based on conditions.
Junit does not support Test Order.
TestNG supports Test Order for ordering test methods according to priority attribute.
TestNG is better when you want to execute tests in specific order.
JUnit supports many ways to disable and enable tests. For example, based on OS, JRE, system properties.
TestNG supports disabling tests but it’s limited in functionality.
JUnit is better when you want to disable or enable tests based on conditions.
JUnit 5 doesn’t support parallel execution yet.
TestNG supports parallel execution if run through XML suite.
TestNG is better for parallel execution as of now, JUnit 5 development is going on to support this feature.
JUnit supports listeners through Launcher API, there is no mechanism to add listeners using annotations.
TestNG supports various types of listeners and can be added using annotations.
TestNG listener support is much better compared to JUnit 5.
The dependency tests are missing in JUnit.
Dependency tests are present in TestNG.
This is an advanced feature in TestNG over JUnit. With this feature, one method can be made as dependent on other such that the method will run only after the dependent method runs and passes else the dependent test won't run
Comparison of annotations and implementation between JUnit and TestNG
1. Test method annotation
There is no difference in the way we specify a method as a test method in both JUnit and TestNG.
2. Suite related annotation
A method with annotation @BeforeSuite is executed once before the current test suite runs. This annotation is applicable only in TestNG.
3. Annotation for a method before Class
This is the annotation for the method to be executed once before the first test method in the class is run.
4. Annotation for a method before Test
This annotation executes once before the methods declared inside <test> tag of testng.xml. This annotation is available for TestNG only.
5. Annotation for the method to be executed before every method with @Test invokes
6. Annotation for the method to be executed after every method with @Test invokes
7. Annotation for method after Test
This annotation executes once after the methods declared inside <test> tag of testng.xml. This annotation is available for TestNG only.
8. Annotation for method after Class
This is the annotation for the method to be executed once after the last test method in the class is run.
9. Suite related annotation
A method with annotation @AfterSuite is executed once after the current test suite runs. This annotation is applicable only in TestNG.
10.Group related annotation
The annotation is available only in TestNG.
The method with annotation @BeforeGroups runs before the test methods belonging to a particular group runs.
The method with annotation @AfterGroups runs after the test methods belonging to a particular group runs.
11. Annotation to disable the execution of the Test method.
JUnit 5 provides an annotation to disable a specific test execution.
Below is a code snippet for how we implement in Junit.
TestNG provides an attribute for @Test i.e. ‘enabled’ with the boolean value which decides if the execution of the method would be disabled or enabled
12. Timeout annotation
The annotation is same for JUnit 5 and TestNG
Below is an example in Junit for Timeout test.
Code example for TestNG
As you can see from the test results, only timeoutTestTwo() is passed because its execution time was less than the timeout time defined. The timeoutTestOne() execution got failed because it took more time to complete than the timeout duration configured.
The timeout can be configured in the testNG.xml also as shown below
<suite name="Time test Suite" time-out="500" verbose="1" >
13. Expected Exception attribute
Exception class states that when the test executes, the exception of the given class is thrown. Exception test makes sure that when there is this predefined exception thrown, it is gracefully caught and notified on the logs. This is supported both in JUnit and TestNG with variation in the way both are declared.
Lets look at an example for Junit. Add an error condition inside the showMessage() method.
Add an exception ArithmeticException to the displayMessage() test case.
The only difference in TestNG is “expectedException” is used in place of “expected” beside the @Test annotation.
14. Order of Execution related annotations
Both JUnit and TestNG support explicitly setting the order of tests for execution. In other words, setting the priority for test cases.
JUnit 5 has annotation @TestMethodOrder() with MethodOrderer package’s built-in class – Alphanumeric.class or OrderAnnotation.class or Random.class as an input parameter for the annotation.
In the below code we can see that MethodOrderer orders all the tests alphabetically irrespective of the order they are declared.
Example for Junit
TestNG includes the attribute ‘priority’ for @Test annotation, which accepts a numerical value.
Example for TestNG
In TestNG, lower the priority number; higher is the priority of the test case method. Priority can contain only integer value. Here we can see that testcase 2 whose priority is -1 is executed first
15. Custom Test name
By default, whenever we run a test, the test class and the test method name is printed in console. Junit5 provides a unique feature where we can mention custom descriptive names for class and test methods using @DisplayName annotation.
In TestNG there is no way to provide a custom name.
16. Dependent Tests
Junit doesn’t offer Dependency testing.
TestNG supports dependency testing. When the second test method wants to be dependent on the first test method, then this could be possible by the use of "dependsOnMethods" attribute. If the first test method fails, then the dependent method on the first test method, i.e., the second test method will not run. Let us understand through an example.
17. Parameterized Tests
Parameterized unit tests are helpful for testing the same code under several conditions. The main idea is to make the unit test method reusable and to test with a different set of inputs.
In Junit5, we have the advantage of test methods consuming data arguments directly from the configured source. By default, JUnit 5 provides a few source annotations like:
@ValueSource: we can use this with an array of values of type Short, Byte, Int, Long, Float, Double, Char, and String
@EnumSource – passes Enum constants as parameters to the test method
@MethodSource – passes external methods generating streams
@CsvSource – uses CSV values as a source for the parameters
Similarly, we have other sources like @CsvFileSource if we need to read a CSV file from classpath and @ArgumentSource to specify a custom, reusable ArgumentsProvider.
Here is a Junit example for parameterized test
Explanation of the code:
· Create a class(preparam). In this example, we are going to input two numbers by using sum (int,int) method which will return the sum of given numbers
· Create a parameterized test class
· Create a constructor that stores the test data. It stores 3 variables
· Create a static method that generates and returns test data.
· Creating a two-dimensional array (providing input parameters for addition). Using asList method we convert the data into a List type. Since, the return type of method input is collection.
· Using @Parameters annotation to create a set of input data to run our test.
· The static method identified by @Parameters annotation returns a Collection where each entry in the Collection will be the input data for one iteration of the test.
Consider the element
Here each array element will be passed to the constructor, one at a time as the class is instantiated multiple times.
· Using @Before annotation to setup the resources (preparam.class here). The @Before annotation is used here to run before each test case. It contains precondition of the test.
· Using @Test annotation to create our test.
· Creating an assert statement to check whether our sum is equivalent to what we expected.
Parameterization in TestNG happens either through @Parameters annotation or through passing parameters in testing.xml file
Similar to TestNG Parameters, DataProviders are a means to pass data to test scripts in TestNG. Using DataProvider in TestNG, we can easily inject multiple values into the same test case. It comes inbuilt in TestNG and is popularly used in data-driven frameworks.
18. Parallel tests
This feature is only available with TestNG. Usually, test cases are kind of threads that are invoked one after the other. However, if you wish to save on the execution time, you can control this in TestNG by setting the tests to be run in parallel and providing the number of threads that need to run at once in testing.xml file. In the below snippet we can see the testng.xml.
Listeners listen to every event that occurs within tests. Listeners are supported both in JUnit as well as TestNG. So, if you wish to perform certain tasks or show a specific message in the log before the test starts, after the test finishes, when the test is skipped, when the test is passed or failed, we have these Listener functions which enable us to do this
JUnit uses the Listener class, and TestNG uses a Listener interface.
TestNG write a listener class defining the Listener interface methods, and the second step is to call this Listener class name using @Listeners annotation in the main class.
JUnit also inherits the methods from the Listener parent class following which a Listener runner class is defined to apply the listener functions on one or more main class.
In this blog, we have compared TestNG and JUnit frameworks. We learned the common features supported in both the frameworks as well as additional features supported only in TestNG. Apparently, there are a couple of extra features only in TestNG, like parallel execution and dependency tests. Most of the features supported by TestNG are also available in JUnit.
There are slight deviations in terms of syntax, terminologies in JUnit vs TestNG for the common features. Those who undermined the strength of JUnit over TestNG would have realized until now i.e., JUnit is also one of the powerful automation frameworks.
And that brings us to the end of my blog “Junit vs TestNG”. In this article, we had a quick look at various ways of writing tests with each of these two test frameworks.
Hope you found my blog interesting and helpful. Please do give a thumbs up if you like it and do not forget to post your valuable feedback.