Let's Learn TestNG Listeners in Selenium

Listeners are interfaces in TestNG framework. Listeners listen to each and every action performed by the test script during test execution. It is mainly used to take screenshot of a failed test, to retry the failed testcase, to log and to generate results.

List of Listeners:

Following are the few Listeners available in TestNG. Some of the Listeners are implemented at a Suite level, some at Method level. Each and every Listener has its own methods to perform a specific task.

· IAnnotationTransformer

· IAnnotationTransformer2

· IHookable

· IConfigurable

· IReporter

· IInvokedMethodListener

· IMethodInterceptor

· IExecutionListener

· IConfigurationListener

· ISuiteListener

· ITestListener

Test Scenario:

If 5 out of 100 test cases in our project fail, first we will check what the test cases failed and why they failed. We can handle this situation in 2 ways.

Solution 1: Manual Retry

Assume those 5 test cases failed due to a defect. Our plan is to retry only the failed test cases when the defect is fixed.

When a test case fails, TestNG automatically generates an XML file named testng-failed.xml under the test-output folder which has the details of the failed test cases. When the defects are fixed, we can just run only the failed test cases manually in testng-failed.xml rather than running the entire project.

A Testcase fails during execution

Listeners watch the testcase fails

When the defect is fixed, run only the

failed testcases in testng-failed.xml

Run is success

Solution 2: Automatic Retry

When the test cases fail due to timeout or internet speed, then TestNG automatically retries and the test case passes.

IRetryAnalyzer interface:

In this blog, we are going to see IRetryAnalyzer interface. It automatically retries the failed testcases. If the test execution status is failed, IRetryAnalyzer automatically retries the failed testcases. We can set the number of times for retry in our code.

Step 1: Go to Eclipse and Create a java class “RetryTest.java” which implements IRetryAnalyzer.

Step 2: Create a java class with @Test. I am purposely making this Testcase failed by giving wrong locators and look for the run status ‘Failed’.

Step 3: If we refresh our project, we can see testng-failed-xml under test-output folder.

Step 4: Connect the Testcase with IRetryAnalyzer by an attribute retryAnalyzer=RetryTests in @Test.

Step 5: If we run the testcase again, it will retry for 3 times automatically.

Adding the attribute “retryAnalyzer=<classname>” in each and every testcase is difficult if we have more number of test cases.

IAnnotationTransformer Interface:

IAnnotationTransformer is an interface which sets attribute values for invocation count during run time rather than code level.

Step 1: Create a java class with @Test. I am purposely making this Testcase failed by giving wrong locators and look for the run status ‘Failed’.

Step 2: Create another class RetryTests . Implement IAnnotationTransformer interface.

IAnnotationTransformer has a method transform() which has a parameter ITestAnnotation which acts as @Test.

Step 3: Create a testng.xml file for the testcase by right click on the project ->TestNG -> Convert to TestNG

I am changing the filename as testing-listener.xml.

Step 4: Add packagename along with class name in listeners tag in testng-listener.xml. We do not need to specify the retryAnalyzer attribute in the @Test annotation of our Testcase.

Step 5: Run testng-listener.xml file. We can see the testcase is retried and run for 3 times.

Note : In general, we don’t use IRetryAnalyzer, IAnnotationTransformer in Cucumber BDD. Because we have to use multiple runner files which needs to converted into many testng.xml files.

Conclusion:

Listeners are an important feature of TestNG framework which is used for many reasons. Retry the failed test case is one among them. Before confirming a testcase is failed, Listeners rerun the testcase to make sure that it is consistently failing.