Generating various Cucumber Reports

Introduction

In this blog we are going to explore various reports generation in Cucumber.These reports requires plugins to be added in testRunner file and creating properties file to display the reports in a different ways. Let us dig in.

When ever we do test execution, it is also require to understand the out put of the execution. Whether it is Manual execution or an Automated, the output of the same has to be in format, which immediately depicts the overall results of the execution. Hence, our framework also should have the same capability to create output or generate test execution reports.

It is essential to know, how better we can generate our Cucumber test reports. As we know that Cucumber is a BDD framework, it does not have a fancy reporting mechanism. In order to achieve this, Cucumber itself has provided a nice feature to generate reports. These are very basic reports, but using the output of these reports anybody can build more detailed reports in selenium.

Cucumber Reports

Now when we understand the importance of Cucumber Reports, let's learn to generate it as well. When we execute Cucumber Scenarios, it automatically generates an output in the eclipse console. There is a default behavior associated with that output and we can also configure that output as per our needs also. So how do we modify the default behavior, let's see this now.

• Pretty Report

The first plugin, we will talk about is Pretty. This provides more verbose output. To implement this, just specify plugin = "pretty" in CucumberOptions. This is what the code looks like:

@CucumberOptions( plugin = { "pretty" } )

Complete TestRunner will look this

package runner;
import org.junit.runner.RunWith;
import org.testng.annotations.DataProvider;
import io.cucumber.junit.Cucumber;
import io.cucumber.testng.CucumberOptions;    

@CucumberOptions(
     features = "src/test/resources/features/",
			glue =  { "StepDefinition", "hooks" },
			plugin = { "pretty"}, 
			monochrome = true			 
	)

	public class testRunner extends AbstractTestNGCucumberTests {
		@Override
		@DataProvider(parallel = false)
		public Object[][] scenarios() {
			return super.scenarios();
		}

Cucumber Reports Output

You must be wondering that all we have seen above is actually good for a test or for a couple of tests. But if we run a full test suite, this report is not much useful in that case. On top of that, it is difficult to keep these console output safe for future use.

Cucumber gives us the capability to generate reports as well in the form of HTML, XML, JSON & TXT. Cucumber frameworks generate very good and detailed reports, which can be shared with all stakeholders. There are multiple options available for reports which can be used depending on the requirement.

• Cucumber HTML Reports

For HTML reports, add html:target/cucumber-reports to the @CucumberOptions plugin option.

This will generate an HTML report at the location mentioned in the format itself.

HTML Report Plugin

@CucumberOptions(
features = "src/test/resources/features/",
			glue =  { "StepDefinition", "hooks" },
			plugin = { "pretty",
				
             "html:target/cucumber-reports/dsalgo.html"},
       		dryRun=false,
			monochrome = true			 
	)   

Report Output Location

HTML Report Output

• Cucumber JSON Report

For JSON reports, add json:target/cucumber-reports/Cucumber.json to the @CucumberOptions plugin option.

This report contains all the information from the gherkin source in the JSON format. This report is meant to be post processed into another visual format by third-party tools, such as Cucumber Jenkins. This report contains all the information from the gherkin source in the JSON format. This report is meant to be post processed into another visual format by third-party tools, such as Cucumber Jenkins.

JSON report Plugin

@CucumberOptions(
features = "src/test/resources/features/",
			glue =  { "StepDefinition", "hooks" },
			plugin = { "pretty",
				
             "json:target/cucumber-reports/Cucumber.json"},
       		dryRun=false,
			monochrome = true			 
	)   

Json report Output Location

JSON Report Output

• Cucumber JUNIT XML Report

For JUNIT reports, add junit:targe/cucumber-reports/Cucumber.xml to the @CucumberOptions plugin option. This report generates XML files just like Apache Ant's junit report task. This XML format is understood by most continuous integration servers, who will use it to generate visual reports.

XML Report Plugin

@CucumberOptions(
features = "src/test/resources/features/",
			glue =  { "StepDefinition", "hooks" },
			plugin = { "pretty",
				
             "junit:target/cucumber-reports/Cucumber.xml"},
       		dryRun=false,
			monochrome = true			 
	)  

XML Report Location

• Cucumber ExtendPDF Report

Cucumber ExtendPDF Report is report which has all test case outputs in PDF format contains in depth reports of the project.For that we need create a properties file and add these following format.

Step 1: Add Cucumber Extent Reporter library to Maven Project

This is really simple, as we have been using Maven Project, all we need to do is to add the dependencies in to the project POM file. Dependencies information can be taken from Maven Repository – Cucumber Extent Reporter.

Dependencies needed to create extent report

<dependency>
    		<groupId>com.aventstack</groupId>
    		<artifactId>extentreports</artifactId>
    		<version>5.0.9</version>
	   </dependency>

		<!-- https://mvnrepository.com/artifact/com.aventstack/extentreports -->
	<dependency>
    	<groupId>tech.grasshopper</groupId>
    	<artifactId>extentreports-cucumber7-adapter</artifactId>
    	<version>1.9.1</version>
	</dependency>

ExtendPDF Plugin

extent.reporter.spark.start=true
extent.reporter.spark.out=test-output/SparkReport/dsalgo.html

extent.reporter.pdf.start=true
extent.reporter.pdf.out=PDFReport/ExtendPDF.pdf

#Screenshot

screenshot.dir=/Screenshots/
screenshot.rel.path=../Screenshots/

basefolder.name=ExtentReports/SparkReport_
basefolder.datetimepattern=d_MMM_YY HH_mm_ss

systeminfo.os=windows
systeminfo.Engineer= Gs
systeminfo.Project= DSALGOProjectDemo

And the TestRunner File will be like this with Extentreports plugin

@CucumberOptions(
features = "src/test/resources/features/",
			glue =  { "StepDefinition", "hooks" },
			plugin = { "pretty",
					
"com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:"},
       		dryRun=false,
			monochrome = true			 
	) 

Report Output Location

It will be created and displayed in a folder each and every time the project has been executed.

Some of ExtendPDF Output

We can have all the reports together. TestRunner file will be

@CucumberOptions(
features = "src/test/resources/features/",
			glue =  { "StepDefinition", "hooks" },
			plugin = { "pretty",
				"junit:target/cucumber-reports/Cucumber.xml",
				"json:target/cucumber-reports/Cucumber.json",	
                 "html:target/cucumber-reports/dsalgo.html",
                 "com.aventstack.extentreports.cucumber.adapter.
                          ExtentCucumberAdapter:"},
 
       		dryRun=false,
			monochrome = true			 
	) 

Conclusion

Cucumber uses reporter plugins to produce reports that contain information about what scenarios have passed or failed. Some plugins are built-in, others have to be installed separately. You can also build your own. This page documents built-in formatter plugins, custom formatters and some common third-party plugins.

Cucumber Reports generate Cloud reports and also custom reports in given format which provides easy way to share our test reports with stakeholders.

In the search for a reporting tools for Cucumber I found a few tools that helped me a lot. Hope you find helpful from my blog. Happy Learning.