In this blog, we are going learn how to run a postman collection and generate reports from Command Line Interface (CLI) and Jenkins using Newman.
Postman is a standalone software testing API (Application Programming Interface) platform to build, test, design, modify, and document APIs. It is a simple Graphic User Interface for sending and viewing HTTP requests and responses.
You can add new collections, update existing collections, update environments, and add and run monitors directly through the API. This enables you to programmatically access data stored in your Postman account.
A command-line interface is a means of interacting with a computer program (or software) by typing line by line commands in your shell (command prompt or terminal).
Newman is a command-line Collection Runner for Postman. It enables you to run and test a Postman Collection directly from the command line. It's built with extensibility in mind so that you can integrate it with your continuous integration (CI) servers and build systems.
Let us see the steps to achieve Newman Reporting for Postman
1. Install Node
2. Install Newman
3. Set System Variables
4. Run Newman
5. Install and generate Html Extra Report
6. Postman Newman integration with Jenkins
7. Generate HTML Extra report in Jenkins.
Step 1: Install Node
· Download the Installer. Download the Windows Installer from NodeJs official website.
· Install Node.js and NPM. After choosing the path, double-click to install .msi binary files to initiate the installation process.
· Check Node.js and NPM Version. Ensure you are using Node.js v4 or above. Your node.js is not installed if you do not get node.js version
Step 2: Install Newman
Install Newman from npm globally on your system, which allows you to run it from anywhere:
Step 3: Set System Variables: Add the Node executable to your system path. Go to the Control Panel > System and Security > System > Advanced System Settings > Environment variables
Step 4: Run Newman
We have completed the reporters installation, but we still we have to execute and translate to our HTML reports.
We can run newman collection in CLI by two methods:
1) Using the collection url from postman
2) Using the collection url from local desktop
Method 1: Using API Key from the postman collection
Select the collection for which you wish to generate the Newman report and click share button
Select Via API option and copy the Collection Access Key
Run the following Newman command in CLI with the appropriate parameters:
Go to the newman path and enter
“newman run accesskey”
Method 2: Using json file location from local desktop
Export the collection from Postman by just selecting Export in the collection folder and then saving them as a *json file. We can run the collection in CLI by mentioning the location of the collection, instead of API access key as below:
“newman run C:\***.json”
Step 5: Install and generate htmlextra report
HTML Extra is an awesome Postman/Newman reporting dashboard which enables us to display postman test results in a more interactive manner.
Install Newman HTML reporter using the command:
“npm -g newman-reporter-htmlextra”
Generate HTML Extra report for the exported collection using the command:
“newman run xxx.json -r htmlextra”
The HTMLExtra report will be saved inside the folder where Newman was executed, inside a folder called Newman, as below
Awesome! The Detailed Newman HTML extra report is generated !
In the summary page, we get a summary of the test results we have run. We get detailed information like number of iterations, assertions, skipped tests, failed tests. From the Total Requests tab, we can get lot of information like baseurl, the request body, response header, and response format. For more details, we can to go to the Failed tests tab to check which scripts failed, status code and assertion error message.
Step 6: Postman Newman integration with Jenkins
Jenkins is an open-source automation server that can act as a continuous integration (CI) server or a continuous delivery (CD) hub. With hundreds of available plug-ins, teams can use Jenkins to build, deploy, and automate almost any project.
Now, we will see the steps to generate the Newman Report in Jenkins.
To Configure Jenkins Job to Run Batch Command:
Go to localhost:8080-Dashboard-New item-Enter item Name-Select Freestyle Project-Click Ok
Install supported plugins like NodeJS Plugin and Test Complete Plugin from manage Jenkins- plugin manager.
To Configure Postman Collection on Jenkins:
In the configuration screen, Select General-Build Steps-Add Build Step-Execute Windows Batch Command and enter the below command and save.
“C:/**/newman run --disable-unicode C:\\**\\AllPrograms_postman_collection.json”
Note:- If you use back slash use one (/) , forward slash use two (\\) inside windows batch command.
We have to run postman collection on Jenkins by selecting ‘Build Now’
Now, we will Generate Newman Reports on Jenkins.
Let us configure and publish Jenkins HTML Reports for Newman.
The Html report for the same collection can be generated by changing the Windows batch command as below:
“C:/**/newman run C:\\Users**\\AllPrograms_postman_collection.json -r htmlextra”
Select “Build Now” for the postman collection in Jenkins and view the console output
The Html Extra report can be accessed from the location mentioned in the console output. It is highlighted in above screen shot, for reference.
So, we have reached the end of the blog. I hope you would have got a good understanding about generating Newman reports, HTML Extra reports, Running Postman Collection from CLI and executing the same in Jenkins.