Effective Broken Link detection using Selenium WebDriver

Image Credit: Lego

Broken links, also called dead links, are those links on a webpage that are not used properly in a webpage that leads to a missing or a non-operating page. These links are often shown in webpage like "404 Not Found," "502 Bad Gateway," "503 Service Unavailable" and more. Most of the time, this error will come under 4xx and 5xx HTTP status codes.

Finding the broken links and making them work is an essential thing to maintain a good user experience. Because the users should not be shown an error page while navigating through the links on a website. It will be a bad user experience.

HTTP Response Codes

When any browser sends a request to a server by clicking a link, the server processes it and responds to the request. While responding, the response will contain an HTTP status code to the receiving end. It will say about the requested result."200 OK" is the most common response code, which says the request was successful and the page will be loaded with no errors. There are more codes that say different. We will discuss that below:

• 2xx Success: The request has been successful.

• 3xx Redirection: It Indicates the further action needs to be taken to complete the request. ex: Redirect to other URL.

• 4xx Client Error: The error indicates the problem due to the request made by the client. May be Dead link or a missing page.

• 5xx Server Error: The error indicates the problem due to the server while processing the request.

Let’s explore the ways to Find Broken Links

Its a time-consuming process to check each link manually in a webpage. But by using automation scripts in Selenium, it is possible to handle this effectively. Let's see how:

• Import all the packages :

 In the below example we have imported all the packages.

• Fetch All Links Using <a> Tag: 

Using the findElements() method in Selenium WebDriver, fetch all <a> tags on the webpage. This method is used to get all the hyperlinks available in the page. We have identified all links in the webpage and iterating them in a loop.

• Request HTTP for All Links: 

Using Selenium WebDriver, stimulate an action to click all the links. Thus, this results in a request being made to the corresponding server against all the links. Here we store the href of anchor tag in url variable. We also check if url is null or empty.

• Fetch the HTTP Response Code for All Links:

Using Selenium WebDriver, fetch the HTTP response code against all the links. The response code is indicative of the result of the HTTP request.

•  Check Whether the Response Code Shows a Valid or Broken Link: 

Now, let's analyze the HTTP response code using Selenium WebDriver against all the links to see if the link is valid or broken. For example, a 200 status code usually means the link is a valid link, while 404 or 5xx status means the link is broken.

• Loop through All the Captured Links: 

We will run a loop through the list of all captured links in Step 1 and repeat Steps 2-4 for each of the links.

Conclusion:

By automating the link validation process using Selenium WebDriver, not only will it reduce human effort, but also a full-proof testing of all possible links can be assured. A broken link will be identified quickly, making the website more reliable and providing users with a good experience.

Happy Learning!