_edited_edited.png)
Web scraping is a process to extract content and data from a website such as Amazon, Ebay, Youtube, Facebook, Trip advisor to name a few. It extracts underlying HTML code and the data is stored in the Database.
Web Scraping is very useful for Market research, Finding leads, comparing products, content analysis, price comparison, data collection for business intelligence, etc.
There are several tools available to extract the data information and one such tool is Octoparse.
Octoparse is a cloud-based web data extraction tool that helps users to extract relevant information from various types of websites. It enables users to scrape unstructured data and save it in different formats including Excel, plain text and HTML.
Few BDD Scenarios for Octoparse
Feature_file_01 :Login to Octoparse scrapping tool
Scenario _TC_01: Successful Sign up to Octoparse scrapping tool
Given : Launch octoparse tool
When : User clicks on Start a Free Trial and enters all the details and clicks on submit
Then : User should be logged in Successfully
Scenario _TC_02: Sign up to Octoparse scrapping tool using existing username
Given : Launch octoparse tool
When : User clicks on Start a Free Trial and enters all the details and clicks on submit
Then : Username already exists message should be displayed
Scenario _TC_03: Login to Octoparse scrapping tool
Given : User installed octoparse tool
When : User Enters Username and Password
Then : Username should be able to login to Octoparse tool
Scenario _TC_04: Forgot Password when Logging to Octoparse scrapping tool
Given : User installed octoparse tool
When : User Enters Username and forgot Password
Then : User should be redirected to forgot password in application web page
Scenario _TC_05: Login to Octoparse scrapping tool (Negative scenario)
Given : User installed octoparse tool
When : User Enters Username and wrong Password
Then : Username or password is wrong message pops out.
Scenario _TC_06: Display home page
Given : User Downloaded octoparse tool
When : User logs in to app
Then : User is navigated to home page
Scenario _TC_07: Display home page
Given : User Downloaded octoparse tool
When : User logs in to app
Then : User navigated to home page
Feature_file_02 : Task Templates
Scenario_TC_08 : Use built in Task Templates
Given: User is logged into Octoparse
When:User clicks on +Task button
then: User is navigated to Task Templates page and Several products are displayed
Scenario_TC_09 : Search templates under built in Task Templates
Given: User is on Task templates page
When:User enters text in search bar and clicks on search
then: Searched template is displayed Succesfully
Scenario_TC_10 : Search templates under built in Task Templates(Negative scenario)
Given: User is on Task templates page
When:User enters incorrect spelling in search bar and clicks on search.(Ex Amzon_
then: Search results are empty message is displayed
Scenario_TC_11 : Choose Category in Task Templates
Given: User is on Task templates page
When:User clicks on any category say Travel
then: Templates related to travel sites must be displayed
Scenario_TC_12 : Sort by Name in Task Templates
Given: User is on Task templates page and selects one category
When:User clicks on Sort by name
then: Templates should be displayed in sorted order
Scenario_TC_13 : Sort by Date Added in Task Templates
Given: User is on Task templates page and selects one category
When:User clicks on Sort by name
then: Templates should be displayed acc to the Date Added
Scenario_TC_14 : Choose a template in Task Templates
Given: User is on Task templates page
When: User selects a template say Ebay
then: All Ebay related templates has to be displayed
Scenario_TC_15 : Load Ebay template
Given: User is on Ebay templates page
When: User selects one ebay template
then: Selected Template should be loaded
Scenario_TC_16 : Use a Ebay template
Given: User is on Ebay templates page and selects one ebay template
When: User clicks on Use template
then: User should be able to enter URL list and Save and Run
and new tab opens up
when user selects local extraction
then data is extracted and displayed
Feature_file_03 : Extracting data from Booking.com
Scenario_TC_17 : Extracting results when applying filters
Given:User is on page "https://www.booking.com/searchresults.html?aid=304142&label=gen173nr-1FCAEoggI46AdIM1gEaKQCiAEBmAExuAEXyAEM2AEB6AEB-AECiAIBqAIDuAKQ8-eDBsACAdICJGUxZTgyZWZmLTM3NWEtNGU5Ni1hZjk2LTM0NjdhOGMxOTY0MdgCBeACAQ&sid=7d397bfb1cb69ad2b6097f7f59d0e899&tmpl=searchresults&age=3&age=7&class_interval=1&dest_id=16&dest_type=country&from_sf=1&group_adults=2&group_children=2&label_click=undef&no_rooms=1&raw_dest_type=country&req_age=12&req_age=12&room1=A%2CA%2C7%2C3&sb_price_type=total&shw_aparth=1&slp_r_match=0&src=searchresults&srpvid=e45d942833a6008a&ss=Bahamas&ssb=empty&ssne=Bahamas&ssne_untouched=Bahamas&top_ufis=1&clear_ht_id=1&rdf="
When: User applies filter for Family friendly properties under property type
Then: 82 Properties should be listed
Scenario_TC_18:Detecting Name
Given:User is on Page https://www.booking.com/hotel/bs/coralsands.html?aid=304142;label=gen173nr-1FCAEoggI46AdIM1gEaKQCiAEBmAExuAEXyAEM2AEB6AEB-AECiAIBqAIDuAKQ8-eDBsACAdICJGUxZTgyZWZmLTM3NWEtNGU5Ni1hZjk2LTM0NjdhOGMxOTY0MdgCBeACAQ;sid=7d397bfb1cb69ad2b6097f7f59d0e899;age=12;age=12;dest_id=16;dest_type=country;dist=0;group_adults=2;group_children=2;hapos=2;hpos=2;no_rooms=1;req_adults=2;req_age=12;req_age=12;req_children=2;room1=A%2CA%2C12%2C12;sb_price_type=total;sr_order=popularity;srepoch=1618606627;srpvid=b70593512179006d;type=total;ucfs=1&#hotelTmpl
When: Detecting Hotel name
Then: Show Coral Sands Hotel
Scenario_TC_19:Detecting rating
Given:User is on page https://www.booking.com/hotel/bs/coralsands.html?aid=304142;label=gen173nr-1FCAEoggI46AdIM1gEaKQCiAEBmAExuAEXyAEM2AEB6AEB-AECiAIBqAIDuAKQ8-eDBsACAdICJGUxZTgyZWZmLTM3NWEtNGU5Ni1hZjk2LTM0NjdhOGMxOTY0MdgCBeACAQ;sid=7d397bfb1cb69ad2b6097f7f59d0e899;age=12;age=12;dest_id=16;dest_type=country;dist=0;group_adults=2;group_children=2;hapos=2;hpos=2;no_rooms=1;req_adults=2;req_age=12;req_age=12;req_children=2;room1=A%2CA%2C12%2C12;sb_price_type=total;sr_order=popularity;srepoch=1618606627;srpvid=b70593512179006d;type=total;ucfs=1&#hotelTmpl
When: Detecting Hotel rating
Then: Show Rating as Awesome 9.0
Scenario_TC_20:Detecting Location
Given: User is on page https://www.booking.com/hotel/bs/coralsands.html?aid=304142;label=gen173nr-1FCAEoggI46AdIM1gEaKQCiAEBmAExuAEXyAEM2AEB6AEB-AECiAIBqAIDuAKQ8-eDBsACAdICJGUxZTgyZWZmLTM3NWEtNGU5Ni1hZjk2LTM0NjdhOGMxOTY0MdgCBeACAQ;sid=7d397bfb1cb69ad2b6097f7f59d0e899;age=12;age=12;dest_id=16;dest_type=country;dist=0;group_adults=2;group_children=2;hapos=2;hpos=2;no_rooms=1;req_adults=2;req_age=12;req_age=12;req_children=2;room1=A%2CA%2C12%2C12;sb_price_type=total;sr_order=popularity;srepoch=1618606627;srpvid=b70593512179006d;type=total;ucfs=1&#hotelTmpl
When: Detecting Hotel Location
Then: Show location as Chapel Street, Harbour Island, Bahamas
Scenario_TC_21:Detecting Price
Given: User is on page https://www.booking.com/hotel/bs/coralsands.html?aid=304142;label=gen173nr-1FCAEoggI46AdIM1gEaKQCiAEBmAExuAEXyAEM2AEB6AEB-AECiAIBqAIDuAKQ8-eDBsACAdICJGUxZTgyZWZmLTM3NWEtNGU5Ni1hZjk2LTM0NjdhOGMxOTY0MdgCBeACAQ;sid=7d397bfb1cb69ad2b6097f7f59d0e899;age=12;age=12;dest_id=16;dest_type=country;dist=0;group_adults=2;group_children=2;hapos=2;hpos=2;no_rooms=1;req_adults=2;req_age=12;req_age=12;req_children=2;room1=A%2CA%2C12%2C12;sb_price_type=total;sr_order=popularity;srepoch=1618606627;srpvid=b70593512179006d;type=total;ucfs=1&#hotelTmpl
When: Detecting price for the Selected dates
Then: Show the price
Feature_file_04:Extracting data from Amazon
Scenario_TC_22:Capturing Price from a page
Given: User is on Page "https://www.amazon.com/Miracase-Compatible-Designed-Protection-Shockproof/dp/B08GFHZC3Y/ref=sr_1_1_sspa?crid=2ABP1OAU2AV1R&dchild=1&keywords=iphone+12+case&qid=1618606217&sprefix=iphone%2Caps%2C211&sr=8-1-spons&psc=1&smid=AFWTUUXY7HYOE&spLa=ZW5jcnlwdGVkUXVhbGlmaWVyPUEzUlRQSjhYMzBUMFVNJmVuY3J5cHRlZElkPUEwMTU0NzY5MkhPM01UVlFKRkpQNSZlbmNyeXB0ZWRBZElkPUEwMjYwMTk4MkE5Q0QwOEVOUkNEUyZ3aWRnZXROYW1lPXNwX2F0ZiZhY3Rpb249Y2xpY2tSZWRpcmVjdCZkb05vdExvZ0NsaWNrPXRydWU=:
When: User detects price
Then : Price 12.99 for new and Fastest Delivery date Tuesday is displayed
Scenario_TC_23:Capturing color of item from a page
Given:User is on Page "https://www.amazon.com/Miracase-Compatible-Designed-Protection-Shockproof/dp/B08GFHZC3Y/ref=sr_1_1_sspa?crid=2ABP1OAU2AV1R&dchild=1&keywords=iphone+12+case&qid=1618606217&sprefix=iphone%2Caps%2C211&sr=8-1-spons&psc=1&smid=AFWTUUXY7HYOE&spLa=ZW5jcnlwdGVkUXVhbGlmaWVyPUEzUlRQSjhYMzBUMFVNJmVuY3J5cHRlZElkPUEwMTU0NzY5MkhPM01UVlFKRkpQNSZlbmNyeXB0ZWRBZElkPUEwMjYwMTk4MkE5Q0QwOEVOUkNEUyZ3aWRnZXROYW1lPXNwX2F0ZiZhY3Rpb249Y2xpY2tSZWRpcmVjdCZkb05vdExvZ0NsaWNrPXRydWU=:
When: User detects color
Then : Navy blue is showed
Scenario_TC_24:Capturing rating for an item from a page
Given:User is on Page "https://www.amazon.com/Miracase-Compatible-Designed-Protection-Shockproof/dp/B08GFHZC3Y/ref=sr_1_1_sspa?crid=2ABP1OAU2AV1R&dchild=1&keywords=iphone+12+case&qid=1618606217&sprefix=iphone%2Caps%2C211&sr=8-1-spons&psc=1&smid=AFWTUUXY7HYOE&spLa=ZW5jcnlwdGVkUXVhbGlmaWVyPUEzUlRQSjhYMzBUMFVNJmVuY3J5cHRlZElkPUEwMTU0NzY5MkhPM01UVlFKRkpQNSZlbmNyeXB0ZWRBZElkPUEwMjYwMTk4MkE5Q0QwOEVOUkNEUyZ3aWRnZXROYW1lPXNwX2F0ZiZhY3Rpb249Y2xpY2tSZWRpcmVjdCZkb05vdExvZ0NsaWNrPXRydWU=:
When: User detects Rating
Then : 4.2 stars with 9522 ratings is displayed
Scenario_TC_25:Capturing screensize
Given:User is on Page "https://www.amazon.com/Miracase-Compatible-Designed-Protection-Shockproof/dp/B08GFHZC3Y/ref=sr_1_1_sspa?crid=2ABP1OAU2AV1R&dchild=1&keywords=iphone+12+case&qid=1618606217&sprefix=iphone%2Caps%2C211&sr=8-1-spons&psc=1&smid=AFWTUUXY7HYOE&spLa=ZW5jcnlwdGVkUXVhbGlmaWVyPUEzUlRQSjhYMzBUMFVNJmVuY3J5cHRlZElkPUEwMTU0NzY5MkhPM01UVlFKRkpQNSZlbmNyeXB0ZWRBZElkPUEwMjYwMTk4MkE5Q0QwOEVOUkNEUyZ3aWRnZXROYW1lPXNwX2F0ZiZhY3Rpb249Y2xpY2tSZWRpcmVjdCZkb05vdExvZ0NsaWNrPXRydWU=:
When: User detects screen size
Then : 6.1 inches