Hi guys,
I have a few working scrape sitemaps on Webscraper io and have a good understanding, but with this one, i am hitting a wall. I am trying to scrape the price and part number (bike make and model also desired but not necessarily needed) but the problem is, the pages have different option/dropdown layouts. Some have strangely coded dropdown boxes which i am struggling to program into the scraper but i don't know if you can program the scraper to work with multiple different variations on product pages.
I am making this post to ask if this job is possible?
Here is an example link of the dropdown boxes (some stages have more options than others, depending on the part)
Here is another example link of a different variation which will be simple to program:
My sitemap has been coded initially to work with the top link i have added, but i cannot seem to work out how to finish it.
Sitemap:
{"_id":"Wunderkind","startUrl":["ABM ergonomic style | Motorcycle parts manufacturer | Shop link","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root"],"selector":"a.categoryLevel2","type":"SelectorLink"},{"id":"subcategory-link","linkType":"linkFromHref","multiple":true,"parentSelectors":["Category link"],"selector":"a.categoryLevel3","type":"SelectorLink"},{"id":"product-link","linkType":"linkFromHref","multiple":true,"parentSelectors":["subcategory-link"],"selector":"a.buttonFrontend","type":"SelectorLink"},{"clickActionType":"real","clickElementSelector":"a.custom-combobox-toggle","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":250,"discardInitialElements":"do-not-discard","id":"product-dropdown-stage1-arrowbox","multiple":false,"parentSelectors":["product-link"],"selector":"body","type":"SelectorElementClick"},{"clickActionType":"real","clickElementSelector":"li.ui-menu-item:not(:contains("Please select"))","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":250,"discardInitialElements":"discard-when-click-element-exists","id":"product-dropdown-stage1-data","multiple":true,"parentSelectors":["product-link"],"selector":"body","type":"SelectorElementClick"},{"clickActionType":"real","clickElementSelector":"#bpStage2 a.custom-combobox-toggle","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":250,"discardInitialElements":"do-not-discard","id":"product-dropdown-stage2-arrowbox","multiple":false,"parentSelectors":["product-link"],"selector":"body","type":"SelectorElementClick"},{"clickActionType":"real","clickElementSelector":"li.ui-state-focus:not(:contains("Please select"))","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":1000,"discardInitialElements":"discard-when-click-element-exists","id":"product-dropdown-stage2-data","multiple":true,"parentSelectors":["product-link"],"selector":"body","type":"SelectorElementClick"}]}
Any help would be massively appreciated!