Is it possible to scrape from a xml sitemap like this: https://www.draeger.com/da_dk/Sitemap.xml
I want it to access each url and fetch data on each url.
The plugin is just giving me following error:
FAILED_TO_CONNECT_TO_CHROME_TAB {"message":"Could not establish connection. Receiving end does not exist."}
So i suspect that they may have some form of bot/scrape protection maybe.
Okay, if i have it as xlsx then how do i scrape then?
after you can parse separate pages
Thanks! However im not sure i fully understand how my sitemap in webscraper plugin should be formed to handle separate pages like a list from a excel sheet?
What exact data do you need from product pages ? Screenshots please...
Well pretty much every bit of information possible. It is for a database.
I am fully aware how to select elements and scrape normally via webscraper chrome addon. But as said in beginning it seems blocked or unable.
{"_id":"draeger-com","startUrl":["https://www.draeger.com/da_dk/Substances/958"],"selectors":[{"id":"name","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"h1 .substancename","type":"SelectorText"},{"id":"formula","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"span.formula","type":"SelectorText"},{"id":"cas","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"tr:contains('CAS-No.:') td","type":"SelectorText"},{"id":"un_num","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"tr:contains('UN-No.:') td","type":"SelectorText"},{"id":"ec_num","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"tr:contains('EC-No.:') td","type":"SelectorText"},{"extractAttribute":"","id":"hazard","parentSelectors":["_root"],"selector":"dw-hazard-sign figcaption","type":"SelectorGroup"},{"extractAttribute":"","id":"known_as","parentSelectors":["_root"],"selector":"ul.limit-values","type":"SelectorGroup"},{"id":"filters","multiple":true,"parentSelectors":["_root"],"selector":"section[data-id=\"filters\"] .product_cards_wrapper","type":"SelectorElement"},{"id":"filter_name","multiple":false,"parentSelectors":["filters"],"regex":"","selector":"h4","type":"SelectorText"},{"id":"filter_link","multiple":false,"parentSelectors":["filters"],"selector":"a.button","type":"SelectorLink"},{"id":"masks","multiple":true,"parentSelectors":["_root"],"selector":"section[data-id=\"masks\"] .product_cards_wrapper","type":"SelectorElement"},{"id":"mask_name","multiple":false,"parentSelectors":["masks"],"regex":"","selector":"h4","type":"SelectorText"},{"id":"masks_link","multiple":false,"parentSelectors":["masks"],"selector":"a.button","type":"SelectorLink"},{"id":"resp","multiple":true,"parentSelectors":["_root"],"selector":"section[data-id=\"poweredairpurifyingrespirator\"] .product_cards_wrapper","type":"SelectorElement"},{"id":"resp_name","multiple":false,"parentSelectors":["resp"],"regex":"","selector":"h4","type":"SelectorText"},{"id":"resp_link","multiple":false,"parentSelectors":["resp"],"selector":"a.button","type":"SelectorLink"}]}
something like this.... but it's abnormal to scrape a web link, containing many other sub-products.... Each item has it's own URL and should be parsed separately...
Thanks, tested importing that sitemap and running it.
But how do i make it run through every url of the sitemap?
.......... download this draeger sitemap generator paste your URLS in yellow cells... copy a sitemap generated on the right (in blue) and paste it into Webscraper....
Oh okay, that way. Thanks! Didnt know we could input urls directly like that. Learned something new.
Thanks alot!
You can scrape directly from here (scroll down a bit). Page expires in 1 month from this posting.