Scraping the data from the list and detail page

Hello Community,

I'm trying to scrape the the name, price, "Hersteller", "GPU Modell" from every product on the list. Also it's important to open every product to scrape two things on the detail page. If site one is done with the 30 products it should iterate the pagination and scrape the same data on every page.
Is it possible to do this with webscraper.io?

Thank you in advance!

Url: https://www.mindfactory.de/Hardware/Grafikkarten+(VGA).html/view_status/9

Sitemap:
{"_id":"mindfactory_richtig","startUrl":["https://www.mindfactory.de/Hardware/Grafikkarten+(VGA).html/view_status/9"],"selectors":[{"id":"Aus","type":"SelectorElement","parentSelectors":["_root","Pagination"],"selector":"div#bProducts > table.table > tbody > tr > td","multiple":true,"delay":0},{"id":"Name","type":"SelectorText","parentSelectors":["Aus"],"selector":"p.pname","multiple":false,"regex":"","delay":0},{"id":"Pagination","type":"SelectorElementClick","parentSelectors":["_root"],"selector":".pagination>li:last-child>a","multiple":true,"delay":"2000","clickElementSelector":".pagination>li:last-child>a","clickType":"clickOnce","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"Hersteller","type":"SelectorText","parentSelectors":["Aus"],"selector":"td.hidden-xs:nth-of-type(1)","multiple":false,"regex":"","delay":0},{"id":"GPU_Modell","type":"SelectorText","parentSelectors":["Aus"],"selector":"td.hidden-xs:nth-of-type(2)","multiple":false,"regex":"","delay":0},{"id":"Edition","type":"SelectorText","parentSelectors":["Aus"],"selector":"td.hidden-xs.hidden-md","multiple":false,"regex":"","delay":0},{"id":"Preis","type":"SelectorText","parentSelectors":["Aus"],"selector":"div.pprice","multiple":false,"regex":"","delay":0},{"id":"Status","type":"SelectorText","parentSelectors":["Aus"],"selector":"a.shipping1","multiple":false,"regex":"","delay":0},{"id":"Artikelnummer","type":"SelectorText","parentSelectors":["Aus"],"selector":"div.col-xs-4 > p:nth-of-type(2)","multiple":false,"regex":"","delay":0}]}

Hi!

Of course it is possible.

You don't really need Element Click selector in order to click 'Next page' button constantly, it can be done with Link selector as well, but you have to set it and _root as it's parent.

It can be even done using just pagination array, like:

https://www.mindfactory.de/Hardware/Grafikkarten+(VGA).html/article_per_page/[1-5]
But it depends on a number of pages, and is not universal.

Your fixed sitemap:

{"_id":"mindfactory_richtig2","startUrl":["https://www.mindfactory.de/Hardware/Grafikkarten+(VGA).html/view_status/9"],"selectors":[{"id":"Aus","type":"SelectorElement","selector":"div#bProducts > table.table > tbody > tr > td","parentSelectors":["Pagination"],"multiple":true,"delay":0},{"id":"Name","type":"SelectorText","selector":"p.pname","parentSelectors":["Aus"],"multiple":false,"regex":"","delay":0},{"id":"Pagination","type":"SelectorLink","selector":"[aria-label='Nächste Seite']","parentSelectors":["_root","Pagination"],"multiple":true,"delay":"3000"},{"id":"Hersteller","type":"SelectorText","selector":"td.hidden-xs:nth-of-type(1)","parentSelectors":["Aus"],"multiple":false,"regex":"","delay":0},{"id":"GPU_Modell","type":"SelectorText","selector":"td.hidden-xs:nth-of-type(2)","parentSelectors":["Aus"],"multiple":false,"regex":"","delay":0},{"id":"Edition","type":"SelectorText","selector":"td.hidden-xs.hidden-md","parentSelectors":["Aus"],"multiple":false,"regex":"","delay":0},{"id":"Preis","type":"SelectorText","selector":"div.pprice","parentSelectors":["Aus"],"multiple":false,"regex":"","delay":0},{"id":"Status","type":"SelectorText","selector":"a.shipping1","parentSelectors":["Aus"],"multiple":false,"regex":"","delay":0},{"id":"Artikelnummer","type":"SelectorText","selector":"div.col-xs-4 > p:nth-of-type(2)","parentSelectors":["Aus"],"multiple":false,"regex":"","delay":0}]}
1 Like

Thank you so much! Wish you a good day.