Describe the problem.
I am trying scrape product details from a webshop from a grocery store. There are 797 pages. However I don't know how I should set up the start url with this pagination.
Url: https://www.colruyt.be/nl/producten
The second page I want to crawl is https://www.colruyt.be/nl/producten?page=2
the last page is https://www.colruyt.be/nl/producten?page=797
Can someone please tell me how I should make up the start url so I can scrape all the pages?
Sitemap:
{"_id":"colruyt","startUrl":["https://www.colruyt.be/nl/producten"],"selectors":[{"id":"Assortiment","type":"SelectorElement","parentSelectors":["_root"],"selector":"a.card","multiple":true,"delay":0},{"id":"Image","type":"SelectorImage","parentSelectors":["Assortiment"],"selector":".card__image img","multiple":false,"delay":0},{"id":"Product","type":"SelectorText","parentSelectors":["Assortiment"],"selector":"p.card__text","multiple":false,"regex":"","delay":0},{"id":"Hoeveelheid","type":"SelectorText","parentSelectors":["Assortiment"],"selector":"p.card__quantity","multiple":false,"regex":"","delay":0}]}