Hi there,
I have been using this tool for my scrapes and they've been working pretty well but I've come across this website where i want to get the data but because of how much entries there are to scape (over 21k), the scraper just terminates towards the end.
The website typically uses server side rendering so the URL is static (e.g. no forms of pagination like page number on the url)
Is it possible to make WebScraper first set the filters (i.e. by brand, i dont mind hardcoding it within the Web Scraper itself if possible) and then start scraping?
This is what i've got at the moment:
Sitemap:
{"_id":"Energy_Display","startUrl":["EPREL Public website when being used in SDR","parentSelectors":["click_pagination"],"type":"SelectorText","selector":".ng-star-inserted .ecl-u-align-items-lg-top span.ecl-u-type-bold","multiple":false,"regex":""},{"id":"power when being used in HDR efficiency mode","parentSelectors":["click_pagination"],"type":"SelectorText","selector":"[label='field.electronic-display.energyClassHDR'] .ecl-u-align-items-lg-top span.ecl-u-type-bold","multiple":false,"regex":""},{"id":"when being used in HDR power mode","parentSelectors":["click_pagination"],"type":"SelectorText","selector":"[label='field.electronic-display.powerOnModeHDR'] .ecl-u-align-items-lg-top span.ecl-u-type-bold","multiple":false,"regex":""}]}
