Hey all, so I used webscraper for a little bit now and this forum is really helpful. However, I have an issue that I cannot seem to solve either by myself or with the help of this forum. I am scraping this museum website to collect all of them. For pagination I use Element Click selector and it seems to run fine for a while, but it always, doesn't matter the changes stops at 612 results collected, but I know there is around 1450 results. I cannot identify the problem. Tried messing with element click delay time from 1000ms to 12000ms, all result to same amount of links.
Url: Listings – Museums.co.uk
Sitemap:
{"_id":"MuseumsUK","startUrl":["https://www.museums.co.uk/listings-without-map/"],"selectors":[{"clickActionType":"real","clickElementSelector":".gt3_btn_text","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickMore","delay":6000,"discardInitialElements":"discard-when-click-element-exists","id":"Pagination","multiple":true,"parentSelectors":["_root","Pagination"],"selector":"a.card__link","type":"SelectorElementClick"},{"id":"Link","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root"],"selector":"a.card__link","type":"SelectorLink"}]}
