Hi,
I am trying to scrape some date from one of my client sites,(a new site)
The below sitemap will scrape the first page but not move to the second, although the pagination selector is there and shows when you check the link movement.
There is no way to start the selction with 50 lots per page as it defaults to 12. However when I first built it and had 50 selected and then did the selection it did scrape all the lots.
Strange now its defaulted back to only 12.
I think the main issue is its only doing the one page and not moving to the next one, if it did this, it wouldn't matter how many lots it started with as it would cycle through them fine, so this seems the main issue.
Any help to fix is appreciated.
The other thing I need to do once this is working is to see how to extract all the images from the gallery for each lot, I have it set to grab the main image but need to work out how to grab all the ones below, maybe they need to be clicked then the main image changes and the next url grabbed.
Many thanks
Sitemap:
{"_id":"silverstonenewsite2021scrapeworking","startUrl":["https://www.silverstoneauctions.com/index.php?option=com_bidding&view=commission&layout=catalogue&id=76"],"selectors":[{"id":"paginnation","type":"SelectorLink","parentSelectors":["_root","paginnation"],"selector":"#pagination-ul a","multiple":true,"delay":0},{"id":"links","type":"SelectorLink","parentSelectors":["_root","paginnation"],"selector":"a.gold","multiple":true,"delay":0},{"id":"Title","type":"SelectorText","parentSelectors":["links"],"selector":"h2","multiple":false,"regex":"","delay":0},{"id":"Image","type":"SelectorImage","parentSelectors":["links"],"selector":".slides .slick-current img","multiple":false,"delay":0},{"id":"Strap","type":"SelectorText","parentSelectors":["links"],"selector":".lot-summary-container p:nth-of-type(1)","multiple":false,"regex":"","delay":0},{"id":"Bullets","type":"SelectorText","parentSelectors":["links"],"selector":".lot-description-container ul","multiple":false,"regex":"","delay":0},{"id":"Lot","type":"SelectorText","parentSelectors":["links"],"selector":"dt:contains('Lot Number') + dd","multiple":false,"regex":"","delay":0},{"id":"Registration No","type":"SelectorText","parentSelectors":["links"],"selector":"dt:contains('Registration Number') + dd","multiple":false,"regex":"","delay":0},{"id":"MOT","type":"SelectorText","parentSelectors":["links"],"selector":"dt:contains('MOT Expiry') + dd","multiple":false,"regex":"","delay":0},{"id":"Chassis Number","type":"SelectorText","parentSelectors":["links"],"selector":"dt:contains('Chassis Number') + dd","multiple":false,"regex":"","delay":0}]}