Load More Stops after one scroll

#1

I am trying to scrape

https://www.smythstoys.com/uk/en-gb/toys/value-and-clearance/clearance/c/SM06010901

and have tried as per the tutorial video to process load more but it does not scroll through all items.

{"_id":"Smyths_Cleareance","startUrl":["Pardon Our Interruption > a","multiple":true,"linkType":"linkFromHref"}]}

i want to be able to open each item on the clearance page and then take data from each individual toy but started just to test if i can scroll through all items

Can you advise as to waht I am doing wrong

#2

Hi,

The 'Load more' button appears to have a glitch since it never stops loading more items. I recommend calculating the nr of pages (931 / 60 = 16) and adding the page parameter to the start URL:

{"_id":"smythstoys-com","startUrl":["https://www.smythstoys.com/uk/en-gb/toys/value-and-clearance/clearance/c/SM06010901?page=16"],"selectors":[{"id":"product-link","linkType":"linkFromHref","multiple":false,"parentSelectors":["wrapper"],"selector":".item-panel > a","type":"SelectorLink"},{"id":"title","multiple":false,"parentSelectors":["wrapper"],"regex":"","selector":"span.name","type":"SelectorText"},{"id":"SKU","multiple":false,"parentSelectors":["wrapper"],"regex":"","selector":"div.hash_code","type":"SelectorText"},{"id":"price","multiple":false,"parentSelectors":["wrapper"],"regex":"","selector":"div.price_di","type":"SelectorText"},{"id":"wrapper","multiple":true,"parentSelectors":["_root"],"selector":"div.product-padding","type":"SelectorElement"}]}