Element Scroll Down can't trigger product load

#1

Hello everyone,
I'm trying to scrap Nike site for sneakers and stumble upon a problem with a infinite scroll. After scraping first 59 sneakers, Element Scroll Down go all the way down of the page and that can't trigger product load. It seems product load only triggered if browser view is on top of product grid. Is there a way to define how far Element Scroll Down go down? I hope you understand what I mean.

Here is the page I'm trying to scrap:
https://store.nike.com/us/en_us/pw/mens-shoes/7puZoi3

Here is my simple crawler:
{"_id":"nike_men","startUrl":["https://store.nike.com/us/en_us/pw/mens-shoes/7puZoi3"],"selectors":[{"id":"products","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"div.exp-product-wall > div.grid-item.fullSize div.grid-item-content","multiple":true,"delay":"1000"},{"id":"name","type":"SelectorText","parentSelectors":["products"],"selector":"p.product-display-name","multiple":false,"regex":"","delay":0},{"id":"price","type":"SelectorText","parentSelectors":["products"],"selector":"span.local","multiple":false,"regex":"","delay":0}]}

Thank you,
Rafis

#2

I have restructured your crawler and changed some selectors so it should work now. I also limited the scroll until item 99. You can change that in the scroller selector, div.product-card:nth-of-type(-n+99)

Usually I would not recommend following an infinite scroll right until the end 'cos some sites can have tens of thousands, or even millions of items. If you really do want to scroll to the end, you can change the selector value to something like (n+6) , which means keep scrolling past item 6. Note that the minus sign is gone. For my test scrape, I used Page load delay (ms): 6000

{"_id":"z_nike_men","startUrl":["https://store.nike.com/us/en_us/pw/mens-shoes/7puZoi3"],"selectors":[{"id":"product_scroller","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"div.product-card:nth-of-type(-n+99)","multiple":true,"delay":"1000"},{"id":"name","type":"SelectorText","parentSelectors":["item_selector"],"selector":"div.product-card__title","multiple":false,"regex":"","delay":0},{"id":"price","type":"SelectorText","parentSelectors":["item_selector"],"selector":"div.product-card__price","multiple":false,"regex":"","delay":0},{"id":"item_selector","type":"SelectorElement","parentSelectors":["_root"],"selector":".product-card__info","multiple":true,"delay":0}]}