Scroll the page down to the end

How to?
#1

The page is not scrolling down to the end.

{"_id":"roots","startUrl":["https://www.augustasportswear.com/"],"selectors":[{"id":"category","type":"SelectorLink","parentSelectors":["_root"],"selector":"h2 a","multiple":true,"delay":0},{"id":"scroll dwn","type":"SelectorElementScroll","parentSelectors":["category"],"selector":"div#content","multiple":true,"delay":"3000"},{"id":"product","type":"SelectorLink","parentSelectors":["scroll dwn"],"selector":"div.image a","multiple":true,"delay":0},{"id":"sku","type":"SelectorText","parentSelectors":["product"],"selector":"div.col5 span.sku","multiple":false,"regex":"","delay":0},{"id":"category name","type":"SelectorText","parentSelectors":["category"],"selector":"h1","multiple":false,"regex":"","delay":0}]}

#2

You need to select the individual items for it to work, so that the scraper knows what item it should be looking for when scrolling. This should work:

{"_id":"roots","startUrl":["https://www.augustasportswear.com/"],"selectors":[{"id":"category","type":"SelectorLink","parentSelectors":["_root"],"selector":"h2 a","multiple":true,"delay":0},{"id":"scroll dwn","type":"SelectorElementScroll","parentSelectors":["category"],"selector":".product","multiple":true,"delay":"3000"},{"id":"product","type":"SelectorLink","parentSelectors":["scroll dwn"],"selector":"div.image a","multiple":false,"delay":0},{"id":"sku","type":"SelectorText","parentSelectors":["product"],"selector":"div.col5 span.sku","multiple":false,"regex":"","delay":0},{"id":"category name","type":"SelectorText","parentSelectors":["category"],"selector":"h1","multiple":false,"regex":"","delay":0}]}

1 Like
#3

Wow, thanks. It was a real headache.

#4

By the way, is "selector": ".product " the children from id "product" or it's an attribute of html tags?

Reference: {"id":"scroll dwn","type":"SelectorElementScroll","parentSelectors":["category"],:grinning:"selector":".product":grinning:,"multiple":true,"delay":"3000"}

#5

.product is a class attribute of a div tag