Use element scroll down selector?

#1

Can anyone upload a simple video of how to use element scroll down selector to get bottom lazy load information?

THANKS!

#2

There is already a very good tutorial on how to use Element Scroll Down (AKA scrollers AKA infinite scroll) from the WS team at:

The video title is "Web Scraper multiple record extraction tutorial" and it doesn't say infinite scroll.
The "infinite scroll" bit actually starts around 3:25, but you should really watch it all the way from the beginning 'cos it explains important concepts such as wrapper elements, element selection, child selectors, keyboard shortcuts etc.

BTW, this same video also covers "Load More" buttons at around 5:12.

#3

Unfortunately, I can't get it to work properly on my test site.

I crawl this page:
https://community.e-world-essen.com/organizations/allExhibitors

I selected "div.grid div.flex" as the marker for the scroll down element. It scrolls down a few times, but then stops very quickly. In the end, there are ~25 data records.

#4

Hi, the page is loading super slowly from my location, I added a longer delay to the scroll selector, probably the page load delay has to be about 9000 also:

{"_id":"e-world-essen-com","startUrl":["https://community.e-world-essen.com/organizations/allExhibitors"],"selectors":[{"delay":9000,"elementLimit":100,"id":"element-wrapper","multiple":true,"parentSelectors":["_root"],"selector":"div.w-full.bg-white","type":"SelectorElementScroll"},{"id":"title","multiple":false,"parentSelectors":["element-wrapper"],"regex":"","selector":"span.font-semibold","type":"SelectorText"}]}

You can increase the record limit by editing the scroll selector.

#5

Yes it is very slow also from my location. With this configuration i could get about 200 entries. Still not all.

{"_id":"eWorld2025","startUrl":["https://community.e-world-essen.com/organizations/allExhibitors"],"selectors":[{"id":"company","multiple":false,"parentSelectors":["deeplink"],"regex":"","selector":"span.break-word.font-semibold","type":"SelectorText"},{"id":"phone","multiple":false,"parentSelectors":["deeplink"],"regex":"","selector":"div.items-center:nth-of-type(2) a.text-gray-500","type":"SelectorText"},{"id":"email","multiple":false,"parentSelectors":["deeplink"],"regex":"","selector":"div.gap-2\\.5:nth-of-type(1) a","type":"SelectorText"},{"id":"address","multiple":false,"parentSelectors":["deeplink"],"regex":"","selector":".p-5 span.font-normal","type":"SelectorText"},{"delay":10000,"elementLimit":500,"id":"scroll","multiple":true,"parentSelectors":["_root"],"selector":"div.grid > .flex","type":"SelectorElementScroll"},{"id":"deeplink","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root"],"selector":"a.flex-col","type":"SelectorLink"}]}
#6

It worked fine for me, so it is probably a random slowness issue on the website itself.

1 Like