I have a page I want to scrape that lazy load's images and their URLs. As a result, when I scrape I get all the data except only a few image URLs on hundreds of entries.
I tried this:
https://www.webscraper.io/documentation#element-scroll-down-selector
When I'm watching it work it goes from the top to the very bottom - the page is very long (180 element tiles) so it never scrolls over the page long enough to cause the lazy loading to fire.
I'm using the "Element attribute" to go through each title. I've tried adding the element scroll down to both the root portion of the graph and the element attribute as a sub item with my other element scrapers. Neither has a different impact.
The page I'm working on does NOT work like this:
https://www.webscraper.io/test-sites/e-commerce/scroll/computers/laptops
Where when you scroll to the bottom it loads more.
The page I'm working on is similar to this:
https://www.newegg.com/Internal-SSDs/SubCategory/ID-636?Tid=11693&PageSize=96
big and long BUT*** the image URLs and images only load if you scroll over the top of them. If you go directly to the bottom it never gives the page the command to load it.
Thoughts?