Element Scroll Down on Walmart.com.mx not working

How to?
#1

Describe the problem.

I am not able to scroll down the page using the Element Scroll Down. When I use it, the page simply does not scroll down.

This is the category of chocolates that I'm trying to scrape:
https://super.walmart.com.mx/despensa/dulces-y-chocolates/chocolates/_/N-ur2lhr?

This is my sitemap code:

{"id":"walmart_final_nuevo","startUrl":["https://super.walmart.com.mx/despensa/dulces-y-chocolates/chocolates//N-ur2lhr?"],"selectors":[{"id":"Element","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"div.product_container__1Z_GP","multiple":true,"delay":"2000"},{"id":"Link","type":"SelectorLink","parentSelectors":["Element"],"selector":".product_name__1669g a","multiple":false,"delay":0}]}

I'm only able to extract the link of the first products that are being shown :frowning:

Please someone help me D:

#2

I haven't figured out how to scroll these types of pages yet. It could be a limitatation in WS's scroller code. As a temporary measure, you could specify a long page load delay, say 15000 (15 sec), then you manually scroll the page yourself before the scraper starts.

A related tip: Try zooming out your browser view for walmart as much as possible; this will make the server display more pages and will also make scrolling faster. You would need to zoom out first before you start scraping.

2019-10-04_104913
2019-10-04_104913.jpg1164×441 150 KB

#3

Thank you very much leemeng. I guess I will use your tip as a temporary measure. I know that this Web Scraper tool is not perfect and you adapt to its limitations.

#4

There's an undocumented feature which allows the scroller work. Thanks to @martins for the tip. Try this:

{"_id":"walmart_final_nuevo_v2","startUrl":["https://super.walmart.com.mx/productos?Ntt=chocolates"],"selectors":[{"id":"Element","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"div.product_container__1Z_GP","multiple":true,"delay":"2500","scrollElementSelector":"#scrollToTopComponent"},{"id":"Link","type":"SelectorLink","parentSelectors":["Element"],"selector":".product_name__1669g a","multiple":false,"delay":0}]}

There's an extra line added to the scroller, "scrollElementSelector": "#scrollToTopComponent"

The undocumented feature is described here - Scroller does not work on certain websites