Hello dear forum, I need a specialist. I set up the scraper but the site has 60k item in a page with scrolling. Scraper stop after 2 mins and get no data. I need just links, images and titles. That's all, what is the problem? Please help, thanks a lot! Why is no data coming?
Url: https://www.trendyol.com/kozmetik
UPDATE: I now have a better understanding of the scroller and am creating a better sitemap
Hi, this site can actually be navigated via page numbers, e.g.
https://www.trendyol.com/kozmetik?pi=1
https://www.trendyol.com/kozmetik?pi=2
etc
So you can change your Start URL to something like https://www.trendyol.com/kozmetik?pi=[1-3] which would only grab the first 3 pages. In the example below, I also set the scroller to scroll down until the bottom element on the left sidebar, "Kampanyalı Ürün".
DELETED
Thank you leemeng but I also tried that. While pi=1,pi=2 working, pi=300 is not working. Each page got 20 items and total items are 71000. So 3550 pages are legit.