I have problem in pagination in Infinite Scroll

How to?
#1

Describe the problem.
I am trying to scrape data from trendyol.com. This website does not have page number as it has infinite scroll option. So, when I try it does not work for me. Help me out if anyone can do. Thanks in advance.
Url: http://trendyol.com

Sitemap:
{id:"sitemap code"}

{"_id":"trendyol","startUrl":["https://www.trendyol.com/erkek-t-shirt-x-g2-c73"],"selectors":[{"id":"Links","type":"SelectorLink","parentSelectors":["_root"],"selector":".p-card-chldrn-cntnr a","multiple":true,"delay":0},{"id":"Title","type":"SelectorText","parentSelectors":["Links"],"selector":".pr-new-br span font font","multiple":false,"regex":"","delay":0},{"id":"Price","type":"SelectorText","parentSelectors":["Links"],"selector":".prc-slg font font","multiple":false,"regex":"","delay":0},{"id":"Additional Info","type":"SelectorText","parentSelectors":["Links"],"selector":"div.pr-in-sz-w","multiple":false,"regex":"","delay":0},{"id":"Product Information","type":"SelectorText","parentSelectors":["Links"],"selector":"div.pr-in-dt","multiple":false,"regex":"","delay":0},{"id":"Image 1","type":"SelectorImage","parentSelectors":["Links"],"selector":".slick-current .selected img","multiple":false,"delay":0},{"id":"Image 2","type":"SelectorImage","parentSelectors":["Links"],"selector":".vertical .selected img","multiple":false,"delay":0},{"id":"Image 3","type":"SelectorImage","parentSelectors":["Links"],"selector":".vertical .selected img","multiple":false,"delay":0},{"id":"Image 4","type":"SelectorImage","parentSelectors":["Links"],"selector":".vertical .selected img","multiple":false,"delay":0},{"id":"Image 5","type":"SelectorImage","parentSelectors":["Links"],"selector":".vertical .selected img","multiple":false,"delay":0},{"id":"Express Delivery","type":"SelectorText","parentSelectors":["Links"],"selector":".dd-txt-vl font font","multiple":false,"regex":"","delay":0},{"id":"Seller","type":"SelectorText","parentSelectors":["Links"],"selector":".sl-nm font font","multiple":false,"regex":"","delay":0},{"id":"Discount(If Available)","type":"SelectorText","parentSelectors":["Links"],"selector":"div.stamp:nth-of-type(2)","multiple":false,"regex":"","delay":0},{"id":"Number of Reviews","type":"SelectorText","parentSelectors":["Links"],"selector":".rvw-cnt-tx font font","multiple":false,"regex":"","delay":0},{"id":"Favorite ","type":"SelectorText","parentSelectors":["Links"],"selector":".fv-dt font font","multiple":false,"regex":"","delay":0},{"id":"Seller Rating","type":"SelectorText","parentSelectors":["Links"],"selector":"a .sl-pn font font","multiple":false,"regex":"","delay":0},{"id":"Cargo","type":"SelectorText","parentSelectors":["Links"],"selector":"div.crg","multiple":false,"regex":"","delay":0}]}

#2

Hello @raja

Scroll selector works but due to how the website is structured it runs into a loop so basically it never ends.
If you keep scrolling you can see that "pi" value changes when you scroll till the very end which is pi=100 you will see that link suddenly transforms and runs into a new loop.

1526×714 156 KB

Transforms
Transforms1338×252 22.9 KB

I suggest making multiple category starting URLs with page range definition which is [1-100]
Here's my sitemap version:

{"_id":"trendyol","startUrl":["https://www.trendyol.com/sr?cid=558846%2C558948&pi=[1-100]"],"selectors":[{"id":"Element-Card","type":"SelectorElement","parentSelectors":["Links"],"selector":"body:has(div[data-fragment-name=\"ProductDetail\"])","multiple":true,"delay":0},{"id":"Links","type":"SelectorLink","parentSelectors":["scroll"],"selector":"a","multiple":true,"delay":0},{"id":"Title","type":"SelectorText","parentSelectors":["Element-Card"],"selector":"h1","multiple":false,"regex":"","delay":0},{"id":"Additional Info","type":"SelectorText","parentSelectors":["Element-Card"],"selector":"div.pr-in-sz-w","multiple":false,"regex":"","delay":0},{"id":"Product Information","type":"SelectorText","parentSelectors":["Element-Card"],"selector":"div.pr-in-dt","multiple":false,"regex":"","delay":0},{"id":"Seller","type":"SelectorText","parentSelectors":["Element-Card"],"selector":"h1 a","multiple":false,"regex":"","delay":0},{"id":"Number of Reviews","type":"SelectorText","parentSelectors":["Element-Card"],"selector":"div.tooltip-wrp + a","multiple":false,"regex":"","delay":0},{"id":"Stars","type":"SelectorText","parentSelectors":["Element-Card"],"selector":"span.score","multiple":false,"regex":"","delay":0},{"id":"Images","type":"SelectorGroup","parentSelectors":["Element-Card"],"selector":"div.slick-track .pd-img img","delay":0,"extractAttribute":"src"},{"id":"Seller Rating","type":"SelectorText","parentSelectors":["Element-Card"],"selector":"div.sl-pn","multiple":false,"regex":"","delay":0},{"id":"scroll","type":"SelectorElement","parentSelectors":["_root"],"selector":"div.p-card-wrppr","multiple":true,"delay":0}]}

Hope it helps.

#3

Thanks a lot for help. Let me check and I will get back to you.

#5

I have the same problem using scrolling on this page. Mercator Spletna Trgovina | Več Kot 13.000 Izdelkov

I can only scrape 400 articles if I use this address. If I change it to Mercator Spletna Trgovina | Več Kot 13.000 Izdelkov I cannot scrape the articles.