Limiting Infinite Scroll - Advise Please

How to?
#1

I'm having trouble trying to figure out how to limit scrolling functionality to the first 500 elements, rather than scrolling for infinity. I've searched the forums and tried to work it out, but admit defeat. Any help greatly appreciated!

Url: Facebook Groups

Sitemap:
{"_id":"facebook-groupcommentscount","startUrl":["Facebook Groups .rq0escxv .hybvsw6c > div > div > div","multiple":true,"delay":2000},{"id":"user","type":"SelectorText","parentSelectors":["element"],"selector":".oajrlxb2.oo9gr5id span","multiple":false,"regex":"","delay":0},{"id":"commentcount","type":"SelectorText","parentSelectors":["element"],"selector":".oajrlxb2 span.iv3no6db.knj5qynh.m9osqain","multiple":false,"regex":"","delay":0},{"id":"reactioncount","type":"SelectorText","parentSelectors":["element"],"selector":".gpro0wi8 span.pcp91wgn","multiple":false,"regex":"","delay":0},{"id":"link","type":"SelectorLink","parentSelectors":["element"],"selector":"a.b1v8xokw","multiple":false,"delay":0},{"id":"caption","type":"SelectorText","parentSelectors":["element"],"selector":"div.ecm0bbzt.hv4rvrfc, .kr520xx4 div.j7igg4fr, div.qt6c0cv9.hv4rvrfc","multiple":false,"regex":"","delay":0}]}

#2

I thought I may be onto something based on leemeng's answer at Extracting first n elements on page with infinite data but I've only gotten myself more confused. I admit defeat!

The I currently have is below.

{"_id":"scroll-test","startUrl":["Facebook Groups [dir] div.ecm0bbzt","multiple":false,"delay":"6000"},{"id":"postlink","type":"SelectorLink","parentSelectors":["element"],"selector":"a.b1v8xokw","multiple":false,"delay":0},{"id":"element","type":"SelectorElement","parentSelectors":["_root"],"selector":".lzcic4wl .rq0escxv .hybvsw6c > div > div > div","multiple":true,"delay":0}]}

How to limit element click to a specific number of times?
#3

I have similar problem, an ecommerce with categories that have 10.000 to 20.000 elements, so I think if try I get out of memory, sure.

#4

You can use :nth selectors to limit scrolling. But Facebook itself is a tricky site and you'll also need some knowledge of HTML and CSS to find proper selectors. This example below scrapes postings from a public group. The scroller is limited to around 55 postings. You can change that by changing the number in :nth-of-type(-n+55)

{"_id":"facebook-group-limit-scroller","startUrl":["https://www.facebook.com/groups/725928420906568/"],"selectors":[{"id":"Separate scroller","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"div > div[role='feed'] > div:nth-of-type(-n+55)","multiple":true,"delay":"2500"},{"id":"Posting wrappers","type":"SelectorElement","parentSelectors":["_root"],"selector":"div > div[role='feed'] > div","multiple":true },{"id":"Posting","type":"SelectorText","parentSelectors":["Posting wrappers"],"selector":"div[dir='auto']","multiple":false,"regex":"" },{"id":"Name","type":"SelectorText","parentSelectors":["Posting wrappers"],"selector":"h2[dir='auto'] > strong,div > a[role='link'] > strong","multiple":false,"regex":"" }]}

Ref:
http://nthmaster.com/
https://www.w3schools.com/cssref/css_selectors.asp

2 Likes
#5

Thank you. Exactly what I was looking for. Great support community here.

#6

Hello @leemeng ,

Is it possible to use this technique to limit the infinite scroll when looking for places?

I tried adding -n + 55 but it doesn't seem to work ...

Thank you !

My sitemap : {"_id":"facebook","startUrl":["https://www.facebook.com/search/places?q=club&filters=eyJlbmFibGVfcGxhY2VfbG9jYXRpb25faWRzOjAiOiJ7XCJuYW1lXCI6XCJwbGFjZV9sb2NhdGlvblwiLFwiYXJnc1wiOlwiMTE0ODgxNTQ1MTkxNzQzXCJ9In0%3D"],"selectors":[{"id":"Scrolldown","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":".o8rfisnq div.qzhwtbm6:nth-of-type(1)","multiple":true,"delay":2000},{"id":"linkto","type":"SelectorLink","parentSelectors":["_root"],"selector":"a.qu0x051f.oo9gr5id","multiple":true,"delay":0}]}