Limit Infinite Scroll

[christofir]

I am trying to scrape this site using infinite scroll, but it never ends (and thus I can never get results). How do I modify sitemap to stop? (eg. after 1000 records)

Url: http://picpanzee.com/tag/seattlemom

Sitemap:
{"_id":"titus-try","startUrl":["http://picpanzee.com/tag/seattlemom"],"selectors":[{"id":"media-footer","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"div.grid-media-footer","multiple":true,"delay":"3000"},{"id":"ig-handle","type":"SelectorLink","parentSelectors":["media-footer"],"selector":"a.grid-media-owner","multiple":false,"delay":0}]}

I've read a few topics here, but still can't get it to work. I tried div.grid-media-footer: nth-of-type(1000), but it didn't work.

Thanks to anyone who can help!

[leemeng]

Sure, check out this sitemap which has a limited scroller. It stops after 42 records. If you want more, just increase the number in the selector: div.grid-media:nth-child(-n+42)

You were on the right track; you just needed the correct element and :nth selector. I'm using a negative child range, -n+42 which means item no. 42 and below. This website has more info on :nth selectors. Also, you may need to increase the scroller's delay (currently 2000) if you're getting a large number of pages.

{"_id":"picpanzee-limit-scroll","startUrl":["http://picpanzee.com/tag/seattlemom"],"selectors":[{"id":"Limited scroll","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"div.grid-media:nth-child(-n+42)","multiple":true,"delay":2000},{"id":"Name","type":"SelectorText","parentSelectors":["Limited scroll"],"selector":"a.grid-media-owner","multiple":false,"regex":"","delay":0},{"id":"Location","type":"SelectorText","parentSelectors":["Limited scroll"],"selector":"a.btn","multiple":false,"regex":"","delay":0},{"id":"When posted","type":"SelectorText","parentSelectors":["Limited scroll"],"selector":"li:nth-of-type(1)","multiple":false,"regex":"","delay":0},{"id":"Link","type":"SelectorLink","parentSelectors":["Limited scroll"],"selector":"a.grid-media-owner","multiple":false,"delay":0}]}