The site is a blog site with pages and an older link at the end of the page. I'm trying to get the content of all the pages.
The url and sitemap is below. Its traversing all pages but scraping only the first and last
Url: http://arnabchoudhuri.blogspot.com/
Sitemap:
{"_id":"test","startUrl":["http://arnabchoudhuri.blogspot.com/"],"selectors":[{"id":"main","type":"SelectorText","parentSelectors":["_root"],"selector":"div.main","multiple":false,"regex":"","delay":0},{"id":"older","type":"SelectorLink","parentSelectors":["_root","older"],"selector":"a.blog-pager-older-link","multiple":false,"delay":0},{"id":"old main","type":"SelectorText","parentSelectors":["older"],"selector":"div.main","multiple":false,"regex":"","delay":0}]}
All help is sincerely appreciated
Thanks
