Pagination limit with Link selector

How to?
#1

Hi there,

I have been giving my best but I am not able to limit my pagination to 15 when it is a Link selector and there are many maaaany pages. I have been trying doing it in "selector":"li.r a" with a :not. It should be easy but it is not working !

Any help? idea of how to do it?
Thanks in advance

Url: http://tenders.es/profesionales/

Sitemap:
{"_id":"tenders","startUrl":["http://tenders.es/profesionales/"],"selectors":[{"id":"profi","type":"SelectorLink","parentSelectors":["_root","next"],"selector":"h4 a","multiple":true,"delay":0},{"id":"verificado","type":"SelectorText","parentSelectors":["profi"],"selector":"span.verified","multiple":false,"regex":"","delay":0},{"id":"categorias","type":"SelectorText","parentSelectors":["profi"],"selector":"ul.specialties li:nth-of-type(n+2) a","multiple":false,"regex":"","delay":0},{"id":"emailselector","type":"SelectorElementClick","parentSelectors":["profi"],"selector":"div.truncate_more","multiple":false,"delay":"1000","clickElementSelector":"a.truncate_show","clickType":"clickOnce","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"web","type":"SelectorText","parentSelectors":["profi"],"selector":"a.domain","multiple":false,"regex":"","delay":0},{"id":"telfselector","type":"SelectorElementClick","parentSelectors":["profi"],"selector":"div.details div.d_movil_telf","multiple":false,"delay":"1000","clickElementSelector":"div.details div.d_movil_texto","clickType":"clickOnce","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"location","type":"SelectorText","parentSelectors":["profi"],"selector":"div.details li:nth-of-type(2)","multiple":false,"regex":"","delay":0},{"id":"email","type":"SelectorText","parentSelectors":["emailselector"],"selector":"parent","multiple":false,"regex":"","delay":0},{"id":"telf","type":"SelectorText","parentSelectors":["telfselector"],"selector":"parent","multiple":false,"regex":"","delay":0},{"id":"next","type":"SelectorLink","parentSelectors":["_root","next"],"selector":"li.r a","multiple":false,"delay":"2000"}]}

#2

You can try and edit your metadata and set your start URL as a range, like this:

http://tenders.es/profesionales/[1-15]

if you are after the first 15 pages, then you can get rid of the pagination selector and it should only scrape the information from there

#3

this is great, I didn't know web scrapper is capable of changing page numbers by itself.

But a quick question, Lets say a web result has total 100 pages result, If i just want from 70 - 100 page does it still work? and There is no need to do pagination rite?

thank you so much

#4

Yes, the ranges should work whatever your starting page should be, as long as you specify the range in the metadata :wink:

#5

Hi @webber,
It doesn't work. I have tried with several and different ranges and with and without / at the end after the ] but the scraper always starts at the end of the range. For example if I do [2-4] starts at 4, 5, 6, 7, 8...
Did you try it with my example? as it doesn't work for me.

#6

Forget what I said, it works now. Thanks!
I didn't know that I am supposed to remove the pagination.