Newbie - Pagination issues

How to?
#1

I have this scraping with another product but it limits me to 200 pages and i have to keep adjusting to continue and there are about 10,000 entries.

I have testing WS with two elements, however the next page never load as it seems to get reset. is there any method of correcting this? any assistance is greatly appreciated.

"https://www.floridabar.org/about/section/section-rp-mbrs/?pageSize=50&pageNumber=1

Sitemap

{"_id":"florida-bar-test","startUrl":["https://www.floridabar.org/about/section/section-rp-mbrs/?pageSize=50&pageNumber=1"],"selectors":[{"id":"NextPage","type":"SelectorLink","parentSelectors":["_root","NextPage"],"selector":"div.search-results-controls:nth-of-type(3) a[title='next page']","multiple":true,"delay":0},{"id":"AttorneyLink","type":"SelectorLink","parentSelectors":["_root","NextPage"],"selector":".profile-name a","multiple":true,"delay":0},{"id":"BarNo","type":"SelectorText","parentSelectors":["AttorneyLink"],"selector":"div.row:nth-of-type(1) p","multiple":false,"regex":"","delay":0},{"id":"Address","type":"SelectorText","parentSelectors":["AttorneyLink"],"selector":"div.row:nth-of-type(2) p:nth-of-type(1)","multiple":false,"regex":"","delay":0},{"id":"OfficeNo","type":"SelectorText","parentSelectors":["AttorneyLink"],"selector":"p:nth-of-type(2) a","multiple":false,"regex":"","delay":0},{"id":"CellNo","type":"SelectorText","parentSelectors":["AttorneyLink"],"selector":"p:nth-of-type(3) a","multiple":false,"regex":"","delay":0},{"id":"Email","type":"SelectorText","parentSelectors":["AttorneyLink"],"selector":"div.row:nth-of-type(3) div.col-sm-9","multiple":false,"regex":"","delay":0},{"id":"County","type":"SelectorText","parentSelectors":["AttorneyLink"],"selector":"div:nth-of-type(6) p","multiple":false,"regex":"","delay":0},{"id":"Circut","type":"SelectorText","parentSelectors":["AttorneyLink"],"selector":"div:nth-of-type(7) .col-xs-12 p","multiple":false,"regex":"","delay":0},{"id":"Admitted","type":"SelectorText","parentSelectors":["AttorneyLink"],"selector":"div:nth-of-type(8) p","multiple":false,"regex":"","delay":0},{"id":"LawSchool","type":"SelectorText","parentSelectors":["AttorneyLink"],"selector":"div:nth-of-type(10) div.col-sm-9","multiple":false,"regex":"","delay":0},{"id":"Practice","type":"SelectorText","parentSelectors":["AttorneyLink"],"selector":"div:nth-of-type(11) div.col-sm-9","multiple":false,"regex":"","delay":0},{"id":"Firm","type":"SelectorText","parentSelectors":["AttorneyLink"],"selector":"div:nth-of-type(12) p","multiple":false,"regex":"","delay":0}]}

#2

The Url does change for each new page for this site, so you can use the Specify multiple urls with ranges method.

#3

Already tried the syntax below with the same result.

https://www.floridabar.org/about/section/section-rp-mbrs/?pageSize=50&pageNumber=[1-271]

I also tried to simplify the pagination by adding multiple pages

https://www.floridabar.org/about/section/section-rp-mbrs/?pageSize=50&pageNumber=1
https://www.floridabar.org/about/section/section-rp-mbrs/?pageSize=50&pageNumber=2

and also get the same result- the page gets cleared.

also, leemeng. thank you for the assistance.

#4

I tired the Multiple urls ranges at the first level and that works, however when drilling down to the details pages the site returns a search page and WS fails to go to the next pagination.

I scarpe the site for all ids and though i could replace the site map with my own version'
basicaly, i wanted to replace the line that details what pages to search manually

with something like this..

["https://www.floridabar.org/directories/find-mbr/?barNum=370680","https://www.floridabar.org/directories/find-mbr/?barNum=91805","https://www.floridabar.org/directories/find-mbr/?barNum=964743","https://www.floridabar.org/directories/find-mbr/?barNum=49761","https://www.floridabar.org/directories/find-mbr/?barNum=150225","https://www.floridabar.org/directories/find-mbr/?barNum=109785","https://www.floridabar.org/directories/find-mbr/?barNum=118362","https://www.floridabar.org/directories/find-mbr/?barNum=66811","https://www.floridabar.org/directories/find-mbr/?barNum=650226","https://www.floridabar.org/directories/find-mbr/?barNum=113103"]

this will remove the pagination and scrape each page individually. I thought i coudl just place 1000 lines at a time. Is there any method of accomplishing this?