Having problem with pagination

[ahmedac]

Describe the problem. I am scrapping zomato.
The fields get scrapped properly but i am having problem with pagination. The first 5 pages get scrapped properly and then it goes in loop to the next 5 pages which doesn't get scrapped. Can anyone help with pagination.

Url: https://www.zomato.com/pune/restaurants

Sitemap:
{"_id":"zomatopune","startUrl":["https://www.zomato.com/pune/restaurants"],"selectors":[{"id":"name","type":"SelectorLink","parentSelectors":["_root","pages"],"selector":"a.result-title","multiple":true,"delay":0},{"id":"restname","type":"SelectorText","parentSelectors":["name"],"selector":".res-name a","multiple":false,"regex":"","delay":0},{"id":"locality","type":"SelectorText","parentSelectors":["name"],"selector":"a.left.fontsize3","multiple":false,"regex":"","delay":0},{"id":"address","type":"SelectorText","parentSelectors":["name"],"selector":".resinfo-icon > span","multiple":false,"regex":"","delay":0},{"id":"phone","type":"SelectorText","parentSelectors":["name"],"selector":".fontsize2 span.tel","multiple":false,"regex":"","delay":0},{"id":"cuisine","type":"SelectorText","parentSelectors":["name"],"selector":".res-info-cuisines a","multiple":true,"regex":"","delay":0},{"id":"services","type":"SelectorText","parentSelectors":["name"],"selector":".res-info-estabs a","multiple":false,"regex":"","delay":0},{"id":"time","type":"SelectorText","parentSelectors":["name"],"selector":".clearfix div.medium","multiple":false,"regex":"","delay":0},{"id":"pages","type":"SelectorLink","parentSelectors":["_root","pages"],"selector":"a.paginator_item:nth-of-type(n+2)","multiple":false,"delay":0},{"id":"more info","type":"SelectorText","parentSelectors":["name"],"selector":"div.res-info-feature-text","multiple":true,"regex":"","delay":0},{"id":"must have","type":"SelectorText","parentSelectors":["name"],"selector":"div.rv_highlights__section:nth-of-type(1) span","multiple":true,"regex":"","delay":0}]}

[leemeng]

Hi, your scraper is almost correct; you merely need to enable the Multiple option for it to work properly. Also, the paginator should be located above the links selector (i.e. before), or you will probably get duplicate results. The example below illustrates. I changed the test URL to one that produces fewer pages:

{"_id":"zomato_pune_v2","startUrl":["https://www.zomato.com/pune/restaurants/chinese?newly-opened=1&category=2"],"selectors":[{"id":"paginator","type":"SelectorLink","parentSelectors":["_root","paginator"],"selector":"a.paginator_item","multiple":true,"delay":0},{"id":"current_page","type":"SelectorText","parentSelectors":["_root","paginator"],"selector":".col-l-4.mtop div","multiple":true,"regex":"","delay":0}]}