I should start off by saying I'm not hugely computer literate so any help would be greatly appreciated as I've been trying to make this work for days now.
I'm trying to scrape the data of each of the restaurants on this page.
https://www.tripadvisor.co.uk/Restaurants-g186338-London_England.html#EATERY_LIST_CONTENTS
So far I've managed to create something that scrapes the data I need, however it only does it for the first 60 restaurants (2 pages).
What do I need to do to extend this to the other 600+ pages of restaurants?
The urls go up increments such as:
My sitemap is as follows:
{"_id":"londonrestaurants","startUrl":["https://www.tripadvisor.co.uk/Restaurants-g186338-London_England.html"],"selectors":[{"id":"restaurantlink","type":"SelectorLink","parentSelectors":["_root","pagination"],"selector":"div.wQjYiB7z:nth-of-type(1) a","multiple":true,"delay":0},{"id":"restaurantname","type":"SelectorText","parentSelectors":["restaurantlink"],"selector":"h1._3a1XQ88S","multiple":false,"regex":"","delay":0},{"id":"address","type":"SelectorText","parentSelectors":["restaurantlink"],"selector":"._2vbD36Hr span._2saB_OSe","multiple":false,"regex":"","delay":0},{"id":"phone","type":"SelectorText","parentSelectors":["restaurantlink"],"selector":"._15QfMZ2L a","multiple":false,"regex":"","delay":0},{"id":"email","type":"SelectorLink","parentSelectors":["restaurantlink"],"selector":"div._36TL14Jn:nth-of-type(2) a","multiple":false,"delay":0},{"id":"website","type":"SelectorLink","parentSelectors":["restaurantlink"],"selector":"._3jdfbxG0 a._2wKz--mA","multiple":false,"delay":0},{"id":"pagination","type":"SelectorLink","parentSelectors":["_root"],"selector":"a.nav","multiple":false,"delay":0}]}
Thank you for your help.