Multiple Link Scrape

How to?
#1

Hi Guys,

I was wondering if there is a way for either of the following:

  1. I would like to scrape a site which works via a link and the country eg. http://www.google.com/England, http://www.google.com/Germany etc. Is there a way that I can scrape the multiple pages (it doesn't have links to follow, only a drop down list to select the country)?

  2. Following on from the above, I took the JSON script and changed it for each of the countries (all working), is it possible to import the whole list? and also is it possible to launch multiple scrapes at once without individually clicking them?

Thanks in advance

#2

You can add multiple start urls. Go to "edit metadata" view. There is a button for adding more urls.

3 Likes
Scraping different url (with similar pages) using same selector
#3

How about 45,000 links?

#4

Hi

export you sitemap
In the Json code, insert you 45 000 url (or less to test)
Then import the Json code in a new sitemap