A bot and a scraper together!

How to?
#1

Is there an option to input text in search field before scraping?
Let's say I want to scrape all the links from Google after typing Chicago in the search field. The website I'm trying to scrape don't have an option to put the search criteria in the URL (like google does), so I need to automate this also. furthermore I have thousands of cities to scrape.

Is there any way to input text in search field before scraping starts? Is there another software I can use with webscraper.io to do this?

#2

Hi

no it is not possible to use input text or use variable with this extension.

#3

do you know maybe it is possible with a combination of another software?

#4

have a look at Octoparse perhaps

#5

Octaparse would be the my reco as well. Maybe dexi.io too. Post your sitemap or web page and welll a look. I suspect it uses iframes?