Hey, I want to know if it's possible to schedule scraping to begin at a specific day and time (even as a recurring task, i.e. every Monday at 2am)
Also, is it possible to schedule several different scrapings to execute one after the other?
I want to scrape different sitemaps while I'm sleeping...
Thanks!
You can schedule scraping jobs on Web Scraper Cloud. You can find more information here: https://www.webscraper.io/documentation/scheduler . Sign up for free trial here: https://cloud.webscraper.io/register .
I need to trigger the webscraper locally because the web site I want to scrape is only accessible from my laptop. My client has implemented heavy security over a VPN. So I can't use a cloud scraper. I want to be able to execute my webscraper job once a day and export as CSV, automatically, into a folder of my choosing (better yet would be direct integration into google sheets, but I'll save that for another day).
How to accomplish this? Chrome macro recorders don't work because they only let you automate interactions within the web page, not the extensions within Chrome itself. Windows macro recorders aren't allowed as they require admin privileges to install.