DevTools Disappeared, Pages Still Scraping, How to Retrieve CSV?

illiniguy · December 9, 2023, 9:46pm

I've been running a scraping session for the past 7 days using a scraper I've used for years. When I checked Chrome today, the DevTools window was closed, and it showed a message that DevTools would need to refresh. However, the Chrome window showed the scraper was still visiting new pages.

With DevTools closed, and the button to download the CSV file gone, how can I retrieved the scraped CSV file? Is it saved locally on my machine that I can retrieve? I'm using a Mac, but I'll take the instructions for Windows if anyone knows.

Thanks.

leemeng · December 9, 2023, 11:40pm

Yes, scraped data is stored on your drive. You can export it later just by accessing the Web Scraper tab again. This will work even after a reboot or shutdown. Load your sitemap, click on its name, and click Export data. If you had scraped a lot of data, it can take some time to export.