Help, whenever I run more than 4 windows of web scraper next morning the entire PC crashes and restart. How to fix this? I am extracting thousands of urls scrape setting 2000 delay, 500 delay start
Apparently your machine can't handle 4 scraper windows at the same time. Run them 1 by 1.
The number of the Chrome browser windows you can run Webscraper on would depend on the factors below. You can adapt for these over time.
(1) Platform: Linux/Ubuntu is more efficient than Microsoft Windows, and the former can run more browser windows,
(2) RAM memory. Try to use more than 8GB RAM, perhaps 16GB or more.
(3) Some other browser extensions might be eating up browser memory. Try to remove non-essential browser plugins/addons/extensions. (Also close non-essential browser tabs)
(4) The Website's graphic and script intensiveness. Some Websites have many images and Javascript running on the background, eating up a lot of memory. This you cannot choose. But you can use the next point to reduce graphic memory usage,
(5) Configure Chrome to not load the images. See this instruction:
The above no-image configuration may affect all your windows in the same browser, such that you cannot browse normally and conveniently. Perhaps use Firefox or Opera for daily use. Then dedicate Chrome for Webscraper.