1 Stemap with 500+ urls anyway to parallel scraping?

[tkkkkkkk]

I have multiple sitemaps that are mostly the same except for the product name and dates in the URL.
I tried to compile all of the sitemaps; the URLs have come up to 500+ URLs.
loading each site containing delays and intervals with additional actions, it just becomes unnecessarily long...I'm just wondering if there's any way to parallel scrape even from a single sitemap.
Is this possible??

[leemeng]

Are you asking how to add a list of URLs as StartUrls for one sitemap? If so, that has been covered here:

Scraping from list of urls

Is there anyway to start scraping from a list of urls instead of a json sitemap? Alternatively is there a free tool that converts a list of plain urls into a json sitemap working with web scraper? Thanks for any help. world33

There are a few ways to do it. For one-off jobs, my faviorite method is to just paste the list into pastelink.net then build or modify a sitemap to start from that pastelink page.

[tkkkkkkk]

No, I'm asking if the extension start like 5 instances of chrome so that it scrapes 5 urls at same time to speed it up.

[leemeng]

You could run additional instances in a VM (I have used Virtualbox with Lubuntu before). or run the Firefox version at the same time. If you need more than that, like dozens or even hundreds of connections per second, you will probably need to use Python. Also see

Multiple layer sitemaps (scraping)

You could launch firefox and chrome at the same time using half of the time Ot buy multiple raspberry pi 3 or 4 and launch eachone with a chrome browser.. Plus chromium browser is compatible with the chrome extensions and maybe firefoz too has some other fork that can use the same addon too. In some os you can use also chrome and chrome beta and chrome dev as 3 separate/different browsers.