Couchdb option is missing on latest webscraper version


The title says it all.

Couchdb option is missing on latest webscraper version. Where i can find it?

I am using version 0.5.2

I also cannot find the option to store to CouchDB.

Has this been removed?

I'm using version 0.5.3


Not sure about this. But, it seems they removed some features to make user use their webscraper cloud..

Can someone verify this?

Hi Web Scraper developers, please comment on whether you want to discourage the below temporary solution.

Here's a way to use CouchDB in newer versions of Web Scraper. However, beware that Web Scraper developers have deprecated this option. This means future versions might completely cut off CouchDB.

How to use CouchDB after Web Scraper 0.5.1 or later

Web Scraper deprecated (removed the option to use) CouchDB starting version 0.5.1.

But you can still use CouchDB in the new versions as a legacy, that is, if you had already configured CouchDB as local storage in installed version 0.4.2 or earlier.

So, if you had lost connection to CouchDB, perhaps because you had reinstalled your computer, OS, Chrome, or Web Scraper extension, here's a way to get around the problem, and use CouchDB in newly installed Web Scraper verson 0.5.1 or later.

There are 3 steps:

(1) Install the older Web Scraper 0.4.2,
(2) Configure it to use CouchDB, then
(3) immediately update to the latest version of Web Scraper.

The 3 steps in more details:

(1) Install the older Web Scraper 0.4.2

(a) Go to a Chrome extension archive site to download Web Scraper version 0.4.2.

(You need to use an archive site because Chrome Web Store only contains the latest version).

(b) Open Chrome browser's extension page

  • Make sure you have backed up any current Web Scraper sitemaps.
  • Removed current Web Scraper.
  • Close and reopen Chrome
  • Install by dragging the downloaded .crx file into the above Chrome browser extension page.

(2) Configure Web Scraper option to use CouchDB as local storage,

(Change "localhost" and "scraper-sitemaps" to suit your configuration in your CouchDB.)

(3) Immediately update to Web Scraper 0.5.3

Back in Web Scraper extension's "Details" page, click "Update". Close and reopen Chrome.

You should now have the latest versoin of Web Scraper, yet one tha is configured to use CouchDB for local storage.

More references

(But try either http://:@localhost:5984/ and http://localhost:5984/)

Our goal wasn't to cause problems for existing users, but to deprecate this feature for new users and to hope that existing users would switch to local storage at some point.

Here is what you can do:

  1. Right click on web scraper icon
  2. Choose "manage extensions"
  3. At the top right corner of the browser enable "developer mode"
  4. Under inspect views click on "background page"
  5. Open console tab
  6. Paste this{storageType: "couchdb"});
  7. Press Enter key
  8. Restart browser
  9. Go to options page and configure CouchDB connection

I'll remove this post in about a month since we still plan to deprecate CouchDB.

Thanks for this alternative.

What non-CouchDB local storages would you recommend using in the future? I shall learn about it for the future.

in Version:
Win10 64 bit;
Chrome: 101.0.4951.67 (Official Build) (64-bit)

i stuck with this step,
as attached the screenshot shows, it prompt "Package is invalid: 'CRX_HEADER_INVALID'." when i dragging the downloaded .crx file into the Chrome browser extension page.

can we fix it?