Works in "data preview", but not when scraping

This site contains multiple list-items that contain hidden anchor tags, which I'm trying to click to advance to the next page. If I click "data preview" I see the list of links to be clicked, but when I run the scrape, it hangs for a while on the starting page, then closes without opening any of the following pages, and with no data scraped.

I've also tried using "element click" on the LIs (as these have a javascript onclick event) but that isn't working either.

Any ideas for what I should try next?

URL: https://www.[redacted].com/drivers/

Sitemap:
{"_id":"cdata","startUrl":["https://www.[redacted].com/drivers/"],"selectors":[{"id":"sources","type":"SelectorLink","parentSelectors":["_root"],"selector":".provider > div > .text > a","multiple":true,"delay":"2000"},{"id":"text","type":"SelectorText","parentSelectors":["sources"],"selector":"h1","multiple":false,"regex":"","delay":0}]}

1 Like

I get the same issue as the OP. I've used Web Scraper a lot in the past (thanks!) but recently - no idea when exactly, don't use it daily - it ONLY works using either of the Preview buttons. It shows all the desired elements and/or it shows all the correct scraped info. With and without "Multiple" checked. It happens on every site, even the provided test sites (which are great).

It opens the specified main page, then it just sits there. if I wait long enough, it closes "scrape complete!". but nothing is gathered.

Version: 0.4.1, tried clearing browser data, re-installing the addon
Browser: Vivaldi 2.5.1525.48 (tried running as admin too)
OS: Windows 10 Home (always up-to-date)

EDIT: it works fine on firefox.

1 Like