No data returned 8/4/2021

Web Scraper version: Downloaded and installed today, 8/4/2021. Can't find version info.
Chrome version: Don't know either. Updated recently.
OS: Windows 10

Sitemap:

{"_id":"paginationtutor","startUrl":["https://www.vice.com/en/topic/marijuana?page=[1-162]"],"selectors":[{"id":"links","type":"SelectorLink","parentSelectors":["_root"],"selector":"a.vice-card-hed__link","multiple":true,"delay":0},{"id":"title","type":"SelectorText","parentSelectors":["links"],"selector":"h1","multiple":false,"regex":"","delay":0}]}

Error Message:

None

Trying to scrape articles.

  1. New sitemap (base URL: MARIJUANA - VICE)
  2. Clicking on it, add new selector
  3. Clicking data preview, seems to work
  4. Clicking first article top left, adding selector (text, name: "title") I clicked on "Done selecting" and successfully added the selector, unluky screenshot.
  5. Checking data preview again
    image
  6. Starting test scrape
  7. Issue: No data returned.
    image
  8. Increasing request time by 10x
    image
  9. Issue remains. Only Scraping links from the first page (12) with original request time (2000 ms), nothing else. (I.e. removed second step of sraping the article title. The endgoal is to scrape each article's link, title, subtitle and text across the 180+ sites)
  10. Success.

Same success if only scraping article data such as the title with the article link as base URL when creating a sitemap. Can't figure the issue out. Tried to meticulously follow various tutorials. Carbon-copied their steps, got different results (no data returned).

Tutorial 1
Tutorial 2

TL, DR: No data returned when going any deeper than 1 layer. No error message. Scraping starts and ends as if functional but no data.

Edit: After tons of experimentation it turns out that it can't be done on the VICE page. It works elsewhere. I would be incredibly thankful if someone figures out how to do it on VICE though, will definitely run into this issue again. Now I'll have to make do with manual scraping.

@stuck Hi. It seems that there's a script that badly interacts with the scraper.

Download this plugin - Toggle JavaScript - Chrome Web Store

And disable the JavaScript.

Test results from the Web Scraper extension:

Test results from Web Scraper Cloud:

Hope it helps.

1 Like

Worked instantly & like a charm. Thank you, you absolute legend!

1 Like