Possible to run Webscraper instance in the same browser tab?

How to?
#1

Currently when we click "Scrape", Webscraper will open a new pop-up Chrome "window" to run the scraping instance. My guess is, this is a Chrome security requirement.

But is it possible to config Webscraper to run the instance in the SAME browser tab? (That is, above the Developer Tool panel)

The reason is that it might open up big possibilities of capturing Ajax HTTP responses with other tools.

Or is there a way for me to go into Chrome's Developer Mode to modify/hack the Webscraper config for my own usage? (I'm no expert but am willing to experiment.)

Sorry if this sounds "greedy" but it might open up good possibilities.

#2

You can scrape within current tab using Preview Data button on the topmost selector. The problem with this is that you cannot stop the scrape without either closing or refreshing tab.

You can also open webscraper within a tab instead of opening dev panel using extension URL:
chrome-extension://jnhgnonknehpejjnehehllkliplmbmhn/devtools_web_scraper_panel.html#/

1 Like
#3

Thanks! I'll try these.

#4

I cannot seem to make Method #2 work (chrome-extension). It still opens a new browser window, that seems to behave independently.

Method #1 (Preview Data on topmost selector) DOES seem to work. Even though it opens a new browser window, the new window is recognized as a related, child, window (by other Chrome extentions working on the page too).

However, Method #1 requires that at the root level, there's only one "topmost selector". So I couldn't have 2 selectors under root, like:

items | div.items
pagination | div.nextpage

I got around it by concatenating their CSS selectors INTO the "items" selector (using comma). That is:

items | div.nextpage, div.items

I placed the "pagination" selector first, assuming the sitemap will run backward here too.

This works when the pagination is a simpler link.

But is there another way, to create a new intermediate wrapper, that is the child of "root", but the "parent" of "items" and "pagination"?

I can already get my stuff to work for now. Just curious. Thanks!