Version 1.29.60 Popup Link does not pop up

chrome · May 11, 2023, 6:34am

Even not working on your test site. Please check. BTW, the selector can be dragged, that a good update.

{"_id":"test","startUrl":["https://www.webscraper.io/test-sites/e-commerce/allinone-popup-links"],"selectors":[{"id":"products","multiple":true,"parentSelectors":["_root"],"selector":"div.thumbnail","type":"SelectorElement"},{"id":"popup","multiple":false,"parentSelectors":["products"],"selector":"img","type":"SelectorPopupLink"},{"id":"title","multiple":false,"parentSelectors":["popup"],"regex":"","selector":"h4:nth-of-type(2)","type":"SelectorText"}]}

ViestursWS · May 12, 2023, 9:35am

@chrome Hello, could you, please, provide a recording that displays this issue?

After performing tests, with the sitemap you provided everything appears to be working as expected, however, please, note that the 'Pop-up' link selector itself will be removed and merged with the 'Link' selector as one of it's type.

chrome · May 15, 2023, 12:24am

There is no Link type field here.

ViestursWS · May 15, 2023, 1:53pm

@chrome Hi, currently, this option is not yet implemented on the official version of the extension.

Alternatively, you can use the older version - 0.6.5 version which is still available for Mozilla Firefox: Web Scraper – Get this Extension for 🦊 Firefox (en-US)

ronaldleonds · May 15, 2023, 4:43pm

Hola! I like it in greetings, after the last update of the extension version in chrome: (1.29.60) , no one of my sitemaps works, I used the PopupLink option to click on some fields, now I change it to the auto type pagination option , but it goes to a point at the ID: SALE on which it does not perform the function, you should go to the sales page, stay there and continue scraping with the new information, but it appears to click and you return to the previous purchase page . Podrian help me with this please. Adiconal wanted to know how active the experimental options of the extension.

Thanks for your collaboration.

{"_id":"binance_zelle","startUrl":["https://p2p.binance.com/es/trade/buy/USDT"],"selectors":[{"id":"MONEDA_VES_COMPRA","parentSelectors":["_root","MONEDA_VES_COMPRA"],"paginationType":"clickOnce","selector":"li#VES","type":"SelectorPagination"},{"id":"BANCO_VES_COMPRA","parentSelectors":["_root","BANCO_VES_COMPRA"],"paginationType":"clickOnce","selector":"li#Banesco","type":"SelectorPagination"},{"id":"TODOS_VES_COMPRA","parentSelectors":["_root","TODOS_VES_COMPRA"],"paginationType":"clickOnce","selector":".css-1any1hu div.css-6hm6tl","type":"SelectorPagination"},{"id":"DATA_VES_COMPRA","parentSelectors":["_root"],"type":"SelectorElementClick","clickElementSelector":"button[aria-label='Page number 2']","clickElementUniquenessType":"uniqueText","clickType":"clickOnce","delay":1500,"discardInitialElements":"do-not-discard","multiple":true,"selector":"div.css-ovjtyv"},{"id":"MONTO_VES_COMPRA","parentSelectors":["DATA_VES_COMPRA"],"type":"SelectorText","selector":"div.css-1m1f8hn","multiple":false,"regex":""},{"id":"LIMITE_VES_COMPRA","parentSelectors":["DATA_VES_COMPRA"],"type":"SelectorText","selector":".css-16w8hmr div.css-vurnku","multiple":false,"regex":""},{"id":"VENTA","parentSelectors":["_root","VENTA"],"paginationType":"auto","selector":"div.css-dvbf59","type":"SelectorPagination"},{"id":"MONEDA_USD_VENTA","parentSelectors":["_root","MONEDA_USD_VENTA"],"paginationType":"clickOnce","selector":"li#USD","type":"SelectorPagination"},{"id":"BANCO_USD_VENTA","parentSelectors":["_root","BANCO_USD_VENTA"],"paginationType":"clickOnce","selector":"li#Zelle","type":"SelectorPagination"},{"id":"TODOS_USD_VENTA","parentSelectors":["_root","TODOS_USD_VENTA"],"paginationType":"clickOnce","selector":"div.css-10nf7hq:nth-of-type(1) .css-1any1hu div.css-6hm6tl","type":"SelectorPagination"},{"id":"DATA_USD_VENTA","parentSelectors":["_root"],"type":"SelectorElementClick","clickElementSelector":"button[aria-label='Page number 2']","clickElementUniquenessType":"uniqueText","clickType":"clickOnce","delay":1500,"discardInitialElements":"do-not-discard","multiple":true,"selector":"div.css-ovjtyv"},{"id":"MONTO_USD_VENTA","parentSelectors":["DATA_USD_VENTA"],"type":"SelectorText","selector":"div.css-1m1f8hn","multiple":false,"regex":""},{"id":"LIMITE_USD_VENTA","parentSelectors":["DATA_USD_VENTA"],"type":"SelectorText","selector":".css-16w8hmr div.css-vurnku","multiple":false,"regex":""}]}

chrome · May 16, 2023, 12:22am

Thank you for the answer. And how to enable the Experimental Features of the extension?

martins · May 16, 2023, 12:53pm

Hi!

We are really sorry about the situation. Previous extension version was using chrome API manifest V2 which allowed script execution in websites. The popup link selector needed script execution to replace window.open() function with a placeholder that would get the link. Unfortunately Chrome manifest V3 doesn't allow script execution. Because of that there is no way to extract popup links from a website right now. There is also no going back since all MV2 extensions will stop working this summer.

Here is what we will do:

we will remove the ability to create new popup link selectors
we will add a feature to link selector that will allow link extraction from other attributes or text. We hope that this feature will cover most of the cases.

wilhelm · May 17, 2023, 8:37am

Hello. is there still no update for chrome yet?
still not working with using type Popup link till this day.

chrome version 113.0.5672.92
webscraper.io extension version 1.29.60

but working on firefox browser tested on

firefox version 113.0.1 (64-bit)
webscraper.io extension version 0.6.5

please we need the recent news. Thanks.

martins · May 19, 2023, 7:37am

There won't be a fix for popup link selector. Chrome browser with manifest version 3 has removed the ability to execute scripts which is needed for popup link selector.

wilhelm · May 20, 2023, 4:38am

is there any way to scrap web that open a new tab window then scrap it inside? without using popup link selector?. or we just need to wait for next update as @ViestursWS say?
this is what i use example:

{"_id":"sscatalog2","startUrl":["https://submit.shutterstock.com/catalog_manager/videos?sort=newest"],"selectors":[{"id":"next","parentSelectors":["_root"],"type":"SelectorElementClick","clickElementSelector":"a:nth-of-type(n+27) li","clickElementUniquenessType":"uniqueText","clickType":"clickOnce","delay":5000,"discardInitialElements":"discard","multiple":true,"selector":"div.component-content:nth-of-type(3)"},{"id":"edit","parentSelectors":["next"],"type":"SelectorPopupLink","selector":"div.cm-edit-button","multiple":true},{"id":"nama","parentSelectors":["edit"],"type":"SelectorText","selector":".o_List_List_itemText div","multiple":false,"regex":""}]}

chrome · June 6, 2023, 2:52am

What if the link is not from the element, it's bundled dynamically with js click events?
Then how to get the popup link?

nivvle · June 14, 2023, 11:54am

Hi Martin,

Is there any update on this? And can you explain the new link selector feature? I'm trying to extract the link from a popup that works in the preview but not while scraping:

{"_id":"bol","startUrl":["https://www.bol.com/nl/nl/p/petrol-industries-heren-voordeel-3-pack-t-shirt-roze-blauw-groen-maat-l/9300000149195449/?bltgh=klrvtxYQDH8WkJR3n9hCPQ.4_25.26.ProductImage"],"selectors":[{"id":"seller_name","parentSelectors":["_root"],"type":"SelectorHTML","selector":".product-seller--l span","multiple":false,"regex":""},{"id":"seller_url","parentSelectors":["_root"],"type":"SelectorElementClick","clickElementSelector":".is_loading span","clickElementUniquenessType":"uniqueText","clickType":"clickOnce","delay":4000,"discardInitialElements":"do-not-discard","multiple":false,"selector":".product-seller--l a"},{"id":"seller_rating","parentSelectors":["seller_url"],"type":"SelectorText","selector":"div.seller-rating--xl","multiple":false,"regex":""},{"id":"seller_more","parentSelectors":["seller_url"],"type":"SelectorLink","selector":"a.ui-btn--outline--secondary","multiple":false},{"id":"seller_offers","parentSelectors":["seller_more"],"type":"SelectorHTML","selector":".large--is-visible a.nav-list__link","multiple":false,"regex":""},{"id":"seller_kvk_name","parentSelectors":["seller_more"],"type":"SelectorHTML","selector":"dt:contains('Handelsnaam') + dd","multiple":false,"regex":""},{"id":"seller_reviews","parentSelectors":["seller_url"],"type":"SelectorText","selector":"div.seller-popup-title__link-customer-reviews","multiple":false,"regex":""}]}

Best regards,
Elvin

ViestursWS · June 14, 2023, 12:32pm

@nivvle Hello, it does not seem as if this element acts as a real pop-up which should appear in a separate window, therefore you should be able to extract this data using an 'Element click' selector instead.

Example:

{"_id":"bol","startUrl":["https://www.bol.com/nl/nl/p/petrol-industries-heren-voordeel-3-pack-t-shirt-roze-blauw-groen-maat-l/9300000149195449/?bltgh=klrvtxYQDH8WkJR3n9hCPQ.4_25.26.ProductImage"],"selectors":[{"id":"seller_name","multiple":false,"parentSelectors":["_root"],"regex":"","selector":".product-seller--l span","type":"SelectorHTML"},{"clickElementSelector":".product-seller--l a","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":4000,"discardInitialElements":"discard-when-click-element-exists","id":"seller_url","multiple":false,"parentSelectors":["_root"],"selector":"div.modal__window","type":"SelectorElementClick"},{"id":"seller_rating","multiple":false,"parentSelectors":["seller_url"],"regex":"","selector":"div.seller-rating--xl","type":"SelectorText"},{"id":"seller_more","multiple":false,"parentSelectors":["seller_url"],"selector":"a.ui-btn--outline--secondary","type":"SelectorLink"},{"id":"seller_offers","multiple":false,"parentSelectors":["seller_more"],"regex":"","selector":".large--is-visible a.nav-list__link","type":"SelectorHTML"},{"id":"seller_kvk_name","multiple":false,"parentSelectors":["seller_more"],"regex":"","selector":"dt:contains('Handelsnaam') + dd","type":"SelectorHTML"},{"id":"seller_reviews","multiple":false,"parentSelectors":["seller_url"],"regex":"","selector":"div.seller-popup-title__link-customer-reviews","type":"SelectorText"}]}