Scrape data from a pop-up (with the same URL)

[Tiitii67]

Hello everyone,

I'm trying to scrape the Name/email that only appear on a pop-up after clicking on a bouton but the URL do not change. Thus, the 'link' or even 'pop-up link' elements on webscraper do not work (or at least I'm not using it right) and the 'element click' also seems useless.

I'll show an example on one single target to simplify the understanding of my issue.
Let's take this link: https://www.degulesider.dk/frisø'r+østbirk/163839400/firma?page=1&query=frisør

Step 1, Clic on the link 'sent email' to open a Pop-up (the URL do not change)

Step 2, extract the name and email on this popup

Here is a "simplify" version of the Sitemap (to focus on my issue):

{"_id":"testfrisor","startUrl":["https://www.degulesider.dk/frisø'r+østbirk/163839400/firma?page=1&query=frisør"],"selectors":[{"delay":0,"id":"email","multiple":false,"parentSelectors":["_root"],"selector":"a.e-icon-envelope","type":"SelectorPopupLink"},{"delay":0,"id":"name","multiple":false,"parentSelectors":["email"],"regex":"","selector":"#email-form h2","type":"SelectorText"}]}

Thanks in advance for the community and for your help

Have a great week

[ViestursWS]

@Tiitii67 Hi, this website uses an anti-scraping mechanism that is based on 3 e-mail variants.

In order to extract the desired data, you will have to use the 'Grouped' selector in a combination with the 'Element click' selector + apply some data post-processing using the Web Scraper Cloud parser.

Sitemap example:

{"_id":"degulesider-dk-1","startUrl":["https://www.degulesider.dk/fris%C3%B8r+savannah+s%C3%B8borg/67225271/firma?page=1&query=fris%c3%b8r"],"selectors":[{"delay":0,"id":"link","multiple":true,"parentSelectors":["_root"],"selector":".name a","type":"SelectorLink"},{"clickElementSelector":"a.e-icon-envelope","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":1000,"discardInitialElements":"discard-when-click-element-exists","id":"popup","multiple":true,"parentSelectors":["link"],"selector":"body","type":"SelectorElementClick"},{"delay":0,"id":"namn","multiple":false,"parentSelectors":["popup"],"regex":"","selector":"#email-form h2","type":"SelectorText"},{"delay":0,"extractAttribute":"","id":"email-v1","parentSelectors":["popup"],"selector":"#email-form div.v span:nth-of-type(3n)","type":"SelectorGroup"},{"delay":0,"extractAttribute":"","id":"email-v2","parentSelectors":["popup"],"selector":"#email-form div.b span:nth-of-type(4n)","type":"SelectorGroup"},{"delay":0,"extractAttribute":"","id":"email-v3","parentSelectors":["popup"],"selector":"#email-form div.z span:nth-of-type(2n)","type":"SelectorGroup"},{"delay":0,"id":"email-v4","multiple":false,"parentSelectors":["popup"],"regex":"","selector":"aaaa","type":"SelectorText"}]}

Hope it helps!

[Tiitii67]

Thanks a lot @ViestursWS ! this helps a lot.

Great community

[sirine_kabboura]

i have the same problem but i can't resolve it
Let's take this link: Redirecting...
step 1 : click on the link 'Preparation Before PFE' to open a Pop-up (the URL do not change)

Step 2, extract the name and description on this popup

Here is a "simplify" version of the Sitemap (to focus on my issue):
{"_id":"test_popup","startUrl":["https://www.facebook.com/9antra.tn/services"],"selectors":[{"id":"posts","parentSelectors":["_root"],"type":"SelectorElement","selector":"div.x1q0q8m5.xyamay9","multiple":true},{"id":"details","parentSelectors":["posts"],"type":"SelectorElementClick","clickActionType":"real","clickElementSelector":"span.xlyipyv","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickMore","delay":5500,"discardInitialElements":"do-not-discard","multiple":true,"selector":"div.x6umtig"}]}

[leemeng]

Facebook uses a lot of React so you will need better selectors. Fortunately, the details are already in the main page HTML so you do not need to click to trigger the popups. Try this one:

{"_id":"facebook-services","startUrl":["https://www.facebook.com/9antra.tn/services"],"selectors":[{"id":"services elements","multiple":true,"parentSelectors":["_root"],"selector":"div[style*='border-radius'] > div:contains('Services') > div:nth-of-type(3) > div:not([class=\"\"])","type":"SelectorElement"},{"id":"Title","multiple":false,"parentSelectors":["services elements"],"regex":"","selector":"span > div[role='button'][tabindex='0']","type":"SelectorText"},{"id":"Cost","multiple":false,"parentSelectors":["services elements"],"regex":".+(?=·)","selector":"div > div:nth-of-type(2) > span[dir='auto']","type":"SelectorText"},{"id":"Duration","multiple":false,"parentSelectors":["services elements"],"regex":"(?<=·).+","selector":"div > div:nth-of-type(2) > span[dir='auto']","type":"SelectorText"},{"id":"Details","multiple":false,"parentSelectors":["services elements"],"regex":"","selector":"div span[style*='webkit-box']","type":"SelectorText"}]}

[sirine_kabboura]

thank you very much it works but i want know the way to scrap from a popup because i have another example that don't have the details in the main page i try this but i didn't resolve the problem
{"_id":"services_test_popup","startUrl":["https://www.facebook.com/9antra.tn/services"],"selectors":[{"id":"posts","multiple":true,"parentSelectors":["_root"],"selector":"div[style*='border-radius'] > div:contains('Services') > div:nth-of-type(3) > div:not([class=""])","type":"SelectorElement"},{"clickActionType":"real","clickElementSelector":"span > div[role='button'][tabindex='0']","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":1000,"discardInitialElements":"discard","id":"popup_body","multiple":true,"parentSelectors":["posts"],"selector":"div[aria-label="Détails"][role="dialog"]","type":"SelectorElementClick"},{"id":"Cost","multiple":false,"parentSelectors":["popup_body"],"regex":".+(?=·)","selector":"div > div:nth-of-type(2) > span[dir='auto']","type":"SelectorText"}]}