Scrape from the pop-up(popup)

How to?
#1

Hello, guys, I have issue with scraping from the popup of the website

I would need to scrape data from the popup

image
image1199×491 73.4 KB

And I Was using "popup link selector" but looks like it stops and not scraping data.

Below is my site map

{"_id":"messe","startUrl":["https://www.hannovermesse.de/en/side-events/speakers/index-2"],"selectors":[{"id":"user","parentSelectors":["_root"],"type":"SelectorElement","selector":"div.exhibitor-list-snippets:nth-of-type(2) section:nth-of-type(1)","multiple":false,"delay":0},{"id":"link","parentSelectors":["user"],"type":"SelectorPopupLink","selector":"div.grid-x","multiple":false,"delay":0},{"id":"name","parentSelectors":["link"],"type":"SelectorText","selector":"h1.set-600-bold","multiple":false,"delay":0,"regex":""}]}

Thnak you for the help

#2

@Norman Hi, it looks like the most viable option will require using an 'Element click' selector instead, as the targeted element, technically, does not function as a 'Pop-Up'.

Example:

{"_id":"messe","startUrl":["https://www.hannovermesse.de/en/side-events/speakers/index-2"],"selectors":[{"delay":0,"id":"name","multiple":false,"parentSelectors":["click-1"],"regex":"","selector":"h1.as-headline","type":"SelectorText"},{"clickElementSelector":"div.as-headline:contains(\"Murtaza\")","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":2000,"discardInitialElements":"discard-when-click-element-exists","id":"click-1","multiple":true,"parentSelectors":["_root"],"selector":"div.speaker-detail-box","type":"SelectorElementClick"}]}

1 Like
#3

Thank you , this works, Could you advise how to make this scripts scrape all other speakers, popup stuck after the first scrape

image
image1262×304 116 KB

{"_id":"messe2","startUrl":["https://www.hannovermesse.de/en/side-events/speakers/index-2"],"selectors":[{"delay":0,"id":"name","multiple":false,"parentSelectors":["click-1"],"regex":"","selector":"h1.as-headline","type":"SelectorText"},{"clickElementSelector":"div.set-100-bold","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickMore","delay":2000,"discardInitialElements":"discard-when-click-element-exists","id":"click-1","multiple":false,"parentSelectors":["_root"],"selector":"div.speaker-detail-box","type":"SelectorElementClick"},{"delay":0,"id":"linkedin","multiple":false,"parentSelectors":["click-1"],"selector":"a:nth-of-type(5)","type":"SelectorLink"},{"delay":0,"id":"title","multiple":false,"parentSelectors":["click-1"],"regex":"","selector":"h3.set-250-regular","type":"SelectorText"},{"delay":0,"id":"company","multiple":false,"parentSelectors":["click-1"],"regex":"","selector":"h3:nth-of-type(3)","type":"SelectorText"},{"delay":0,"id":"tel","multiple":false,"parentSelectors":["click-1"],"regex":"","selector":"div.speaker-detail-phone","type":"SelectorText"}]}

this one works, but it gives result only after scraping whole elements, and because it takes a lot of time, it crashes without any result, how to make that result of scraped data was shown after each scrape

{"_id":"messe_forum","startUrl":["https://www.hannovermesse.de/en/side-events/speakers/index-2"],"selectors":[{"delay":0,"id":"name","multiple":false,"parentSelectors":["click-1"],"regex":"","selector":"h1.as-headline","type":"SelectorText"},{"clickElementSelector":"div.set-100-bold","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":2000,"discardInitialElements":"discard-when-click-element-exists","id":"click-1","multiple":true,"parentSelectors":["_root"],"selector":"div.speaker-detail-box","type":"SelectorElementClick"},{"delay":0,"id":"title","multiple":false,"parentSelectors":["click-1"],"regex":"","selector":"h3.set-250-regular","type":"SelectorText"},{"delay":0,"id":"company","multiple":false,"parentSelectors":["click-1"],"regex":"","selector":"h3:nth-of-type(3)","type":"SelectorText"},{"delay":0,"id":"tel","multiple":false,"parentSelectors":["click-1"],"regex":"","selector":"div.speaker-detail-phone","type":"SelectorText"},{"delay":0,"id":"linkedin","multiple":false,"parentSelectors":["click-1"],"selector":"a:nth-of-type(5)","type":"SelectorLink"}]}

#4

@Norman Unfortunately, the results will start to appear once the scraper has clicked through all of the available results.

Learn more: My scraping job is running, although no results are being returned - Web Scraper Knowledge Base

1 Like