How to extract data from a visual popup?

How to?
#1

I'm trying to extract data (logo image, name, location, etc) from this website : Meet the startups that joining us at Web Summit 2021

The problem is that when clicking on an image, a visual popup appears (the url doesn't change) so i tried various selectors (site map example i used : home >> popup selector (with the Multiple option) >> text selectors for name, location etc) but none of them works (i could see the data but after scraping no data is available)

Can someone please tell me what selectors to use to scrap the data in the visual popups ?

#2

@Manny Hi there, you should be able to get through with 2 'Element click' selectors.

Example:

{"_id":"websummit-com","startUrl":["https://websummit.com/startups/featured-startups?q=eyJwYWdlIjoxLCJjb25maWd1cmUiOnsiaGl0c1BlclBhZ2UiOjQ4LCJ0YWdGaWx0ZXJzIjpbIndzMjEiXX19"],"selectors":[{"id":"open-click","parentSelectors":["_root"],"type":"SelectorElementClick","clickElementSelector":"a.MuiPaper-root","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":"1200","discardInitialElements":"discard-when-click-element-exists","multiple":true,"selector":"html"},{"id":"slider-tray-click","parentSelectors":["open-click"],"type":"SelectorElementClick","clickElementSelector":"button[class*=\"buttonNext\"]","clickElementUniquenessType":"uniqueHTMLText","clickType":"clickMore","delay":"900","discardInitialElements":"do-not-discard","multiple":true,"selector":"ul.carousel__slider-tray--horizontal > div"},{"id":"title","parentSelectors":["slider-tray-click"],"type":"SelectorText","selector":"h1","multiple":false,"delay":0,"regex":""},{"id":"description","parentSelectors":["slider-tray-click"],"type":"SelectorText","selector":"p","multiple":false,"delay":0,"regex":""},{"id":"link-1","parentSelectors":["slider-tray-click"],"type":"SelectorLink","selector":"a[target=\"_blank\"]:has(path[d*=\"M3.9\"])","multiple":false,"delay":0}]}

1 Like
#3

Hey @viesturs @Manny , could you please advise, how to fix that scraper is only scraping one page? And there are lots of duplicates( can you please check. Thank you

#4

@Norman Hi, try to click on the first listing for each of the pages.

Example:

{"_id":"websummit-com","startUrl":["https://websummit.com/startups/featured-startups?q=eyJwYWdlIjoxLCJjb25maWd1cmUiOnsiaGl0c1BlclBhZ2UiOjQ4LCJ0YWdGaWx0ZXJzIjpbIndzMjEiXX19"],"selectors":[{"clickElementSelector":"a.MuiPaper-root:nth(0)","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":1200,"discardInitialElements":"discard-when-click-element-exists","id":"open-click","multiple":true,"parentSelectors":["_root"],"selector":"html","type":"SelectorElementClick"},{"id":"slider","multiple":true,"parentSelectors":["_root"],"selector":"ul.carousel__slider-tray--horizontal > div","type":"SelectorElement"},{"id":"title","multiple":false,"parentSelectors":["slider"],"regex":"","selector":"h1","type":"SelectorText"},{"id":"description","multiple":false,"parentSelectors":["slider"],"regex":"","selector":"p","type":"SelectorText"},{"id":"link-1","multiple":false,"parentSelectors":["slider"],"selector":"a[target=\"_blank\"]:has(path[d*=\"M3.9\"])","type":"SelectorLink"}]}