Hi, I need help for scraping mails on this website. I tried many different extensions and python programs but none of them could do it. Basically, this is a trade fair and there are many exhibitors. Each of them has a "View Information" button, which opens a popup window(also all of them has their own link), and the mails are in these popup windows. Even if there is a way to scrape links for each of these popup windows, I can make a email scrapper with python to go through these websites. But I couldn't find a way to find these links with the scrapper. Any help is appreciated.
Hi, please see the sitemap below for reference. Note that the results will be shown only after all the pages have been scraped. I have set a 'Element limit' for the scroll-down selector, you can increase it, to get more results.
{"_id":"messe-stuttgart","startUrl":["https://www.messe-stuttgart.de/fairhandeln/besucher/ausstellerverzeichnis#/"],"selectors":[{"delay":2000,"elementLimit":40,"id":"scroll-down","multiple":true,"parentSelectors":["_root"],"selector":"div.ed-item","type":"SelectorElementScroll"},{"clickActionType":"real","clickElementSelector":"[class=\"ed-item\"] a","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":1000,"discardInitialElements":"do-not-discard","id":"info-click","multiple":true,"parentSelectors":["_root"],"selector":"_parent_","type":"SelectorElementClick"},{"id":"e-mail","multiple":false,"parentSelectors":["info-click"],"regex":"","selector":"[title=\"E-Mail schreiben\"]","type":"SelectorText"}]}
First of all, thank you a lot for the help. But there is a slight problem, when I use this sitemap it clicks only half of the exhibitors. For example, it clicks at the 1st one, 3rd one, 5th one, 7th one, and goes on. Skipping one out of every two. Is this a problem on my end, do you know how can I solve this?
Yes, you are right. It was necessary to add a click to close the pop-up.
{"_id":"messe-stuttgart","startUrl":["https://www.messe-stuttgart.de/fairhandeln/besucher/ausstellerverzeichnis#/"],"selectors":[{"id":"scroll-down","parentSelectors":["_root"],"type":"SelectorElementScroll","selector":"div.ed-item","multiple":true,"delay":2000,"elementLimit":30},{"id":"info-click","parentSelectors":["_root"],"type":"SelectorElementClick","clickActionType":"real","clickElementSelector":"[class=\"ed-item\"] a","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":1000,"discardInitialElements":"do-not-discard","multiple":true,"selector":"_parent_"},{"id":"e-mail","parentSelectors":["info-click"],"type":"SelectorText","selector":"[title=\"E-Mail schreiben\"]","multiple":false,"regex":""},{"id":"close-window","parentSelectors":["info-click"],"type":"SelectorElementClick","clickActionType":"real","clickElementSelector":".modal-close i","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":1000,"discardInitialElements":"do-not-discard","multiple":false,"selector":"_parent_"}]}
Thank you so much, you are a life saver. It works perfectly. It was not scraping the mails at first, but I figured out that it was because I was translating the page to english. When I turned the translation off it worked with no problems. I just wanted to write this in case it might be useful to anyone who has a similar problem.