Help with "Click to obtain information" link

Background info:

Am scraping the below website:

Which then opens up sites such as the below:

On these sites, I am able to scrape all the data apart from the "Phone" information, as that needs to be clicked to be found out.

Here is my code so far, have tried obtaining the phone information with a "Element click" feature but does not seem to be working.

{"_id":"fruitlogistica","startUrl":["https://www.virtualmarket.fruitlogistica.de/en/exhibitors?view=2&limit=96"],"selectors":[{"id":"link","type":"SelectorLink","parentSelectors":["_root"],"selector":".ngn-content-box-title a","multiple":true,"delay":0},{"id":"name","type":"SelectorText","parentSelectors":["link"],"selector":"bdi[itemprop='name']","multiple":false,"regex":"","delay":0},{"id":"address","type":"SelectorText","parentSelectors":["link"],"selector":"address","multiple":false,"regex":"","delay":0},{"id":"website","type":"SelectorElementAttribute","parentSelectors":["link"],"selector":"a[itemprop='url']","multiple":false,"extractAttribute":"href","delay":0},{"id":"phone","type":"SelectorElementClick","parentSelectors":["link"],"selector":"bdi[itemprop='telephone']","multiple":false,"delay":0,"clickElementSelector":".ngn-box__content button","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"description","type":"SelectorText","parentSelectors":["link"],"selector":"[itemprop='description'] bdi","multiple":false,"regex":"","delay":0},{"id":"contact","type":"SelectorText","parentSelectors":["link"],"selector":"a bdi[itemprop='name']","multiple":false,"regex":"","delay":0},{"id":"contact role","type":"SelectorText","parentSelectors":["link"],"selector":"div[itemprop='jobTitle']","multiple":false,"regex":"","delay":0}]}

Any advice would be appreciated :slight_smile:

I have managed to resolve it simply by using the text field - not so complicated - please find map below:

{"_id":"fruitlogistica","startUrl":["https://www.virtualmarket.fruitlogistica.de/en/exhibitors?view=2&limit=96"],"selectors":[{"id":"link","type":"SelectorLink","parentSelectors":["_root"],"selector":".ngn-content-box-title a","multiple":true,"delay":0},{"id":"name","type":"SelectorText","parentSelectors":["link"],"selector":"bdi[itemprop='name']","multiple":false,"regex":"","delay":0},{"id":"address","type":"SelectorText","parentSelectors":["link"],"selector":"address","multiple":false,"regex":"","delay":0},{"id":"website","type":"SelectorElementAttribute","parentSelectors":["link"],"selector":"a[itemprop='url']","multiple":false,"extractAttribute":"href","delay":0},{"id":"phone","type":"SelectorText","parentSelectors":["link"],"selector":"bdi[itemprop='telephone']","multiple":false,"regex":"","delay":0},{"id":"description","type":"SelectorText","parentSelectors":["link"],"selector":"[itemprop='description'] bdi","multiple":false,"regex":"","delay":0},{"id":"contact","type":"SelectorText","parentSelectors":["link"],"selector":"a bdi[itemprop='name']","multiple":false,"regex":"","delay":0},{"id":"contact role","type":"SelectorText","parentSelectors":["link"],"selector":"div[itemprop='jobTitle']","multiple":false,"regex":"","delay":0}]}

Additional question now, does anyone know how I could get this code to do all the pages at once? Without chrome crashing? Or would this be too much?

Great that you are experimenting and finding solutions.

There is indeed a known issue with WS crashing when there is too much data.

This site has pages which change the Url, e.g. https://www.virtualmarket.fruitlogistica.de/en/exhibitors?view=2&page=3

So you can just use the Specify multiple urls with ranges method to limit the number of pages per session. About 10 pages per session should be Ok (10 x 96 = 960 companies)..

Hi Lee,

Thanks a lot for the help - appreciated - working like a treat now.

:slight_smile: