How to scrape phone numbers that are hidden

How to?
#1

hello, i'm trying to scrape the cities, the name and the phone numbers of the different companies.
with this sitemap the scrapper only click on the first "mostra telefono" and then goes to the next page
if somebody could help me getting it to click on every button and storing the phone numbers that would be perfect

Url: https://www.paginegialle.it/ricerca/concessionario%20moto/Campania/p-19
Sitemap:

{"_id":"concessionariomotocampania","startUrl":["https://www.paginegialle.it/ricerca/concessionario%20moto/Campania/p-[1-24]"],"selectors":[{"id":"Nom","type":"SelectorText","selector":"h1.fn a","parentSelectors":["_root"],"multiple":true,"regex":"","delay":0},{"id":"Ville","type":"SelectorText","selector":"section.vcard:nth-of-type(2) span.locality","parentSelectors":["_root"],"multiple":false,"regex":"","delay":0},{"id":"Phone","type":"SelectorElementClick","selector":"section.vcard:nth-of-type(1) div.hidden-phone-elem","parentSelectors":["_root"],"multiple":true,"delay":"1000","clickElementSelector":"div.btn.btn-yellow","clickType":"clickOnce","discardInitialElements":false,"clickElementUniquenessType":"uniqueHTMLText"},{"id":"phonenumber","type":"SelectorText","selector":"span.phone-label","parentSelectors":["Phone"],"multiple":false,"regex":"","delay":0}]}

#2

Hi,

you do not need to click any buttons, as the phone hidden inside the div element, so you just have to pick the right 'spot', you can easily find needed element using Chrome browser 'Elements' tab, using element inspect tool (Ctrl + Shift + C).

Here's the updated sitemap:

{"_id":"concessionariomotocampania","selectors":[{"id":"Nom","type":"SelectorElement","selector":"section.vcard:nth-of-type(n) div.col.contentCol","parentSelectors":["_root"],"multiple":true,"delay":"0"},{"parentSelectors":["Nom"],"type":"SelectorText","multiple":false,"id":"Company name","selector":"h1.fn a","regex":"","delay":""},{"parentSelectors":["Nom"],"type":"SelectorText","multiple":false,"id":"Location","selector":"div.street-address ","regex":"","delay":""},{"parentSelectors":["Nom"],"type":"SelectorText","multiple":false,"id":"Phone","selector":"span.phone-label","regex":"","delay":""}],"startUrl":"https://www.paginegialle.it/ricerca/concessionario%20moto/Campania/p-[1-24]"}

I'd recommend you to use CouchDB so results order will persist during scrape. Also got to mention that if you decide to use CouchDB you will have to go from end to the beginning page, e.g. from last page to first, it will result exactly opposite from beginning to the ending page.

Hope I helped :slight_smile: