How to grab the links of the search results if the link url is not a href link?

How to?
#1

Why I am not able to grab/follow the links of the search results of this search engine?

(if it helps here there is the same search engine in english language too: Italian Business Register | Italian companies official data )

The url has a token in the search results, so to update the sitemap link and to get the search results you shoudl do that:

  1. Go to Registro Imprese | I dati Ufficiali della Camera di Commercio
    and then enter this text "SOCIETA SPORTIVA" into the field "Nome Impresa o Attività" and then click on the button "CERCA":
    image
    image1190×355 38.5 KB

    then you will see the search results:
    image
    image1306×926 108 KB

Url: Registro Imprese | I dati Ufficiali della Camera di Commercio

Sitemap:
{"_id":"registroimprese-it","startUrl":["https://www.registroimprese.it/ricerca-libera-e-acquisto?p_p_id=ricercaportlet_WAR_ricercaRIportlet&p_p_lifecycle=0&p_p_state=normal&_ricercaportlet_WAR_ricercaRIportlet_cur=1&_ricercaportlet_WAR_ricercaRIportlet_pageToken=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2OTU0Nzk5MzEsImNvdW50IjoyNTB9.D5gR7-2Z8yXIA6PDJXMgGhPubFMyQ2RNSi_n_1_4HEA"],"selectors":[{"id":"page","paginationType":"clickMore","parentSelectors":["_root","page"],"selector":"#_ricercaportlet_WAR_ricercaRIportlet_BrPageIteratorBottom .pager li:nth-of-type(3) a","type":"SelectorPagination"}]}

#3

Using this code it grab me the same links for 3 times:

{"_id":"registroimprese-it2","startUrl":["https://www.registroimprese.it/ricerca-libera-e-acquisto?p_p_id=ricercaportlet_WAR_ricercaRIportlet&p_p_lifecycle=0&p_p_state=normal&_ricercaportlet_WAR_ricercaRIportlet_cur=1&_ricercaportlet_WAR_ricercaRIportlet_pageToken=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2OTU1NjA1ODksImNvdW50Ijo1MDB9.0_KJOVWUMeQaT8oAAoUS8rbNOk6M7KGh7MbReAEj_oQ"],"selectors":[{"id":"link","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root"],"selector":"a.linkRisultatiRicercaTabella","type":"SelectorLink"},{"id":"next","paginationType":"linkFromAttributes","parentSelectors":["_root","next"],"selector":"#_ricercaportlet_WAR_ricercaRIportlet_BrPageIteratorTop .pager li:nth-of-type(3) a","type":"SelectorPagination"}]}

Ok in some way I can try to cut the doubles quickly, before that the token make them to expire..
Do you have some suggestions about how to insert thousands of links into the sitemap?

#5

There's a site called Pastelink which can convert your URL list to clickable links, plus it will publish them on a page. So you can adjust your sitemap to scrape from that page (Start URL):

image
image977×813 87.7 KB