Scraping data from a page with load more button

How to?
#1

Hi,

I'm trying to scrap data from this test page : it show a list of shop with clickable links. Each link will display details of each shop (phone, instagram link, address etc..). The shop list page has a "click to see more result" button (voir plus de résultats - in French) and I'm having trouble to scrap data from this. I followed the tutorial on load more example but can't figure out in this specific case.

51
Capture d’écran 2018-01-16 à 11.46.51.jpg445×583 54.4 KB
14
Capture d’écran 2018-01-16 à 11.51.14.png1113×530 172 KB

Any help would be appreciated !

Test page
https://www.pagesmode.com/frnch?ville=PARIS(75009)#resultat

Sitemap:
{"_id":"test","startUrl":["https://www.pagesmode.com/frnch?ville=PARIS(75009)#resultat"],"selectors":[{"id":"load more","type":"SelectorElementClick","selector":"h3 a","parentSelectors":["_root","load more"],"multiple":false,"delay":0,"clickElementSelector":"div.load-more span.text","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"Shop page","type":"SelectorLink","selector":"h3 a","parentSelectors":["_root","load more"],"multiple":true,"delay":0},{"id":"adress","type":"SelectorText","selector":"div.line1","parentSelectors":["Shop page"],"multiple":false,"regex":"","delay":0},{"id":"instagram","type":"SelectorLink","selector":"div.socials a:nth-of-type(2)","parentSelectors":["Shop page"],"multiple":false,"delay":0},{"id":"telephone","type":"SelectorText","selector":"a.text","parentSelectors":["Shop page"],"multiple":false,"regex":"","delay":0},{"id":"ville","type":"SelectorText","selector":"div.line2 span:nth-of-type(2)","parentSelectors":["Shop page"],"multiple":false,"regex":"","delay":0},{"id":"arrondissement","type":"SelectorText","selector":"div.line2 span:nth-of-type(1)","parentSelectors":["Shop page"],"multiple":false,"regex":"","delay":0},{"id":"facebook","type":"SelectorLink","selector":"div.socials a:nth-of-type(1)","parentSelectors":["Shop page"],"multiple":false,"delay":0},{"id":"nom","type":"SelectorText","selector":"h1.title","parentSelectors":["Shop page"],"multiple":false,"regex":"","delay":0}]}

#2

You don't have to make element click selector recursive. If you want to scrape only URLs, it is better to use element attribute selector and set attribute to href. Here is the fixed sitemap:

{"_id":"test_pagesmode","startUrl":["https://www.pagesmode.com/frnch?ville=PARIS(75009)#resultat"],"selectors":[{"id":"load more","type":"SelectorElementClick","selector":"h3 a","parentSelectors":["_root"],"multiple":true,"delay":"2000","clickElementSelector":"div.load-more span.text","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"Shop page","type":"SelectorLink","selector":"_parent_","parentSelectors":["load more"],"multiple":false,"delay":0},{"id":"adress","type":"SelectorText","selector":"div.line1","parentSelectors":["Shop page"],"multiple":false,"regex":"","delay":0},{"id":"instagram","type":"SelectorElementAttribute","selector":"div.socials a:nth-of-type(2)","parentSelectors":["Shop page"],"multiple":false,"extractAttribute":"href","delay":0},{"id":"telephone","type":"SelectorText","selector":"a.text","parentSelectors":["Shop page"],"multiple":false,"regex":"","delay":0},{"id":"ville","type":"SelectorText","selector":"div.line2 span:nth-of-type(2)","parentSelectors":["Shop page"],"multiple":false,"regex":"","delay":0},{"id":"arrondissement","type":"SelectorText","selector":"div.line2 span:nth-of-type(1)","parentSelectors":["Shop page"],"multiple":false,"regex":"","delay":0},{"id":"facebook","type":"SelectorElementAttribute","selector":"div.socials a:nth-of-type(1)","parentSelectors":["Shop page"],"multiple":false,"extractAttribute":"href","delay":0},{"id":"nom","type":"SelectorText","selector":"h1.title","parentSelectors":["Shop page"],"multiple":false,"regex":"","delay":0}]}