Help with scrapping https://www.ffjudo.com/les-clubs

How to?
#1

Hi there !

I have been trying for a couple of days to scrape the following website: https://www.ffjudo.com/les-clubs . I can easily scrape the list of Judo clubs shown on the left side of the page. :

image
image881×706 39.5 KB

Nevertheless, I can not make Web Scrapper do the following actions :
1 - Click on the div of one club
2 - Then click on the link called "Voir la fiche du club"
3 - Then do it for all the Judo Clubs

image
image892×760 190 KB

Sitemap:
{"_id":"FFjudo","startUrl":["https://www.ffjudo.com/les-clubs/[1-5]"],"selectors":[{"id":"record_wrapper","parentSelectors":["_root"],"type":"SelectorElementClick","clickActionType":"real","clickElementSelector":"div.grid2","clickElementUniquenessType":"uniqueText","clickType":"clickOnce","delay":2000,"discardInitialElements":"do-not-discard","multiple":true,"selector":"div.grid2"},{"id":"Location","parentSelectors":["LeftListing"],"type":"SelectorText","selector":"span.h4","multiple":false,"regex":"","multipleType":"singleColumn"},{"id":"email","parentSelectors":["LeftListing"],"type":"SelectorText","selector":".email a","multiple":false,"regex":"","multipleType":"singleColumn"},{"id":"ClubName","parentSelectors":["LeftListing"],"type":"SelectorText","selector":"h2","multiple":false,"regex":"","multipleType":"singleColumn"},{"id":"Sport","parentSelectors":["LeftListing"],"type":"SelectorText","selector":".discipline span","multiple":false,"regex":"","multipleType":"singleColumn"},{"id":"LeftListing","parentSelectors":["record_wrapper"],"type":"SelectorElement","selector":"div.card_club","multiple":true,"scroll":false,"elementLimit":0},{"id":"Map","parentSelectors":["record_wrapper"],"type":"SelectorElementClick","clickActionType":"real","clickElementSelector":"div.card_club","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":2000,"discardInitialElements":"do-not-discard","multiple":true,"selector":"div.carte"},{"id":"LinkClub","parentSelectors":["Map"],"type":"SelectorLink","selector":".leaflet-popup-content a","multiple":true,"linkType":"linkFromHref"},{"id":"Adresse","parentSelectors":["LinkClub"],"type":"SelectorText","selector":"div.dojo","multiple":false,"regex":"","multipleType":"singleColumn"}]}

Anyone can help me with that ?

#2

Hi,

This will not be possible to do in 1 sitemap as opening the listing links breaks the click selector sequence.

You can create a sitemap to scrape all the listing URLs and then create a separate sitemap with these URLs set as start URLs. Or, if you are familiar with regex, you can extract the address data from a script within the website's HTML.

Here is a sitemap for scraping the URLs. You can also check the Data preview for the script selector. It will return all the address data at once, but you will have to use regex to get the data in a readable format.

{"_id":"FFjudo-links","startUrl":["https://www.ffjudo.com/les-clubs/[1-5]"],"selectors":[{"clickActionType":"real","clickElementSelector":"div.card_club","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":1000,"discardInitialElements":"do-not-discard","id":"Map","multiple":true,"parentSelectors":["_root"],"selector":"div.carte","type":"SelectorElementClick"},{"id":"LinkClub","linkType":"linkFromHref","multiple":true,"parentSelectors":["Map"],"selector":".leaflet-popup-content a","type":"SelectorLink"},{"id":"script","multiple":false,"multipleType":"singleColumn","parentSelectors":["_root"],"regex":"","selector":"script:contains('nomClub')","type":"SelectorText"}]}