Scrape interactive map + get info from redirection to other page

How to?
#1

Describe the problem:

I want to scrape this website: https://www.sport2000.fr/nos-magasins, but I need to collect data directly on this website + on page redirection to get the email address:

  • Main info of shops are available directly on the website through pagination (bottom of the page 1 to 133) - OR - in the interactive map with one pop up / shop
  • Email address for each shop is available when you click on the shop name, it goes to an other webpage

Any idea how to process this ? I cant figure out how to use webscraper in this case.

My sitmap so far (trying to use map pop-up rather than pagination) :

{"_id":"sport2000-2","startUrl":["https://www.sport2000.fr/nos-magasins"],"selectors":[{"id":"elementclick","type":"SelectorElementClick","parentSelectors":["_root"],"selector":".gm-style-iw-d div.holder","multiple":true,"delay":0,"clickElementSelector":"[tabindex] > div:nth-of-type(3)","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"link","type":"SelectorLink","parentSelectors":["elementclick"],"selector":"a","multiple":false,"delay":0},{"id":"adresse","type":"SelectorText","parentSelectors":["elementclick"],"selector":"span[itemprop='street-address']","multiple":false,"regex":"","delay":0},{"id":"name","type":"SelectorText","parentSelectors":["elementclick"],"selector":"a","multiple":false,"regex":"","delay":0},{"id":"tel","type":"SelectorText","parentSelectors":["elementclick"],"selector":"div.phone","multiple":false,"regex":"","delay":0},{"id":"email adress","type":"SelectorText","parentSelectors":["link"],"selector":"a.store_link","multiple":false,"regex":"","delay":0}]}

Thank you in advance!

#2

The store links at the bottom are plain HTML links (a href) so you can just use Type: Link to click into them.

Something like this:
{"_id":"z_sport2000-test","startUrl":["https://www.sport2000.fr/nos-magasins"],"selectors":[{"id":"Store wrappers","type":"SelectorElement","parentSelectors":["_root"],"selector":"#slide_pane_0 div.box","multiple":true,"delay":0},{"id":"Store name and URL","type":"SelectorLink","parentSelectors":["Store wrappers"],"selector":"[itemprop='name'] a","multiple":false,"delay":0},{"id":"Address","type":"SelectorText","parentSelectors":["Store wrappers"],"selector":".txt > span","multiple":false,"regex":"","delay":0},{"id":"Tel","type":"SelectorText","parentSelectors":["Store wrappers"],"selector":"span ~ p","multiple":false,"regex":"","delay":0},{"id":"Email","type":"SelectorText","parentSelectors":["Store name and URL"],"selector":"a.store_link","multiple":false,"regex":"","delay":0}]}

You'd need to specify a long enough Page load delay to allow the bottom data to load; I used 8000.

#3

@leemeng great thank you! I can see that it collects now email addresses. But only on one page.
How can I manage to scrape each page (1 to 133).

I tried to replace Element selector by Click element selector but it does not seem to work.

Sitemap:
{"_id":"sport2000-leemeng","startUrl":["https://www.sport2000.fr/nos-magasins"],"selectors":[{"id":"Store wrappers","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"#slide_pane_0 div.box","multiple":true,"delay":0,"clickElementSelector":".nav-bar div:nth-of-type(n+2)","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"Store name and URL","type":"SelectorLink","parentSelectors":["Store wrappers"],"selector":"[itemprop='name'] a","multiple":false,"delay":0},{"id":"Address","type":"SelectorText","parentSelectors":["Store wrappers"],"selector":".txt > span","multiple":false,"regex":"","delay":0},{"id":"Tel","type":"SelectorText","parentSelectors":["Store wrappers"],"selector":"span ~ p","multiple":false,"regex":"","delay":0},{"id":"Email","type":"SelectorText","parentSelectors":["Store name and URL"],"selector":"a.store_link","multiple":false,"regex":"","delay":0}]}

#4

@leemeng would you have any idea how to manage pagination from 1 to 133 ?
I(ve tried an other way but failed again :frowning:

Thank you again for your help.

#5

ya try this:
{"_id":"z_sport2000-test2","startUrl":["https://www.sport2000.fr/nos-magasins"],"selectors":[{"id":"Store wrappers","type":"SelectorElement","parentSelectors":["_root","Paginator"],"selector":"div[itemtype='http://data-vocabulary.org/Organization']","multiple":true,"delay":0},{"id":"Store name and URL","type":"SelectorLink","parentSelectors":["Store wrappers"],"selector":"[itemprop='name'] a","multiple":false,"delay":0},{"id":"Address","type":"SelectorText","parentSelectors":["Store wrappers"],"selector":".txt > span","multiple":false,"regex":"","delay":0},{"id":"Tel","type":"SelectorText","parentSelectors":["Store wrappers"],"selector":"span ~ p","multiple":false,"regex":"","delay":0},{"id":"Email","type":"SelectorText","parentSelectors":["Store name and URL"],"selector":"a.store_link","multiple":false,"regex":"","delay":0},{"id":"Paginator","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"ul.results-holder","multiple":false,"delay":"4100","clickElementSelector":"div > div.nav-bar > div.navBtn.btn-next","clickType":"clickMore","discardInitialElements":"discard-when-click-element-exists","clickElementUniquenessType":"uniqueText"}]}