Can't scrape more than one page

How to?
#1

Hello,

Url: Plan My Run - L'assistant Running en ligne

I've made sitemap to scrape email adress from the given website. I need to click on the little red eye button to load a new page containing informations.

So this sitemap is working for the first page :

{"_id":"planmyrun","startUrl":["Plan My Run - L'assistant Running en ligne a.waves-effect","multiple":true,"delay":0},{"id":"email","type":"SelectorText","parentSelectors":["link"],"selector":"[ng-if='race.email'] span","multiple":false,"regex":"","delay":0}]}

Now, I'd like to make it work for all the pages.

Here's the sitemap visiting the pages :
{"_id":"test_page","startUrl":["Plan My Run - L'assistant Running en ligne .pagination div.ng-scope:nth-of-type(n+4)","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"links","type":"SelectorLink","parentSelectors":["click_pagination"],"selector":"a[ng-href='#/racedetail/32764']","multiple":true,"delay":0}]}

The thing is I can't combine these two sitemaps to visit all links in each page. It's only visiting all the pages, no data is scraped.

Please, can someone help me ?

Thanks in advance.

#2

Hi @Loic_Cosy

Please, when pasting your sitemap here. Mark it and use "preformatted text" option otherwise your JSON might be invalid and it's not possible to see your sitemap.

Preformatted Text
Preformatted Text1096×81 6.7 KB

Invalid Json
Invalid Json2544×588 45.1 KB

Other than that it was not hard to figure out what you are after.
Note that results will start to appear only after Web Scraper has clicked through all of the pages.

{"_id":"panmyrun-fr","startUrl":["https://planmyrun.fr/#/raceengine?r=17"],"selectors":[{"id":"wrapper-click-results-only after-click-completes","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.REcontainer > div:contains(\"km\")","multiple":true,"delay":2000,"clickElementSelector":"a:contains(\"chevron_right\"):nth(0)","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"link","type":"SelectorLink","parentSelectors":["wrapper-click-results-only after-click-completes"],"selector":"a","multiple":false,"delay":0},{"id":"element-card","type":"SelectorElement","parentSelectors":["link"],"selector":"body:has(div.racelabel)","multiple":true,"delay":0},{"id":"e-mail","type":"SelectorText","parentSelectors":["element-card"],"selector":"[ng-if='race.email'] span","multiple":false,"regex":"","delay":0},{"id":"race title","type":"SelectorText","parentSelectors":["element-card"],"selector":"div.racelabel","multiple":false,"regex":"","delay":0}]}

Hope it helps! :slightly_smiling_face: