Describe the problem.
I am trying to scrape data of elite swimmers from the FINA website. This site links to sub-pages with individual athlete data, where I am intending to scrape the data from.
The issue is that the root page has a 'Show more' button and a need for the user to scroll down for the page to continue loading. I've made use of the click and scroll selectors but for some reason, the click function does not automatically continue to click on 'Show more' after one or two loops, unless I include multiple scroll+click selectors. I am hoping to avoid this because of the lengthy list of swimmers for some of the countries.
I've been working on this for the last week and have pretty much hit a roadblock and would greatly appreciate any assistance.
Cheers
Url:
The root site is : <fina.org | Official FINA Website >
An example of an athlete sub page is:
Sitemap:
{"_id":"fina_aus","startUrl":["fina.org | Official FINA Website Length","name":"Pool Length","extract":true},{"header":"Age*","name":"Age*","extract":true},{"header":"Competition","name":"Competition","extract":true},{"header":"Comp Country","name":"Comp Country","extract":true},{"header":"Date","name":"Date","extract":true}],"delay":0,"tableDataRowSelector":"tr.athlete-table__row","tableHeaderRowSelector":"tr.athlete-table__header-row"},{"id":"link2","type":"SelectorLink","parentSelectors":["link2"],"selector":"a[title='Profile']","multiple":false,"delay":0}]}