Can't find a solution regarding element click selector problem

How to?
#1

Hi y'all,

for a research regarding gym trends in Germany, I need to scrape the website, phone number and of course the name of gyms from a few sites.
Everything is going great, but with this particular site there is a problem.
There are 1514 gyms on the page, but it always stops at 501 gyms - so I guess there is a problem with the "load more" - button.
I followed every step in the tutorial and as I said, until 501 it works.

Any help would be so great!

Url: https://qualitrain.net/studios/jobs

Sitemap:
{"_id":"nameaddress","startUrl":["https://qualitrain.net/studios/jobs"],"selectors":[{"id":"card","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.thumbnail","multiple":true,"delay":"2000","clickElementSelector":"a.btn","clickType":"clickMore","discardInitialElements":true,"clickElementUniquenessType":"uniqueText"},{"id":"phone","type":"SelectorText","parentSelectors":["card"],"selector":"a.listing-contact.listing--phone","multiple":false,"regex":"","delay":0},{"id":"Links","type":"SelectorLink","parentSelectors":["card"],"selector":"a.listing-contact.listing--website","multiple":false,"delay":0}]}

#2

Hi there!

If you didn't notice, page starts to lag hard once you press load more button more than 11 times, I would recommend you to increase delay up to 10 seconds (10000 ms) for page to successfully load the results.

#3

Hi iconoclast,

thank you so much!

Works fine now, really appreciate your advice:)

Best regards,

leoncino

#4

@iconoclast

Actually, I just had a look at the data again - it stops at 501, even with the increased delay.

You got any idea how to proceed?

Thank you so much for your time and advice!