How to limit element click to a specific number of times?

Is there any way to limit element click to some number? Like I want the scraper to click a maximum of 30 times on the "Show More" button and It should collect data from all elements that load in the first 30 clicks and then it should move on. Moreover, this sitemap doesn't work in a cloud scraper.

Url: https://www.inkitt.com/genres/romance

Sitemap:
{"_id":"inkitt","startUrl":["https://www.inkitt.com/genres/romance"],"selectors":[{"id":"click","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"[data-story-tracking-page='1'] a.storyCard-title","multiple":false,"delay":"100","clickElementSelector":"a.animated","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"Novel Name","type":"SelectorLink","parentSelectors":["click"],"selector":"parent","multiple":false,"delay":0},{"id":"Author","type":"SelectorLink","parentSelectors":["Novel Name"],"selector":"a[data-cta='Profile Picture']","multiple":true,"delay":0},{"id":"Rating and Reviews","type":"SelectorText","parentSelectors":["Novel Name"],"selector":"dd.rating","multiple":false,"regex":"","delay":0},{"id":"Insta","type":"SelectorLink","parentSelectors":["Author"],"selector":".user-instagram_url a","multiple":false,"delay":0},{"id":"Facebook","type":"SelectorLink","parentSelectors":["Author"],"selector":".user-facebook_url a","multiple":false,"delay":0},{"id":"Twitter","type":"SelectorLink","parentSelectors":["Author"],"selector":".user-twitter_url a","multiple":false,"delay":0},{"id":"Single Link","type":"SelectorLink","parentSelectors":["Author"],"selector":".user-value a","multiple":false,"delay":0},{"id":"Website","type":"SelectorLink","parentSelectors":["Author"],"selector":".user-homepage_url a","multiple":false,"delay":0},{"id":"Author Bio","type":"SelectorText","parentSelectors":["Author"],"selector":"h2.user-description","multiple":false,"regex":"","delay":0},{"id":"Sub Genre","type":"SelectorText","parentSelectors":["Novel Name"],"selector":"dd.genres","multiple":false,"regex":"","delay":0}]}

Hi Faheem956

as far as I know, there no such limit element click feature. It seems you can limit the range Ref: Limiting Infinite Scroll - Advise Please - #2 by Mike_Nisbet

I take another way round by looking at the href of the "Show More" button, there you will see a page number variable where you can tweak it for first 30 pages.

Lego

1 Like

Thanks for your input. Pagination worked but I have noticed a weird issue. If there are multiple Novels by a single author, It scraped only the first one and skipped the rest of the Novels by that same author.

My best guest is after Author, you shall add a Elements selector to scrap the Novels.

Fortunately, the Show More here keeps track of how many "pages" that have been clicked. Many other sites do not. So you can make use of a :Not condition like:

a.genrePage_nextPageButton:not([data-nextpagenum="8"])

This means, click Show More until page 7 (i.e. click as long as nextpagenum is not 8). I suggest you test with a small number first.

The relevant section is:

<a class="genrePage_nextPageButton" data-href="/genres/romance/get_story_list_light?page=3&amp;period=alltime&amp;sort=popular" data-nextpage="" data-nextpagenum="3" data-trackingname="Genre Page Next page" rel="next">
Show more

2 Likes