Ajax Pagination - how to start on any page?

Hi, i have an scraper for a site (i am not sharing the URL becase you need an account, but it is free, so if you need it let me know to share it)

It has a pagination with 837 pages, and the scrape works fine, but it stoped on page 179 or 180, so i don't know why it stoped,but i will to try with more time between request. I want to know if it is possible to start on a different page?

My other question, it is a way to know why the scraper stops or a way to debug the issue?

Note: the pagination works with javascript without URL change (the URL is the same always), so, the only way to change the page is editing the javascript on the element.

Thank you very much for your help

You might simply be running out of RAM (too much data). The workaround is to limit your paginator (there is usually a way) and scrape, say, 155 pages at a time. You may need to restart Chrome or even Windows between each session.

To resume on a different page, you could add a custom delay (one-time delay) of say, 30 sec, at the top of your sitemap so you have time to manually click until the intended page appears before the actual scraping starts.

For custom delay, I just use Element Click and pick an inactive part of the page which doesn't do anything if clicked. We're just using its delay feature and not actually activating or scraping anything. With a custom delay, you also do not need to use extremely long delays for Page load delay (ms).

Excellent advice! i will to do some tests, and i will let you know. Thank you very much!