Trouble scraping a site

I hope you're all doing well. I'm currently working on a web scraping project, and I'm encountering some difficulties in building the scraper. I was wondering if anyone could lend a hand and provide some guidance or assistance.

-Starting URL:
-Select "Development applications" radio button and click "Next"
-Click "lodgment date" tab, select "last 30 days" radio button, and click "search"
-Retrieve and return all results across multiple pages

@DataDigger Hi, it appears that this website stores the previously performed navigation steps within the browser cookies.

In order to scrape the desired data, make sure to manually replicate the steps you demonstrated in the video and launch the scrape starting from the 'application details' page.


{"_id":"test-2","startUrl":["[1-10]"],"selectors":[{"id":"wrapper","multiple":true,"parentSelectors":["_root"],"selector":".ContentPanel tr:nth-of-type(n+2)","type":"SelectorElement"},{"id":"app-nr","multiple":false,"parentSelectors":["wrapper"],"regex":"","selector":"td:nth-of-type(1)","type":"SelectorText"},{"id":"date","multiple":false,"parentSelectors":["wrapper"],"regex":"","selector":"td:nth-of-type(2)","type":"SelectorText"},{"id":"app-desc","multiple":false,"parentSelectors":["wrapper"],"regex":"","selector":"td:nth-of-type(3)","type":"SelectorText"},{"id":"property-address","multiple":false,"parentSelectors":["wrapper"],"regex":"","selector":"td:nth-of-type(4)","type":"SelectorText"},{"id":"approved","multiple":false,"parentSelectors":["wrapper"],"regex":"","selector":"td:nth-of-type(5)","type":"SelectorText"}]}

The website comes up with an error when you type it in the address bar directly.
The plan is to setup a monthly scheduled scrape? Is this possible or will i always need to navigate to the application details page manually?

@DataDigger It will not be possible as the pagination, in this case, is based on the JavaScript 'DoPostBack' function execution which unfortunately, currently, the scraper can't handle.

Ok thanks for your help