Newbie here. I am working on a project that will scrape indeed job postings. I started with the code posted at ScrapeHero however I am having issue getting the pagination to stop after a certain number of pages. I’ve tried editing the “next” selector element based on some other posts on here but no luck (nth-child(-n+200)). The profession I am scraping has over 10000 results so I want to limit the number of scrapes results. Anyone have any suggestions using the above code?
The url does change for each new page on Indeed, so you can just use the Specify multiple urls with ranges method. See "Use range url with increment".