I want to do the same as in the video: Web Scraper intro tutorial - YouTube (4:25 min) while at the same time having dynamic pagination: Web Scraper pagination tutorial - YouTube. I tried doing so, but my code doesnt seem to work as I would like it to be working. For this method I would have to visit all the links of the products from the 1st page, after that return to first page and click the 2nd page link, right? and that is why it can never work.
I could at maximum scrape only the 1st page after which it continues repeatedly scraping the same 1st page, but for that I used the paginaltion range method [0-250:25] which of course didnt work for all pages as they are dynamic.
Furthermore, what I tried to do is visit the 1st product link and from there to go to the next product by pointing an arrow and meanwhile selecting the data I need. I tried doing that, but the webpage stopped loading after the 3rd product.
The page is private and that is why its harder to for somebody to help me with it.
What method is there for me to try and make it work?
Thanks in advance for your help!