Hi,
I am trying to scrape from below website. But I can only extract first page.
Page style:
Page 1 of 99+
For this website, you can just append the page number, &page=xxx to the URL to load the page directly. So you can use the "range url with increment" method listed under Specify multiple urls with ranges
You can also control the number of items shown per page (16, 32. 64) with &show=xx
For example, this will load page 126 with 32 items:
https://www.rexelusa.com/s/industrial-control-automation?cat=wri9hn&show=32&page=126
You will need to do a bit of math to find out the max page you want to stop at (total items needed / items per page).
you have to change you start URL to (for example):
https://www.rexelusa.com/s/industrial-control-automation?cat=wri9hn&show=64&page=[1-50] to scrape pages 1 through 50.... If you want to change this range - you can do it by changing the number of pages....