Hi there!
The website URL provide very useful data — page number, so you can use pagination brackets [ ... ]
instead of clicking Next
.
All you have to analyze is number of pages to scrape, it's just 172.
You have to envelop starting and ending page within brackets, separated by a minus sign.
Please note that the beginning page has a 0 as a page number, so starting from 0 to 171 equals 172 pages.
Try this sitemap:
{"_id":"san_pablo","startUrl":["https://www.farmaciasanpablo.com.mx/medicamentos/c/06?q=%3Aname-asc&page=[0-171]"],"selectors":[{"id":"Item_grouped","type":"SelectorElement","selector":"div.col-md-9 div.col-xs-12","parentSelectors":["_root"],"multiple":true,"delay":0},{"id":"Name","type":"SelectorText","selector":"p.item-title","parentSelectors":["Item_grouped"],"multiple":false,"regex":"","delay":0},{"id":"Dose","type":"SelectorText","selector":"p.item-subtitle","parentSelectors":["Item_grouped"],"multiple":false,"regex":"","delay":0},{"id":"Price","type":"SelectorText","selector":"p.item-prize","parentSelectors":["Item_grouped"],"multiple":false,"regex":"","delay":0},{"id":"Out of Stock","type":"SelectorElementAttribute","selector":"button.btn","parentSelectors":["Item_grouped"],"multiple":false,"extractAttribute":"disabled","delay":0}]}
To have results being strictly ordered, I would recommend you to use CouchDB.
For CouchDB, please refer to this page: https://webscraper.io/documentation#storage-backends