36 Result Limit... Impossible

[cwmacker]

The site returns 18 results per page, there are 49 pages in this example with a total of 882 results. The final scrape only gets 36. I get all the data as expected from each result I get.

I have tried increasing the 2000 to 5000 and get the same result.

Url: Classic Wholesalers

Sitemap:
{"_id":"classic_top_sellers","startUrl":["https://www.classicwholesalers.co.za/top-sellers/"],"selectors":[{"id":"product_listings","parentSelectors":["_root","pages"],"type":"SelectorLink","selector":"a.text-decoration-none","multiple":true},{"id":"item_no","parentSelectors":["product_listings"],"type":"SelectorText","selector":"tr:contains('Item No.') td","multiple":false,"regex":""},{"id":"description","parentSelectors":["product_listings"],"type":"SelectorText","selector":"tr:contains('Description') td","multiple":false,"regex":""},{"id":"jhb_stock","parentSelectors":["product_listings"],"type":"SelectorText","selector":"tr:contains('JHB Stock') td","multiple":false,"regex":""},{"id":"price","parentSelectors":["product_listings"],"type":"SelectorText","selector":"tr:contains('Price') td","multiple":false,"regex":""},{"id":"min_pack_size","parentSelectors":["product_listings"],"type":"SelectorText","selector":"tr:contains('Min Pack Size') td","multiple":false,"regex":""},{"id":"pages","parentSelectors":["_root"],"type":"SelectorLink","selector":"a.page-link","multiple":true}]}

[ViestursWS]

@cwmacker Hi, that's because your 'pagination' setup is incorrect. To fix that you should make the 'pages' selector a 'parent' to itself as well.

[cwmacker]

Thank you so much for getting back to me. Im sorry but I'm a real newby at this. Is there anyway you might be able to tell me or show me how to do that please?

Kind regards

[ViestursWS]

@cwmacker Hi. Click on 'Edit' button for the 'pages' selector, navigate to the 'Parent Selectors' section, hold the CTRL button and press on the 'pages', click 'Save selector' and start scraping.

[cwmacker]

Thank you. So I just did this and when I start the scrape it just cycles from page 1 to page 48 without actually scraping any data.

[cwmacker]

Hey there, so actually practising some patience, your solution worked. It just cycled through to the last page first then it goes into each product page to grad the data. Thank you for the help