Pagination not working but should be straightforward

How to?
#1

Trying to get pagination working on what should be something straightforward but it's proving problematic - tried various pagination options including auto

{"_id":"musgrove-pag","startUrl":["https://www.musgravemarketplace.ie/home"],"selectors":[{"clickElementSelector":"span.dropdown__item:nth(0) a","clickElementUniquenessType":"uniqueText","clickType":"clickOnce","delay":500,"discardInitialElements":"do-not-discard","id":"Click","multiple":false,"parentSelectors":["_root"],"selector":"body","type":"SelectorElementClick"},{"id":"Category","multiple":true,"parentSelectors":["_root"],"selector":"li.main-navigation-level1-item > span > a[href]:contains('Ambient')","type":"SelectorLink"},{"id":"Sub-category","multiple":true,"parentSelectors":["Category"],"selector":".filter-item .filter-item a","type":"SelectorLink"},{"id":"Matrix-pages","paginationType":"clickOnce","parentSelectors":["Sub-category","Matrix-pages"],"selector":".product-list-paging li.ng-star-inserted:nth-of-type(n+2) a","type":"SelectorPagination"},{"id":"Product-matrix","multiple":true,"parentSelectors":["Matrix-pages"],"selector":"a.product-tile__title","type":"SelectorLink"}]}
#2

@beau Have you also tried using 'Link' type pagination - li.pagination-list-next a?

#3

Not yet - I'll try that. I did try element click but that didn't work

#4

Yes, that worked - thanks again :slight_smile:

Sometimes the easiest solution evades me - I keep looking for the complex solutions LOL

#5

OK, just ran the scraper and it did start to scrape the pages but it is not scraping the final page - for example on

It scrapes page 1 and 2 but does not scrape page 3
Is there anything that you can see that would prevent the pagination selector from grabbing this as the element we are using appears on that page.
Maybe I need to try and extend the page load time as the site is quite slow?

#6

Extended the page load time to 20,000mS (20 seconds) and it still skips page 3 - but if I manually check the Data Preview on page 2 it shows me the page 3 link at the top. Just weird.

{"_id":"musgrave","startUrl":["https://www.musgravemarketplace.ie/home"],"selectors":[{"clickElementSelector":"span.dropdown__item:nth(0) a","clickElementUniquenessType":"uniqueText","clickType":"clickOnce","delay":500,"discardInitialElements":"do-not-discard","id":"Click","multiple":false,"parentSelectors":["_root"],"selector":"body","type":"SelectorElementClick"},{"id":"Category","multiple":true,"parentSelectors":["_root"],"selector":"li.main-navigation-level1-item > span > a[href]:not(:contains('Wine'))","type":"SelectorLink"},{"id":"Sub-category","multiple":true,"parentSelectors":["Category"],"selector":".filters-panel__container > ish-category-navigation > div > div > ul > li > ish-category-navigation > div > div > ul > li > a","type":"SelectorLink"},{"id":"Matrix-pages","paginationType":"linkFromHref","parentSelectors":["Sub-category","Matrix-pages"],"selector":"li.pagination-list-next a","type":"SelectorPagination"},{"id":"Product-matrix","multiple":true,"parentSelectors":["Matrix-pages"],"selector":"a.product-tile__title","type":"SelectorLink"},{"id":"Product-name","multiple":false,"parentSelectors":["Product-matrix"],"regex":"","selector":"span[itemprop='name']","type":"SelectorText"},{"id":"Product-SKU","multiple":false,"parentSelectors":["Product-matrix"],"regex":"","selector":".col-xs-12 span[itemprop='sku']","type":"SelectorText"},{"id":"Product-size","multiple":false,"parentSelectors":["Product-matrix"],"regex":"","selector":"span.mr-3","type":"SelectorText"},{"id":"Product-price","multiple":false,"parentSelectors":["Product-matrix"],"regex":"","selector":".price-container[itemprop='offers'] div.current-price","type":"SelectorText"},{"id":"Product-VAT","multiple":false,"parentSelectors":["Product-matrix"],"regex":"","selector":".product-id[ishvatable] span","type":"SelectorText"},{"id":"Product-image","multiple":false,"parentSelectors":["Product-matrix"],"selector":".product-main-image img","type":"SelectorImage"},{"id":"Product-description-html","multiple":false,"parentSelectors":["Product-matrix"],"regex":"","selector":"div.mat-expansion-panel-body","type":"SelectorHTML"},{"id":"Product-information","multiple":false,"parentSelectors":["Product-matrix"],"regex":"","selector":"div [class*=\"attribute__group ng-tns-c6\"]","type":"SelectorText"},{"id":"Product-stock","multiple":false,"parentSelectors":["Product-matrix"],"regex":"","selector":".stock-message span","type":"SelectorText"},{"id":"Product-sale-price","multiple":false,"parentSelectors":["Product-matrix"],"regex":"","selector":".price-container[itemprop='offers'] span.sale-price","type":"SelectorText"}]}
#7

OK, this is completely wierd

Installed the webscraper extension into Firefox and copied over the sitemap and it picks up all the pages. However Chrome refuses to get the last page.

So it's kinda fixed in that I can scrape the site but no idea why Chrome isn't working :slight_smile:

#8

Forget last post - it managed to get all 3 pages of Pets in FF but then failed on the next category, only selecting the first page of 5

Back to the drawing board :frowning: