Pagination Next button won't go past first page? Tried various things

How to?
#1

I can't figure out why the pagination element click with the Next button won't work. I have followed the tutorial, I have tried selecting the button different ways but it won't go through all the pages. It doesn't appear to be a link selector since the page URL doesn't change and there are no page ranges, etc. It only grabs the first page of data in my scrape. Even tried the BETA but that didn't even start the process.

Stanton Carpet

Sitemap:
{"_id":"stantoncarpetcomm","startUrl":["https://www.stantoncarpet.com/commercial-carpet"],"selectors":[{"id":"paginationwrap","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"#content > div > form > div:nth-child(6) > div > div.product-list.row > div","multiple":true,"delay":"900","clickElementSelector":"#btnNext","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"maincollectionspages","type":"SelectorLink","parentSelectors":["paginationwrap"],"selector":"a","multiple":true,"delay":0},{"id":"colorprodpages","type":"SelectorLink","parentSelectors":["maincollectionspages"],"selector":"div.col-sm-6:nth-of-type(n+2) a:nth-of-type(1)","multiple":true,"delay":0},{"id":"construction","type":"SelectorText","parentSelectors":["colorprodpages"],"selector":"li:contains('Construction:')","multiple":false,"regex":"","delay":0},{"id":"colorprodname","type":"SelectorText","parentSelectors":["colorprodpages"],"selector":"li:contains('Color:')","multiple":false,"regex":"","delay":0},{"id":"subcollectionname","type":"SelectorText","parentSelectors":["colorprodpages"],"selector":"li:contains('Collection:')","multiple":false,"regex":"","delay":0},{"id":"patternrepeat","type":"SelectorText","parentSelectors":["colorprodpages"],"selector":"li:contains('Pattern Repeat:')","multiple":false,"regex":"","delay":0},{"id":"yarncontentcategory","type":"SelectorText","parentSelectors":["colorprodpages"],"selector":"li:contains('Fiber:')","multiple":false,"regex":"","delay":0},{"id":"description","type":"SelectorText","parentSelectors":["colorprodpages"],"selector":"div.col-md-9:nth-of-type(2) p:nth-of-type(3)","multiple":false,"regex":"","delay":0},{"id":"image","type":"SelectorImage","parentSelectors":["colorprodpages"],"selector":"img.img-555x555","multiple":false,"delay":0},{"id":"maincollectionname","type":"SelectorText","parentSelectors":["colorprodpages"],"selector":"span.product-name","multiple":false,"regex":"","delay":0}]}

Would appreciate it if someone would take a look and tell me what I did wrong. Thanks.

-Jenny

#2

For this site, you can just use a "manual" method to make it show all items, thus eliminating need for pagination. Basically you use a long page load delay (say 15000, 15 sec) which gives you time to change Items/Page to All.

image

This example sitemap gets all 133 items:

{"_id":"forum-stantoncarpet","startUrl":["https://www.stantoncarpet.com/commercial-carpet"],"selectors":[{"id":"Item wrappers","type":"SelectorElement","parentSelectors":["_root"],"selector":"div.main-content div.col-sm-3","multiple":true},{"id":"Title","type":"SelectorText","parentSelectors":["Item wrappers"],"selector":"div.filter-name-container","multiple":false,"regex":""},{"id":"Link","type":"SelectorLink","parentSelectors":["Item wrappers"],"selector":"div > a","multiple":false}]}

#3

leemeng,
Now that is interesting!! That made all the difference! While I did try with ALL selected, it wasn't from "inside" the scrape window itself and my delay was just the standard. Didn't know adjustments could be made inside the scrape window. Thank you!

I am still curious though how it would be done with the pagination/Next. Do you have insight on what my error was with that setup?

Thanks, again!