Ecommerce Pagination for books

How to?
#1

Describe the problem. The sitemap only scrapes the page 1, which are books 1-20, but will not scrape the remaining 29 books that are on page 2 and 3. I've watched and tried all of the solutions that are available without success.

Any help would be amazing!

Url: http://barnesandnoble.com

Sitemap:
{"id":"kids_paperbacks_test","startUrl":["https://www.barnesandnoble.com/b/books/kids//N-1z141tjZ29Z8q8Ztu1?Nrpp=40&page=1"],"selectors":[{"delay":0,"id":"subject_link","multiple":true,"parentSelectors":["_root"],"selector":"#refinements #sidebar-section-0 a","type":"SelectorLink"},{"delay":0,"id":"topic_link","multiple":true,"parentSelectors":["subject_link"],"selector":"#refinements #sidebar-section-0 a","type":"SelectorLink"},{"id":"pagination","paginationType":"linkFromHref","parentSelectors":["topic_link","pagination"],"selector":"img.full-shadow","type":"SelectorPagination"},{"delay":0,"id":"book links","multiple":true,"parentSelectors":["pagination"],"selector":"a.pImageLink","type":"SelectorLink"},{"delay":0,"id":"title","multiple":false,"parentSelectors":["book links"],"regex":"","selector":"h1","type":"SelectorText"}]}

#2

@Daryl77 Hi. Can you show the full path of how you accessed this page? As it seems to be session based(the link is valid only for the computer which has the cookie information stored in its browser).

#3

@Daryl77 Anyway, after inspecting the book listing pages it seems that the best way to paginate through them is by using a 'link' selector - a.next-button:not([aria-disabled="true"])

Sitemap example(don't forget to change the starting URL):

{"_id":"barnesandnoble-com-kids","startUrl":["https://www.barnesandnoble.com/b/books/kids/_/N-1fZ29Z8q8Ztu1?Nrpp=20&page=1"],"selectors":[{"delay":0,"id":"pagination","multiple":true,"parentSelectors":["_root","pagination"],"selector":"a.next-button:not([aria-disabled=\"true\"])","type":"SelectorLink"},{"delay":0,"id":"book-link","multiple":true,"parentSelectors":["_root","pagination"],"selector":".product-info-title a","type":"SelectorLink"}]}

Hope it helps.

#4

Thanks @ViestursWS this did the trick! Is there a way, using the cloud webscraper, to get the url for each of the book images?

#5

Hi @Daryl77

Yes, you can use the "Grouped" selector - div#prodImage div.slick-track div.slick-slide img with an ''Attribute name" - src.