Having trouble with getting all item pages to scrape

#1

Thank you for your help! This has been driving me crazy for over a week.

{"_id":"TruckPro","startUrl":["https://www.truckpro.com/search/butt%20connector/"],"selectors":[{"id":"pagination","parentSelectors":["_root","pagination"],"paginationType":"auto","type":"SelectorPagination","selector":"li:nth-of-type(n+2) a.page-link"},{"id":"element-card","parentSelectors":["pagination"],"type":"SelectorElement","selector":"div.prdListData","multiple":true},{"id":"ProductLink","parentSelectors":["element-card"],"type":"SelectorLink","selector":"a.link","multiple":false,"linkType":"linkFromHref"},{"id":"ProductName","parentSelectors":["ProductLink"],"type":"SelectorText","selector":"span.product-vendor-value","multiple":false,"regex":""},{"id":"Price","parentSelectors":["ProductLink"],"type":"SelectorText","selector":".product-price div.price-wrapper","multiple":false,"regex":""},{"id":"ShortDescription","parentSelectors":["ProductLink"],"type":"SelectorText","selector":"div.product-name","multiple":false,"regex":""},{"id":"Specfications","parentSelectors":["ProductLink"],"type":"SelectorHTML","selector":"div.specValues, h3.heading-product-cross-reference","multiple":false,"regex":""},{"id":"Product Image","parentSelectors":["ProductLink"],"type":"SelectorImage","selector":".image-gallery-image img","multiple":false}]}

Start URL: https://www.truckpro.com/search/butt%20connector/

#2

Hi, the website renders all items only after scrolling to the bottom, thus a combination of scroll and pagination selectors is necessary. Check out the sitemap below and see if it does the trick

{"_id":"TruckPro","startUrl":["https://www.truckpro.com/search/butt%20connector/"],"selectors":[{"delay":2000,"elementLimit":100,"id":"scroll","multiple":true,"parentSelectors":["_root","pagination"],"selector":"div.prdListData","type":"SelectorElementScroll"},{"id":"pagination","paginationType":"auto","parentSelectors":["_root","pagination"],"selector":"[aria-label=\"Next page\"][aria-disabled=\"false\"]","type":"SelectorPagination"},{"id":"ProductLink","linkType":"linkFromAttributes","multiple":true,"parentSelectors":["pagination"],"selector":"[data-bv-show=\"inline_rating\"]","type":"SelectorLink"},{"id":"Price","multiple":false,"parentSelectors":["ProductLink"],"regex":"","selector":".product-price div.price-wrapper","type":"SelectorText"},{"id":"Specfications","multiple":false,"parentSelectors":["ProductLink"],"regex":"","selector":"div.specValues, h3.heading-product-cross-reference","type":"SelectorHTML"}]}
#3

Thank you so much!! This was so helpful! Happy New Year!