Load/Show More issue or wrong selectors issue?

How to?
#1

I'm stuck trying to determine if I have an issue with the setup of my Show More pagination or if it is my selectors in general for this site?

https://www.crossleyax.com/collections-1

I want it to go to each collection, then each product within each collection (this is where pagination starts), scrape the model/name, image, and the link to pdf for each product.

It seems to run through it but in the end produces no results.

Sitemap:
{"_id":"crossaxtwo","startUrl":["https://www.crossleyax.com/collections-1"],"selectors":[{"id":"collections","type":"SelectorLink","parentSelectors":["_root"],"selector":"div._1Cfot:nth-of-type(n+3) a._2edl5","multiple":true,"delay":0},{"id":"prodwrap","type":"SelectorElementClick","parentSelectors":["collections"],"selector":"._3Zx3f _3irO4","multiple":true,"delay":2000,"clickElementSelector":"button._3we0w","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"prod","type":"SelectorText","parentSelectors":["prodpage"],"selector":"h3","multiple":false,"regex":"","delay":0},{"id":"image","type":"SelectorImage","parentSelectors":["prodpage"],"selector":"._2fhDq img","multiple":false,"delay":0},{"id":"prodpage","type":"SelectorPopupLink","parentSelectors":["collections","prodwrap"],"selector":"div:nth-of-type(n+9) div._1-08X","multiple":true,"delay":0},{"id":"prodpdflink","type":"SelectorLink","parentSelectors":["prodpage"],"selector":"a._1lLL9","multiple":false,"delay":0}]}

Can anyone help me figure out why it doesn't scrape?

Thanks for any suggestions.

#2

Hi @JennyK

The pagination idea would never work because the loop is endless. I'd rather open the modal one by one, by using element-click selector.

My example:

{"_id":"crossaxtwo","startUrl":["https://www.crossleyax.com/collections-1"],"selectors":[{"id":"collections","type":"SelectorLink","parentSelectors":["_root"],"selector":"div#SITE_PAGES div[data-testid=\"inline-content\"] div[data-testid=\"mesh-container-content\"] a","multiple":true,"delay":0},{"id":"wrapper-click-more","type":"SelectorElementClick","parentSelectors":["collections"],"selector":"body","multiple":true,"delay":"1000","clickElementSelector":"button[data-testid=\"matrix-gallery-show-more-button\"]","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"click-open","type":"SelectorElementClick","parentSelectors":["collections"],"selector":"body","multiple":true,"delay":"1000","clickElementSelector":"div[data-testid=\"gallery-item-title\"]","clickType":"clickOnce","discardInitialElements":"discard","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"product-link","type":"SelectorElementAttribute","parentSelectors":["click-open"],"selector":"h3[itemprop=\"name\"] + a","multiple":false,"extractAttribute":"href","delay":0},{"id":"product-name","type":"SelectorText","parentSelectors":["click-open"],"selector":"h3","multiple":false,"regex":"","delay":0},{"id":"product-image","type":"SelectorElementAttribute","parentSelectors":["click-open"],"selector":"div#imageZoomComp > div > wix-image > img","multiple":false,"extractAttribute":"src","delay":0},{"id":"close-click","type":"SelectorElementClick","parentSelectors":["click-open"],"selector":"_parent_","multiple":false,"delay":"1200","clickElementSelector":"div[data-testid=\"close\"]","clickType":"clickOnce","discardInitialElements":"discard","clickElementUniquenessType":"uniqueCSSSelector"}]}

Hope it helps!

#3

viesturs,
Thank you, this worked. Now I have to look at it in depth to understand how it did what it did.