Element Click doesn't seem to be working?

How to?
#1

Im trying to extract multiple images from this site.. I've tried many things, but I cant figure out how to get element click to work. I set the click selector to the "next button," and the selector to the area around the image. I clicked on multiple, and discard initial elements. Then I went inside that selector and created an image selector for the current image.

I've been struggling with this feature for a while now. I've found workarounds before, but I can't figure it out this time. It'd be nice to know how to use this selector!

Thank you

Url: https://www.horizonfitness.com/ex-59-elliptical

Sitemap:
{"_id":"horizon","startUrl":["https://www.horizonfitness.com/"],"selectors":[{"id":"Category","type":"SelectorLink","parentSelectors":["_root"],"selector":"a.f4","multiple":true,"delay":0},{"id":"Info","type":"SelectorLink","parentSelectors":["Category"],"selector":"a.expanded","multiple":true,"delay":0},{"id":"name","type":"SelectorText","parentSelectors":["Info"],"selector":"article.product-details h1","multiple":false,"regex":"","delay":0},{"id":"Short Description","type":"SelectorHTML","parentSelectors":["Info"],"selector":"div.product-copy","multiple":false,"regex":"","delay":0},{"id":"Price","type":"SelectorText","parentSelectors":["Info"],"selector":"span.featured-price__sale-price span:nth-of-type(2)","multiple":false,"regex":"","delay":0},{"id":"Specifications","type":"SelectorHTML","parentSelectors":["Info"],"selector":"section.specs-tab","multiple":false,"regex":"","delay":0},{"id":"Image","type":"SelectorElementClick","parentSelectors":["Info"],"selector":"article.product-details div.small-12.medium-order-1","multiple":true,"delay":0,"clickElementSelector":"button.slick-next","clickType":"clickOnce","discardInitialElements":true,"clickElementUniquenessType":"uniqueText"},{"id":"image","type":"SelectorImage","parentSelectors":["Image"],"selector":"div.slick-initialized:nth-of-type(2) div.slick-slide.slick-current img","multiple":true,"delay":0}]}

#2

I updated the sitemap, but it still doesn't scrape more than 2 images.

Any help would be sincerely appreciated

{"_id":"horizon","startUrl":["https://www.horizonfitness.com/"],"selectors":[{"id":"Category","type":"SelectorLink","parentSelectors":["_root"],"selector":"a.f4","multiple":true,"delay":0},{"id":"Info","type":"SelectorLink","parentSelectors":["Category"],"selector":"a.expanded","multiple":true,"delay":0},{"id":"name","type":"SelectorText","parentSelectors":["Info"],"selector":"article.product-details h1","multiple":false,"regex":"","delay":0},{"id":"Short Description","type":"SelectorHTML","parentSelectors":["Info"],"selector":"div.product-copy","multiple":false,"regex":"","delay":0},{"id":"Price","type":"SelectorText","parentSelectors":["Info"],"selector":"span.featured-price__sale-price span:nth-of-type(2)","multiple":false,"regex":"","delay":0},{"id":"Specifications","type":"SelectorHTML","parentSelectors":["Info"],"selector":"section.specs-tab","multiple":false,"regex":"","delay":0},{"id":"Image","type":"SelectorElementClick","parentSelectors":["Info"],"selector":"article.product-details","multiple":true,"delay":0,"clickElementSelector":"button.slick-next","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueHTML"},{"id":"image","type":"SelectorImage","parentSelectors":["Image"],"selector":"div.slick-slide.slick-current img","multiple":false,"delay":0}]}

#3

Hello there!

In order to get all images you must first analize HTML scructure that images are being held within (and whether images are loaded already or won't load unless you click on them). You can simply use Browser's built-in Inspect Tool, to find out the parent container for all images, in your case, it's:

div.product-slidey-container.

Next you can create an Element Attribute selector and set it's selection to div.product-slidey-container img
to pick all images inside it, and set src as an attribute to extract, not forgetting to set Multiple option in it.

Your updated sitemap:

{"_id":"horizon-test","startUrl":["https://www.horizonfitness.com/"],"selectors":[{"id":"Category","type":"SelectorLink","parentSelectors":["_root"],"selector":"a.f4","multiple":true,"delay":0},{"id":"Info","type":"SelectorLink","parentSelectors":["Category"],"selector":"a.expanded","multiple":true,"delay":0},{"id":"name","type":"SelectorText","parentSelectors":["Info"],"selector":"article.product-details h1","multiple":false,"regex":"","delay":0},{"id":"Short Description","type":"SelectorHTML","parentSelectors":["Info"],"selector":"div.product-copy","multiple":false,"regex":"","delay":0},{"id":"Price","type":"SelectorText","parentSelectors":["Info"],"selector":"span.featured-price__sale-price span:nth-of-type(2)","multiple":false,"regex":"","delay":0},{"id":"Specifications","type":"SelectorHTML","parentSelectors":["Info"],"selector":"section.specs-tab","multiple":false,"regex":"","delay":0},{"id":"Image","type":"SelectorElementAttribute","parentSelectors":["Info"],"selector":"div.product-slidey-container img","multiple":true,"extractAttribute":"src","delay":0},{"id":"image","type":"SelectorImage","parentSelectors":["Image"],"selector":"div.slick-slide.slick-current img","multiple":false,"delay":0}]}
#4

Thanks so much for the reply! I used the updated sitemap, but it appears that the scraper is creating new rows for every image, the goal is to just add the images onto the existing row for the product. Instead it's duplicating every column, except for the image column, which is different. I hope this makes sense.

Is there any way to fix this?

#5

You can do it by creating extra selectors that select certain image by it's type, using `:nth-of-type(#)' selector.

I've updated your sitemap, but I did not create selectors for all images and left it for you to understand the logic and practice:

{"_id":"horizon-test2","startUrl":["https://www.horizonfitness.com/"],"selectors":[{"id":"Category","type":"SelectorLink","parentSelectors":["_root"],"selector":"a.f4","multiple":true,"delay":0},{"id":"Info","type":"SelectorLink","parentSelectors":["Category"],"selector":"a.expanded","multiple":true,"delay":0},{"id":"name","type":"SelectorText","parentSelectors":["Info"],"selector":"article.product-details h1","multiple":false,"regex":"","delay":0},{"id":"Short Description","type":"SelectorHTML","parentSelectors":["Info"],"selector":"div.product-copy","multiple":false,"regex":"","delay":0},{"id":"Price","type":"SelectorText","parentSelectors":["Info"],"selector":"span.featured-price__sale-price span:nth-of-type(2)","multiple":false,"regex":"","delay":0},{"id":"Specifications","type":"SelectorHTML","parentSelectors":["Info"],"selector":"section.specs-tab","multiple":false,"regex":"","delay":0},{"id":"img1","type":"SelectorElementAttribute","parentSelectors":["Info"],"selector":"div.slick-slide:nth-of-type(1) img","multiple":false,"extractAttribute":"src","delay":0},{"id":"img2","type":"SelectorElementAttribute","parentSelectors":["Info"],"selector":"div.slick-slide:nth-of-type(2) img","multiple":false,"extractAttribute":"src","delay":0},{"id":"img3","type":"SelectorElementAttribute","parentSelectors":["Info"],"selector":"div.slick-slide:nth-of-type(3) img","multiple":false,"extractAttribute":"src","delay":0},{"id":"img4","type":"SelectorElementAttribute","parentSelectors":["Info"],"selector":"div.slick-slide:nth-of-type(4) img","multiple":false,"extractAttribute":"src","delay":0},{"id":"img5","type":"SelectorElementAttribute","parentSelectors":["Info"],"selector":"div.slick-slide:nth-of-type(5) img","multiple":false,"extractAttribute":"src","delay":0},{"id":"img6","type":"SelectorElementAttribute","parentSelectors":["Info"],"selector":"div.slick-slide:nth-of-type(6) img","multiple":false,"extractAttribute":"src","delay":0},{"id":"img7","type":"SelectorElementAttribute","parentSelectors":["Info"],"selector":"div.slick-slide:nth-of-type(7) img","multiple":false,"extractAttribute":"src","delay":0},{"id":"img8","type":"SelectorElementAttribute","parentSelectors":["Info"],"selector":"div.slick-slide:nth-of-type(8) img","multiple":false,"extractAttribute":"src","delay":0}]}
1 Like