Popup Image Gallery Problem

How to?
#1

The scraper follows all the links & pops up the image galleries, however only one row is returned & images are "null". I just realized that the gallery images are thumbnails leading to an image carousel.

Image Popup
Image Popup1586×126 25.1 KB

Url:

Sitemap:
{"_id":"music_shop_yelp_test","startUrl":["https://www.yelp.co.uk/search?find_desc=music+shop&find_loc=London"],"selectors":[{"id":"links","parentSelectors":["_root"],"type":"SelectorLink","selector":".css-1egxyvc a","multiple":true},{"id":"name","parentSelectors":["links"],"type":"SelectorText","selector":"h1","multiple":false,"regex":""},{"id":"address","parentSelectors":["links"],"type":"SelectorText","selector":"p.css-qyp8bo","multiple":false,"regex":""},{"id":"image_popup","parentSelectors":["links"],"type":"SelectorPopupLink","selector":"a.css-psnyb4","multiple":false},{"id":"image","parentSelectors":["image_popup"],"type":"SelectorImage","selector":".css-13kx8xl img","multiple":false}]}

#2

@Ajak Hello, in order to scrape the desired data you can use an 'Element click' selector in a combination with 'Grouped' selector.

Also, a template sitemap for this website is available within the 'Community sitemaps' section in Web Scraper Cloud: Web Scraper

Screenshot from 2022-10-27 15-30-28
Screenshot from 2022-10-27 15-30-282195×782 57.2 KB

Sitemap example:

{"_id":"yelp-businesses","startUrl":["https://www.yelp.co.uk/biz/fopp-london?osq=music+shop"],"selectors":[{"id":"business-page-wrapper","multiple":true,"parentSelectors":["_root"],"selector":"html:has(div[data-hypernova-key*=\"BizDetails\"]):has(h1)","type":"SelectorElement"},{"clickElementSelector":"div[class*=\"photo-header-content-container\"] a:contains(\"photos\")","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":2000,"discardInitialElements":"discard-when-click-element-exists","id":"photo-click","multiple":true,"parentSelectors":["business-page-wrapper"],"selector":"div[data-testid=\"page\"]","type":"SelectorElementClick"},{"extractAttribute":"src","id":"images","parentSelectors":["business-page-wrapper"],"selector":"div[data-testid=\"thumbnail\"] img ","type":"SelectorGroup"}]}

#3

Thanks for the site-map, I added link selectors to it & retrieved images from sites on the first page. I plan to import the results into web directory listings using CSV & WP All-Import. Problem is there are too many images, one site has 40 images. Is there a way to limit the results for example, to 6 images per site?

#4

@Ajak Hi, yes, it is possible to limit the number of extractable images to 6 by using the jQuery selector ':nth-of-type', more specifically - div[data-testid="page"] > div > div:nth-of-type(-n+6) div[data-testid="thumbnail"] img

Learn more: Selectors | jQuery API Documentation

#5

I modified the selector to: div:nth-of-type(-n+6) div[data-testid="thumbnail"] img but no change, maybe I need to modify the nth-of-type formula?