Each Product Page has different selector references

How to?
#1

Hi All,

I am experiencing something strange when trying to scrape the product information from this site, in that each product page (although looking similar in layout) seems to have a different selector reference. In the example below I am trying to scrape the Description and Box Contents as HTML. I used IP50HD-TX — Blustream Australia as reference product, but then noticed product IP50HD-RX — Blustream Australia returned NULL, so then added it's selector reference, and then product ACM200 — Blustream Australia returned NULL.

Not sure what I need to do here.

Help please :slight_smile:

Url: https://www.blustream.com.au/

Sitemap:
{"_id":"blustream_aus_description_and_in_the_box","startUrl":["https://www.blustream.com.au/"],"selectors":[{"delay":0,"id":"product_link","multiple":true,"parentSelectors":["wrapper"],"selector":".min-font-set a","type":"SelectorLink"},{"delay":0,"id":"product_wrapper","multiple":false,"parentSelectors":["product_link"],"selector":"section.white","type":"SelectorElement"},{"delay":0,"id":"wrapper","multiple":true,"parentSelectors":["menu"],"selector":"div.sqs-col-4","type":"SelectorElement"},{"delay":0,"id":"menu","multiple":true,"parentSelectors":["_root"],"selector":".header-display-desktop a.header-nav-folder-title, .header-display-desktop .header-nav-folder-item a","type":"SelectorLink"},{"delay":0,"id":"desc_in_the_box_HTML_grouped","multiple":false,"parentSelectors":["product_link"],"regex":"","selector":"div#block-b3b16d82b5578bf9fc73, #block-8513e4030ffa8b77ffab div","type":"SelectorHTML"}]}

#2

@wrighty Hi. These elements are based on the product ID. In order to avoid creating several selector variants, you should manually specify this selector to - div.sqs-block-content:has(h4:contains("Description"))

Learn more:

#3

Thanks @ViestursWS once again, thanks so much for your help :slight_smile:

#4

Hey @ViestursWS just wondering whether you know whether there is a way to scrape video URL's?

Example sitemap
{"_id":"jands-biamp-devio-images","startUrl":["https://www.jands.com.au/brands/biamp/devio","https://www.jands.com.au/brands/biamp/conferencing-solutio","https://www.jands.com.au/brands/biamp/desono","https://www.jands.com.au/brands/biamp/tesira"],"selectors":[{"clickElementSelector":"button.cv-refresh","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickMore","delay":2000,"discardInitialElements":"do-not-discard","id":"product-wrapper","multiple":true,"parentSelectors":["_root"],"selector":"div.product","type":"SelectorElementClick"},{"delay":0,"id":"SKU","multiple":false,"parentSelectors":["product-wrapper"],"regex":"","selector":"span.widget-productlist-code","type":"SelectorText"},{"delay":0,"id":"model-description_URL","multiple":true,"parentSelectors":["product-wrapper"],"selector":".widget-productlist-title a","type":"SelectorLink"},{"clickElementSelector":"a.item-link","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":500,"discardInitialElements":"do-not-discard","id":"productimage-wrapper","multiple":false,"parentSelectors":["model-description_URL"],"selector":".active a.modal-window-alternate, a.item-link, .product-gallery-slider div.owl-dots, section.widget-product-gallery","type":"SelectorElementClick"},{"delay":0,"id":"additioanl-images","multiple":true,"parentSelectors":["productimage-wrapper"],"selector":".active img","type":"SelectorImage"}]}

Trying to scrape the video URL's for pages such as https://www.jands.com.au/brands/biamp/tesira

Also, do you have an email or pm I can contact you on? I am looking to engage someone to set up scraping templates.

Cheers, Ian

#5

@wrighty Hi, it appears that the video source is embedded into an iframe and it can be extracted using an 'Element attribute' selector.

iframe source
iframe source1943×1182 455 KB

Example:

{"_id":"jands-com-au-video-link","startUrl":["https://www.jands.com.au/brands/biamp/tesira"],"selectors":[{"delay":0,"extractAttribute":"src","id":"video-link","multiple":false,"parentSelectors":["_root"],"selector":"div.fluid-width-video-wrapper iframe ","type":"SelectorElementAttribute"}]}