Products with AND without variation

How to?
#1

Hi,

Is it possible to scrape a page (with infinite scrolling; I've seen the documentation) where some products have variants and others don't?

huda
huda1219×684 120 KB

The behavior I'd need:

  • If there are no variations, click to open the product page and retrieve the information I need.
  • If there are variations, click to open the product page, then click on each variant to retrieve the information I need.

If possible, how exactly do I handle this?

Url: Shop Huda Beauty Makeup – Foundation, Blush, Concealer & More | HUDA BEAUTY

Thank you for your help!

#2

Hi,

There is no need to differentiate between products with or without variations. If a product does not have variants, the variant clicks will be skipped, and the scraper will collect only the data available.

Here is a basic setup to get you started:

{"_id":"hudabeauty","startUrl":["https://hudabeauty.com/us/en_US/huda-beauty"],"selectors":[{"elementLimit":0,"id":"product-wrapper","multiple":true,"parentSelectors":["_root"],"scroll":false,"selector":"div.product-tile","type":"SelectorElement"},{"id":"link","linkType":"linkFromHref","multiple":false,"parentSelectors":["product-wrapper"],"selector":"a[itemprop='url']","type":"SelectorLink"},{"clickActionType":"real","clickElementSelector":".py-2 > div.attribute .color-guide-swatch a","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":1000,"discardInitialElements":"discard-when-click-element-exists","id":"select-color","multiple":true,"parentSelectors":["link"],"selector":"body","type":"SelectorElementClick"},{"id":"title","multiple":false,"multipleType":"singleColumn","parentSelectors":["select-color"],"regex":"","selector":"h1","type":"SelectorText"},{"extractAttribute":"title","id":"color","multiple":false,"multipleType":"singleColumn","parentSelectors":["select-color"],"selector":".product-wrapper .selected [aria-current=\"true\"]","type":"SelectorElementAttribute"}]}