I want to scrape the product page info (price, size, etc). I have tried everything to get it to iterate through all the option and not click on the "not available" size options.
If I add a :not() to the size selector to exclude not available options then it wont iterate through the sizes
If I dont add a "not" to the size selector then it doesn't retrieve all possible size and flavor combos
I have tried every selector:not() combo i can think of and nothing will work.
Sadly, Being able to data scraping chewy.com is the best thing to happen to me work wise this year and now i'm worried its lost forever. Please help!
Url: ORIJEN Original Grain-Free Dry Cat Food, 12-lb bag - Chewy.com
Sitemap: can get it to iterate through flavor but not size
{"_id":"chewy_flavor_and_size_test","startUrl":["Chewy.com Radio","multiple":true,"parentSelectors":["Flavor Radio"],"selector":"parent","type":"SelectorElementClick"},{"clickElementSelector":"[data-testid='sku-selector-choice-ensemble'] input","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":1000,"discardInitialElements":"discard-when-click-element-exists","id":"Flavor Radio","multiple":true,"parentSelectors":["_root"],"selector":"body","type":"SelectorElementClick"},{"id":"Size","multiple":false,"parentSelectors":["Size Radio"],"regex":"","selector":"div.kib-chip-choice--selected[data-testid='sku-selector-choice']","type":"SelectorText"},{"id":"title","multiple":false,"parentSelectors":["Size Radio"],"regex":"","selector":"h1","type":"SelectorText"},{"id":"flavor","multiple":false,"parentSelectors":["Size Radio"],"regex":"","selector":"div.kib-chip-choice--selected[data-testid='sku-selector-choice-ensemble']","type":"SelectorText"},{"id":"Price","multiple":false,"parentSelectors":["Size Radio"],"regex":"","selector":".n5u5RNp-e2RYq6wqk3Q9y div","type":"SelectorText"}]}