I have been working on this for a few days now as it will save me a lot of time in the long run. I am trying to get the following info:
Thumbnail images (nearly always successful)
Description from the "Product details" box (almost never successful)
Specific things from the "Specifications" box, such as weight, color, LWH, etc. (almost never successful)
As said in the title, SOMETIMES it will work after I have clicked data preview, but it almost never works when initiating the actual scraping job. My sitemap is linked below, apologies if it is spaghetti, but I am very new to this. Open to any and all criticism. Thank you.
Url:https://www.homedepot.com/p/Hugger-52-in-LED-Indoor-Black-Ceiling-Fan-with-Light-Kit-AL383LED-BK/301162049
Sitemap:
{"_id":"Homed","startUrl":["https://www.homedepot.com/p/Hugger-52-in-LED-Indoor-Black-Ceiling-Fan-with-Light-Kit-AL383LED-BK/301162049"],"selectors":[{"id":"clickimage","paginationType":"clickOnce","parentSelectors":["_root","clickimage"],"selector":"span.mediagallery__thumbnailicons--count","type":"SelectorPagination"},{"id":"thumbnails","multiple":true,"parentSelectors":["clickimage"],"selector":".overlay__side-content__product-images img","type":"SelectorImage"},{"id":"Brand","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"h2.sui-text-base","type":"SelectorText"},{"id":"Title","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"h1","type":"SelectorText"},{"id":"Model","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"div.sui-flex:nth-of-type(2) .sui-font-regular.sui-text-xs span","type":"SelectorText"},{"id":"Finish","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"tr:contains('Color/Finish') p.sui-font-regular","type":"SelectorText"},{"id":"Finish2","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"tr:nth-of-type(15):contains('Housing Color') p.sui-font-regular","type":"SelectorText"},{"id":"Weight","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"tr:contains('Assembled Weight (lbs.)') p.sui-font-regular","type":"SelectorText"},{"id":"Height","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"tr:contains('Assembled Height (in.)') p.sui-font-regular","type":"SelectorText"},{"id":"Depth","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"tr:contains('Assembled Depth (in.)') p.sui-font-regular","type":"SelectorText"},{"id":"Width","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"tr:contains('Assembled Width (in.)') p.sui-font-regular","type":"SelectorText"},{"id":"NumBlades","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"tr:contains('Number of Blades') p.sui-font-regular","type":"SelectorText"},{"id":"Desc","multiple":true,"parentSelectors":["_root"],"regex":"","selector":".sui-gap-4 .sui-gap-2 p, .sui-w-full.sui-list-disc li:nth-of-type(1), .sui-w-full li:nth-of-type(n+2) span","type":"SelectorText"},{"id":"UPC","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"body.upc","type":"SelectorText"}]}