If you click on each of the products in the URL below, you'll see that they display prices differently.
In some cases, discounts are applied, and in other cases, the price is just displayed.
In this case, I have a lot of products that don't work when I try to scrape the price as text.
So I've created a script with a number of different cases, but it doesn't work.
Am I doing something wrong?
{"_id":"Amazon_Scraper_240127","startUrl":["https://www.amazon.com/s?k=Boating+Equipment&i=sporting&rh=n%3A3375251%2Cp_85%3A2470955011%2Cp_72%3A1248957011%2Cp_36%3A1500-5000%2Cp_n_condition-type%3A6503254011&s=exact-aware-popularity-rank&dc&page=[1-380]&crid=HSY8J8BMLS7G&qid=1706962962&rnid=386589011&sprefix=%2Caps%2C609"],"selectors":[{"id":"single","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root"],"selector":".s-widget-container > span .a-size-mini a","type":"SelectorLink"},{"id":"title","multiple":false,"parentSelectors":["single"],"regex":"","selector":"span.a-size-large","type":"SelectorText"},{"id":"price","multiple":false,"parentSelectors":["single"],"regex":"[0-9]+\.[0-9]+","selector":".a-accordion-active div.a-spacing-top-mini","type":"SelectorText"},{"id":"image1","multiple":false,"parentSelectors":["single"],"regex":"{"hiRes":"(.*),","selector":"body","type":"SelectorHTML"},{"id":"brand","multiple":false,"parentSelectors":["single"],"regex":"","selector":".po-brand span.po-break-word","type":"SelectorText"},{"id":"description","multiple":false,"parentSelectors":["single"],"regex":"","selector":"div#productDescription","type":"SelectorHTML"},{"id":"short_description","multiple":false,"parentSelectors":["single"],"regex":"","selector":"div.a-spacing-medium.a-spacing-top-small","type":"SelectorHTML"},{"id":"asin","multiple":false,"parentSelectors":["single"],"regex":"","selector":"li:nth-of-type(6) .a-list-item span:nth-of-type(2)","type":"SelectorText"},{"id":"categories","multiple":false,"parentSelectors":["single"],"regex":"","selector":"#wayfinding-breadcrumbs_container","type":"SelectorText"},{"id":"price2","multiple":false,"parentSelectors":["single"],"regex":"[0-9]+\.[0-9]+","selector":"#apex_desktop_newAccordionRow div.a-spacing-none","type":"SelectorText"},{"id":"price3","multiple":false,"parentSelectors":["single"],"regex":"[0-9]+\.[0-9]+","selector":"#apex_desktop_newAccordionRow td.a-span12.a-color-secondary","type":"SelectorText"},{"id":"price4","multiple":false,"parentSelectors":["single"],"regex":"[0-9]+\.[0-9]+","selector":"#sns-base-price div.a-column","type":"SelectorText"},{"id":"price5","multiple":false,"parentSelectors":["single"],"regex":"[0-9]+\.[0-9]+","selector":"div#apex_desktop_newAccordionRow","type":"SelectorText"}]}
Hi, could you please post the sitemap as Preformatted text?
{"_id":"Amazon_Scraper_240127","startUrl":["https://www.amazon.com/s?k=Boating+Equipment&i=sporting&rh=n%3A3375251%2Cp_85%3A2470955011%2Cp_72%3A1248957011%2Cp_36%3A1500-5000%2Cp_n_condition-type%3A6503254011&s=exact-aware-popularity-rank&dc&page=[1-380]&crid=HSY8J8BMLS7G&qid=1706962962&rnid=386589011&sprefix=%2Caps%2C609"],"selectors":[{"id":"single","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root"],"selector":".s-widget-container > span .a-size-mini a","type":"SelectorLink"},{"id":"title","multiple":false,"parentSelectors":["single"],"regex":"","selector":"span.a-size-large","type":"SelectorText"},{"id":"price","multiple":false,"parentSelectors":["single"],"regex":"[0-9]+\.[0-9]+","selector":".a-accordion-active div.a-spacing-top-mini","type":"SelectorText"},{"id":"image1","multiple":false,"parentSelectors":["single"],"regex":"{"hiRes":"(.*),","selector":"body","type":"SelectorHTML"},{"id":"brand","multiple":false,"parentSelectors":["single"],"regex":"","selector":".po-brand span.po-break-word","type":"SelectorText"},{"id":"description","multiple":false,"parentSelectors":["single"],"regex":"","selector":"div#productDescription","type":"SelectorHTML"},{"id":"short_description","multiple":false,"parentSelectors":["single"],"regex":"","selector":"div.a-spacing-medium.a-spacing-top-small","type":"SelectorHTML"},{"id":"asin","multiple":false,"parentSelectors":["single"],"regex":"","selector":"li:nth-of-type(6) .a-list-item span:nth-of-type(2)","type":"SelectorText"},{"id":"categories","multiple":false,"parentSelectors":["single"],"regex":"","selector":"#wayfinding-breadcrumbs_container","type":"SelectorText"},{"id":"price2","multiple":false,"parentSelectors":["single"],"regex":"[0-9]+\.[0-9]+","selector":"#apex_desktop_newAccordionRow div.a-spacing-none","type":"SelectorText"},{"id":"price3","multiple":false,"parentSelectors":["single"],"regex":"[0-9]+\.[0-9]+","selector":"#apex_desktop_newAccordionRow td.a-span12.a-color-secondary","type":"SelectorText"},{"id":"price4","multiple":false,"parentSelectors":["single"],"regex":"[0-9]+\.[0-9]+","selector":"#sns-base-price div.a-column","type":"SelectorText"},{"id":"price5","multiple":false,"parentSelectors":["single"],"regex":"[0-9]+\.[0-9]+","selector":"div#apex_desktop_newAccordionRow","type":"SelectorText"}]}
Hi, the JSON code is still not valid. Did you copy the code directly from the WebScraper extension?
{"_id":"Amazon_Scraper_240127","startUrl":["https://www.amazon.com/s?k=coral+blush&i=beauty&rh=n%3A3760911%2Cp_85%3A2470955011%2Cp_72%3A1248873011%2Cp_36%3A1000-&s=relevancerank&dc&page=[1-11]&crid=3SVN9ERYT81TK&qid=1707318884&rnid=386662011&sprefix=coral+blush%2Cbeauty%2C316&ref=sr_pg_2"],"selectors":[{"id":"single","parentSelectors":["_root"],"type":"SelectorLink","selector":".s-widget-container > span .a-size-mini a","multiple":true,"linkType":"linkFromHref"},{"id":"title","parentSelectors":["single"],"type":"SelectorText","selector":"span.a-size-large","multiple":false,"regex":""},{"id":"price","parentSelectors":["single"],"type":"SelectorText","selector":"#corePrice_feature_div > div > div > span.a-price.a-text-price.header-price.a-size-base.a-text-normal","multiple":false,"regex":""},{"id":"image1","parentSelectors":["single"],"type":"SelectorHTML","selector":"body","multiple":false,"regex":"{\"hiRes\":\"(.*),"},{"id":"brand","parentSelectors":["single"],"type":"SelectorText","selector":".po-brand span.po-break-word","multiple":false,"regex":""},{"id":"description","parentSelectors":["single"],"type":"SelectorHTML","selector":"div#productDescription","multiple":false,"regex":""},{"id":"short_description","parentSelectors":["single"],"type":"SelectorHTML","selector":"div.a-spacing-medium.a-spacing-top-small","multiple":false,"regex":""},{"id":"asin","parentSelectors":["single"],"type":"SelectorText","selector":"li:nth-of-type(6) .a-list-item span:nth-of-type(2)","multiple":false,"regex":""},{"id":"categories","parentSelectors":["single"],"type":"SelectorText","selector":"#wayfinding-breadcrumbs_container","multiple":false,"regex":""},{"id":"price2","parentSelectors":["single"],"type":"SelectorText","selector":"#corePrice_desktop > div > table > tbody > tr > td.a-span12 > span.a-price.a-text-price.a-size-medium.apexPriceToPay","multiple":false,"regex":""},{"id":"price3","parentSelectors":["single"],"type":"SelectorText","selector":"#sns-base-price > div > span.a-price.a-text-price.a-size-medium.apexPriceToPay","multiple":false,"regex":""},{"id":"price4","parentSelectors":["single"],"type":"SelectorText","selector":"#sns-base-price > div > div > span.a-price.a-text-price.header-price.a-size-base.a-text-normal","multiple":false,"regex":""},{"id":"price5","parentSelectors":["single"],"type":"SelectorText","selector":"#corePrice_desktop > div > table > tbody > tr:nth-child(2) > td.a-span12 > span.a-price.a-text-price.a-size-medium.apexPriceToPay","multiple":false,"regex":""},{"id":"price6","parentSelectors":["single"],"type":"SelectorText","selector":"#corePriceDisplay_desktop_feature_div > div.a-section.a-spacing-none.aok-align-center.aok-relative > span.a-price.aok-align-center.reinventPricePriceToPayMargin.priceToPay","multiple":false,"regex":""},{"id":"price7","parentSelectors":["single"],"type":"SelectorText","selector":"#corePrice_feature_div > div > div > div > div > span.a-price.a-text-normal.aok-align-center.reinventPriceAccordionT2","multiple":false,"regex":""},{"id":"price8","parentSelectors":["single"],"type":"SelectorText","selector":"#corePriceDisplay_desktop_feature_div > div.a-section.a-spacing-none.aok-align-center.aok-relative > span.a-price.aok-align-center.reinventPricePriceToPayMargin.priceToPay","multiple":false,"regex":""},{"id":"price9","parentSelectors":["single"],"type":"SelectorText","selector":"#corePrice_feature_div > div > div > span.a-price.aok-align-center","multiple":false,"regex":""}]}
Hi, have you tried scraping the price from the listing wrapper?
{"_id":"amazon-price","startUrl":["https://www.amazon.com/s?k=coral+blush&i=beauty&rh=n%3A3760911%2Cp_85%3A2470955011%2Cp_72%3A1248873011%2Cp_36%3A1000-&s=relevancerank&dc&page=[1-11]&crid=3SVN9ERYT81TK&qid=1707318884&rnid=386662011&sprefix=coral+blush%2Cbeauty%2C316&ref=sr_pg_2"],"selectors":[{"id":"product-wrapper","multiple":true,"parentSelectors":["_root"],"selector":"[data-component-type=\"s-search-result\"]","type":"SelectorElement"},{"id":"price","multiple":false,"parentSelectors":["product-wrapper"],"regex":"","selector":".a-price .a-offscreen","type":"SelectorText"}]}