Problem:
When trying to download the images of the sitemap biocbd_de, I get the following error (please see the "CONSOLE OUTPUT"). I suspect it is a character in the image URL that creates a problem in the python code?
Further question for clarification:
Can I adapt the python code in image-downloader.py or give it an argument in the commad line / terminal to only download images that are of the "type":"SelectorImage" or even better: only the images in the CSV column "image_primary_link-src"? Because otherwise, if the code downloads all images in all rows it would download images, which I don't need.
[CONSOLE OUTPUT START]
(base) [my path to dropbox]\Web Scraper\biocbd_de>image-downloader.py 00001154763-2019-03-17-biocbd_de.csv
INFO:executed by python 3
INFO:importing data from 00001154763-2019-03-17-biocbd_de.csv
INFO:downloading image https://lk8on2kca0e1row121qgjym9-wpengine.netdna-ssl.com/wp-content/uploads/2018/05/image-1-10.jpg
INFO:downloading image https://lk8on2kca0e1row121qgjym9-wpengine.netdna-ssl.com/wp-content/uploads/2018/05/Gold.jpg
INFO:downloading image https://lk8on2kca0e1row121qgjym9-wpengine.netdna-ssl.com/wp-content/uploads/2018/05/WEB-Mock-up_BOTTLE_BOX_SPECTRUM_HANF10_resize.jpg
INFO:downloading image https://lk8on2kca0e1row121qgjym9-wpengine.netdna-ssl.com/wp-content/uploads/2019/02/WEB-Mock-up_BOTTLE_BOX_SPECTRUM_HANF5_resize.jpg
INFO:downloading image https://lk8on2kca0e1row121qgjym9-wpengine.netdna-ssl.com/wp-content/uploads/2019/03/WEB-Mock-up_BOTTLE_BOX_SPECTRUM_HANF_PFEFFERMINZE_resize.jpg
Traceback (most recent call last):
File "C:\DATA\DK3_Speicher\Dropbox\Dropbox\Apps\Web Scraper\biocbd_de\image-downloader.py", line 131, in
main(sys.argv)
File "C:\DATA\DK3_Speicher\Dropbox\Dropbox\Apps\Web Scraper\biocbd_de\image-downloader.py", line 123, in main
download_csv_file_images(csv_filename)
File "C:\DATA\DK3_Speicher\Dropbox\Dropbox\Apps\Web Scraper\biocbd_de\image-downloader.py", line 115, in download_csv_file_images
for row in csvreader:
File "C:\Users\ta_user\Miniconda3\lib\csv.py", line 112, in next
row = next(self.reader)
File "C:\Users\ta_user\Miniconda3\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 526: character maps to
[CONSOLE OUTPUT END]
Environment Info
Image Downloader v0.1.1.
Python v 3.7.0
Url: https://biocbd.de/product-category/cbd-oele/
Sitemap:
{"_id":"biocbd_de","startUrl":["https://biocbd.de/product-category/cbd-oele/"],"selectors":[{"id":"product_link","type":"SelectorLink","parentSelectors":["_root"],"selector":"div.product-info a.woocommerce-LoopProduct-link","multiple":true,"delay":0},{"id":"title","type":"SelectorText","parentSelectors":["product_link"],"selector":"h1.product_title","multiple":false,"regex":"","delay":0},{"id":"price","type":"SelectorText","parentSelectors":["product_link"],"selector":"div.summary span.woocommerce-Price-amount","multiple":false,"regex":"","delay":0},{"id":"short_summary","type":"SelectorText","parentSelectors":["product_link"],"selector":"div.summary div.description p","multiple":false,"regex":"","delay":0},{"id":"detail_text","type":"SelectorHTML","parentSelectors":["product_link"],"selector":"div.woocommerce-Tabs-panel.woocommerce-Tabs-panel--description","multiple":false,"regex":"","delay":0},{"id":"rating","type":"SelectorText","parentSelectors":["product_link"],"selector":"div.woocommerce-product-rating div.star-rating strong.rating","multiple":false,"regex":"","delay":0},{"id":"rating_n","type":"SelectorText","parentSelectors":["product_link"],"selector":"div.woocommerce-product-rating div.star-rating span.rating","multiple":false,"regex":"","delay":0},{"id":"product_option_standard_volume","type":"SelectorText","parentSelectors":["product_link"],"selector":"div.summary a.mtt-variation.mtt-var-selected","multiple":false,"regex":"","delay":0},{"id":"image_primary_link","type":"SelectorImage","parentSelectors":["product_link"],"selector":"div.mtt-wc-gallery div.mtt-wc-gallery-pic img","multiple":false,"delay":0}]}
Thanks in advance!