Hi, I have made a sitemap to pull product information from a website. The data previews always show data and when I run the scrape some of the selector data is missing for a record while others work fine. I adjusted the interval and timeout values a bit but it still seems to randomly miss data. I am not sure if it's actually "randomly" missing data or if there's a reason why maybe specific ones don't get the data.
I manually checked a product where the info data was missing and all the data preview was populated with data for that product, so i don't know why it wouldn't get the data when scraping.
Url: https://www.danmurphys.com.au/dm/home
Sitemap:
{"_id":"danmurphys-product-info","startUrl":["https://www.danmurphys.com.au/dm/home"],"selectors":[{"id":"linkCategory","type":"SelectorLink","parentSelectors":["_root"],"selector":".col-md-4 a","multiple":true,"delay":0},{"id":"linkViewAll","type":"SelectorLink","parentSelectors":["linkCategory"],"selector":"a.col-white","multiple":false,"delay":0},{"id":"linkPagination","type":"SelectorLink","parentSelectors":["linkViewAll","linkPagination"],"selector":"a.anchor-text:nth-of-type(2)","multiple":false,"delay":0},{"id":"linkProductURL","type":"SelectorElementAttribute","parentSelectors":["linkViewAll","linkPagination"],"selector":"shop-product-card.ng-star-inserted .not-offers a, .row > li .not-offers a","multiple":true,"extractAttribute":"href","delay":0},{"id":"productBrand","type":"SelectorText","parentSelectors":["linkProductPage"],"selector":"span[itemprop='brand']","multiple":false,"regex":"","delay":0},{"id":"productName","type":"SelectorText","parentSelectors":["linkProductPage"],"selector":"span[itemprop='name']","multiple":false,"regex":"","delay":0},{"id":"elementDetails","type":"SelectorElement","parentSelectors":["linkProductPage"],"selector":"ul.attribute-list","multiple":false,"delay":0},{"id":"productInfo","type":"SelectorGroup","parentSelectors":["elementDetails"],"selector":"li","delay":0,"extractAttribute":""},{"id":"linkProductPage","type":"SelectorLink","parentSelectors":["linkViewAll","linkPagination"],"selector":"shop-product-card.ng-star-inserted .not-offers a, .row > li .not-offers a","multiple":true,"delay":0}]}