Hello.
I am new to scraping and have tried scraping this site, where I use Selector Scroll Down to identify the listings on the overview page then I use SelectorLink to go into individual product listing to scrape the data.
However, the issue that I am facing right now is that some of the individual product listings links show up as "null" and as a result, there is no individual product details scraped from that link. "Null" appears in 40% of the data scraped.
The sitemap is here:
{"_id":"shopee-drills-2","startUrl":["https://shopee.sg/search?keyword=drill&page=[1]&sortBy=relevancy"],"selectors":[{"id":"ElementScroll","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"div._1T9dHf","multiple":true,"delay":"5000"},{"id":"Link","type":"SelectorLink","parentSelectors":["ElementScroll"],"selector":"a","multiple":true,"delay":"2000"},{"id":"Product_Name","type":"SelectorText","parentSelectors":["Link"],"selector":"div.qaNIZv","multiple":false,"regex":"","delay":0},{"id":"Price","type":"SelectorText","parentSelectors":["Link"],"selector":"div._3n5NQx","multiple":false,"regex":"","delay":0},{"id":"Ratings Number","type":"SelectorText","parentSelectors":["Link"],"selector":"div.flex-auto div.flex div.flex:nth-of-type(2) div._3Oj5_n","multiple":false,"regex":"","delay":0},{"id":"Sold_Monthly","type":"SelectorText","parentSelectors":["Link"],"selector":"div._22sp0A","multiple":false,"regex":"","delay":0},{"id":"Distributor","type":"SelectorText","parentSelectors":["Link"],"selector":"div._3Lybjn","multiple":false,"regex":"","delay":0},{"id":"Distributor_Ratings","type":"SelectorText","parentSelectors":["Link"],"selector":"div.TuJk3S:nth-of-type(1) div.-Gna22 span._1rsHot","multiple":false,"regex":"","delay":0},{"id":"Response_Rate","type":"SelectorText","parentSelectors":["Link"],"selector":"div.TuJk3S:nth-of-type(2) span._1rsHot.OuQDPE","multiple":false,"regex":"","delay":0},{"id":"Distributor_Number of Products","type":"SelectorText","parentSelectors":["Link"],"selector":"a.F6kLFJ span._1rsHot","multiple":false,"regex":"","delay":0},{"id":"Distributor_Follower","type":"SelectorText","parentSelectors":["Link"],"selector":"div.TuJk3S:nth-of-type(3) div.-Gna22:nth-of-type(2) span._1rsHot","multiple":false,"regex":"","delay":0}]}
Perhaps a kind soul can look through the map and advice, I will deeply appreciate it! Thank you!