Hey guys, Please help me,
I was trying to scrape a URL with site fairprice -singapore and selecting dettol as product. I see that it shows 113 products. When i scrape it i get only 20 products. Although this page do not have any pagination etc. Can you please guide me where am i going wrong and why only 20 product details are been shown.
url- https://www.fairprice.com.sg/search?query=dettol&filter=brand%3Adettol
Use element scroll down to load all first.
The element scroll selector will scroll down the page until there are no new elements loaded.
Dear wsdc,
Thank you so much for your support and guiding me. I did use the element scroll down option. Only issue I am facing here is that , I am getting 60 records even after several trials not more than that. I had given delay in element scroll to even 5000.
Do you have any suggestion which will help scrapping the required rows of 113
I don't know if there is a more elegant way, but doubling the scroll element down five times is enough for 185 elements.
{"_id":"fairprice","startUrl":["https://www.fairprice.com.sg/search?query=dettol&filter=brand%3Adettol"],"selectors":[{"id":"scroll1","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"main","multiple":false,"delay":2000},{"id":"scroll2","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"main","multiple":false,"delay":2000},{"id":"scroll3","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"main","multiple":false,"delay":2000},{"id":"scroll4","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"main","multiple":false,"delay":2000},{"id":"scroll5","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"main","multiple":false,"delay":2000}]}