Hello everyone, just joined and I am having some issues getting my infinite scroll to work properly on Kijiji Autos. I am able to get the page to scroll, just not in the exact way I would like. When I run the scraper, it scrolls to the bottom of the page like it is supposed to, but then more content is dynamically loaded to the web page and instead of scrolling further, the scraper will shoot to the top of the page and then down to the bottom again and then quits. I am always able to get a few lines of data back so that part is working. I have tried so many different elements in the Scroll selector... some work better than others, but I can never get more than about 50-75 records on any given scrape. There should be somewhere in the neighbourhood of like 20,000 records. If I could even get a quarter of those records, I would be happy. Not sure what I am doing wrong here, but any information, help, or suggestions would be greatly appreciated. Thanks!
Url:
https://www.kijijiautos.ca/cars/#con=USED&od=down&sb=ct&st=DEALER&yc=2000%3A2019
Sitemap:
{"_id":"kijiji","startUrl":["https://www.kijijiautos.ca/cars/#con=USED&od=down&sb=ct&st=DEALER&yc=2000%3A2019"],"selectors":[{"id":"Scroll","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"footer[class="_3DTnFS8D90JekGzuZVLbmt"]","multiple":true,"delay":"5000"},{"id":"Vehicle","type":"SelectorText","parentSelectors":["_root"],"selector":"h2[class="_359WVWY0I2CHgHUvAk-JVF _3okhtoMhqThVhoq94SeapG _38CFNcqLlJdgoyDxNk2Vcf 14Nqyg9Gv-h-nJ_lZalHW"]","multiple":true,"regex":"","delay":0}]}