I'm trying to scrape "https://www.autodoc.co.uk/" but not even able to get the home page. I'm using scrapy splash and scrapy_cloudflare_middleware to bypass cloudflare and render javascript. Anyone have any tips to scrape this website? I want to avoid selenium as this website is huge and selenium will take a lot of time
Just delay the load time and do it manually once.
Tried doing that but its not working.