How do I know what websites allow this extension to scrape its contents? Not looking for any legal trouble. Any ideas?
For example here is the website I'm looking to scrape from:
I read somewhere that this application acts as a user and therefore technically isn't web scraping so this wouldn't violate redfin terms of service. How can I know what the robots.txt allows about scraping?