I need to scrap data from a web application which requires login. Is there a way I can post the login information in the sitemap and proceed further with scraping.
On most modern websites, login is handled via sessions (cookies) or tokens so you would only need to login to the website before you start the scraping task (before launching WS). Once you start scraping, the website should "remember" that you have already logged in, and should no longer show a login prompt.
If this doesn't work, you can try using Chrome's Autofill feature (remember name and password). Then you only need to click the "Login" or "Continue" button.
Does this work with the cloud?
For me it's working when i run ws in the browser, but i want to know if it also works in the cloud scraper version.
Hi,
Please open a trial account and contact support with this inquiry. In most cases the cloud solution will be able to execute a login, but there are exceptions.