For the life of me, I cannot get the scraper to open a link. I have a page with a series of links in cells of a table. Nothing I try will get the scraper to click on a link, let alone click on each one and scrape the page that opens. What am I missing?
Let's start with a question: what is a start URL you are trying to afford ?
This is the site: Touchstone:Accreditation & Assessment Management System - Customer Portal. The first challenge is that you have to enter some search terms to get the table with the links. I used the Website State Setup function, but even though it will demo just fine, it doesn't actually run every time. When it does run, it stops on the first selector. What's interesting is that if you click the link, it will open the desired page. If you copy and paste the link, you get a black page. So that may be another issue. Perhaps this is too complex to scrape?
I am also having an issue saving a sitemap. I start to create the sitemap in one window, but have to navigate to another page to complete the sitemap. Once I do that, it opens a new window and I lose the webscraper. I have to re-open it and resume my build. Then the complete sitemap is only available in that window. If I close out and re-open, my sitemap is lost.
here is your data...
Suppose it's not a big problem for you to scrape all necessary data inside each URL I provided.... ))
Thanks, its amazing to me that you were able to pull the URLs as links. I still don't have a way to scrape it on my own, but I think the lesson here is to simply pay someone who knows how to do this stuff LOL.
1 Like