Link scraping not working

How to?
#1

I am having trouble extracting a link from this site: California School Dashboard (CA Dept of Education).

I would like to extract the link to the webpage that is hyperlinked as "View More Details," but when I use the selector tool to select the hyperlink, the URL is not extracted. Hovering over the link, I see that it says javascript:void(0) even though the user is redirected to a new page after clicking on it. I think the javascript:void(0) might be part of the problem but I'm not sure how to get around it. I've uploaded a screenshot of the link I'm referring to.

Thank you for your help!

Screen Shot 2023-08-31 at 12.14.22 PM
Screen Shot 2023-08-31 at 12.14.22 PM538×894 122 KB

#3

You're right that the data that I want is on that page - called the Academic Performance Report - but the problem is that I need to scrape data off 1,200 different versions of the Academic Performance Report for 1,200 different schools. I don't have the direct URLs to the Academic Performance Report for any schools. I only have 1,200 URLs to the California School Dashboard for each school, so I need the Web Scraper tool to navigate to the link to the Academic Performance Report for each school. Are you saying it won't be possible?

#4

You might not need to do any clicking, maybe just a bit of work to generate a list of URLs.

The report page URLs seem to be in this format:

https://www.caschooldashboard.org/reports/xxxxxxxxxx/2022

where xxxxx is a unique number for each school.

You can append some words to each URL to load these three sections directly - Academic Performance, Academic Engagement and Conditions and Climate.

https://www.caschooldashboard.org/reports/xxxxxxxxxx/2022/academic-performance
https://www.caschooldashboard.org/reports/xxxxxxxxxx/2022/academic-engagement
https://www.caschooldashboard.org/reports/xxxxxxxxxx/2022/conditions-and-climate

You can drill further into each subcategory by appending its name, for example, the direct link to Brockton's "English Learner Progress" is

https://www.caschooldashboard.org/reports/19647336016133/2022/academic-performance#english-learner-progress

You can figure out the pattern by observing how the URL changes as you click on each sub-section. It should be the same pattern for all schools:

image
image1493×727 145 KB

Also see: Add suffix/prefix to URLs, or build URLs from scratch

#5

Thank you for looking into this!

I had the same thought. The problem is that I don't know the unique number for each school. I could try to do some research to find out now that I know I won't be able to have Web Scraper access the link from the previous page.

I really appreciate your help!

#7

How does knowing the county help with the unique code? Do you understand how the 14-digit code is derived?

I misspoke earlier when I said I had 1,200 URLs that linked to the California School Dashboard (these URLs contain the unique code, so if I did have them, I wouldn't be in this predicament). I only have 1,200 URLs for each school's directory page. Here is the one for Brockton Elementary: School Page. From there, you can get to the California School Dashboard under "Accountability." I was able to direct the Web Scraper to do this, which is how I got to the dashboard in the first place. But now I'm stuck.