Hi, I'm new to webscraping and looking for some help.
I'm looking to scrape the information in the table at the URL below, which I can do. However, I want to filter by each sport in the right-hand filter column, and click "Show Results", and scrape each sport individually. Would anyone be able to assist me? I have no idea how to set up the scraper to click through the dropdown as described, and the tutorials don't seem to show that either.
Thanks in Advance!
Url: https://bucscore.bucs.org.uk/BUCSPoints.aspx
This sitemap exports the data I want on the first table, but I don't know how to apply it to each table in the dropdown as described above.
Sitemap:
{"_id":"bucs","startUrl":["https://bucscore.bucs.org.uk/bucspoints.aspx?season=26"],"selectors":[{"id":"table data","type":"SelectorTable","parentSelectors":["_root"],"selector":"table#pointsTable","multiple":true,"columns":[{"header":"University Points:","name":"University Points:","extract":true},{"header":"League \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\tPredicted Points","name":"League \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\tPredicted Points","extract":false},{"header":"Cup\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\tMinimum Points","name":"Cup\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\tMinimum Points","extract":false},{"header":"Indiv","name":"Indiv","extract":false},{"header":"Total","name":"Total","extract":true},{"header":"Compare","name":"Compare","extract":false}],"delay":0,"tableDataRowSelector":"tbody tr","tableHeaderRowSelector":"thead tr"}]}