Morning all
Firstly, a big thanks for the Web Scrapping program...sublime work!!
Am trying to Scrape Uni course data from:
University League Tables 2020
Our league tables rank UK universities both nationally and in 70 subjects
and have the following set up:
{"_id":"meagain","startUrl":["[https://www.thecompleteuniversityguide.co.uk/league-tables/rankings#allSubjects"],"selectors":{"id":"viewcourse","type":"SelectorLink","parentSelectors":["_root"],"selector":"a.vw_crlnk[data-ga-label='University of Cambridge']","multiple":false,"delay":0},{"id":"all links","type":"SelectorLink","parentSelectors":["viewcourse"],"selector":".desk_pg li:nth-of-type(n+2) a","multiple":true,"delay":0},{"id":"courseslinks","type":"SelectorLink","parentSelectors":["all links"],"selector":"a[data-provider-index-pos]","multiple":true,"delay":0},{"id":"course","type":"SelectorText","parentSelectors":["courseslinks"],"selector":"h1","multiple":false,"regex":"","delay":0},{"id":"ucas","type":"SelectorText","parentSelectors":["courseslinks"],"selector":"div.cdshd1","multiple":false,"regex":"","delay":0},{"id":"study","type":"SelectorText","parentSelectors":["courseslinks"],"selector":".crs_det li:nth-of-type(3) p","multiple":false,"regex":"","delay":0},{"id":"start","type":"SelectorText","parentSelectors":["courseslinks"],"selector":".crs_det li:nth-of-type(4) p","multiple":false,"regex":"","delay":0},{"id":"summary","type":"SelectorText","parentSelectors":["courseslinks"],"selector":".smrdes p:nth-of-type(1)","multiple":false,"regex":"","delay":0},{"id":"deadline","type":"SelectorText","parentSelectors":["courseslinks"],"selector":"p.apdt","multiple":false,"regex":"","delay":0},{"id":"entry","type":"SelectorText","parentSelectors":["courseslinks"],"selector":"p.ucas_pt","multiple":false,"regex":"","delay":0},{"id":"entry details","type":"SelectorText","parentSelectors":["courseslinks"],"selector":"p.lvl_lst","multiple":false,"regex":"","delay":0},{"id":"address","type":"SelectorText","parentSelectors":["courseslinks"],"selector":".undetl li:nth-of-type(2) p","multiple":false,"regex":"","delay":0}]}
for some reason, it wont scrape the data from the first page :
but will return the following pages:
https://www.thecompleteuniversityguide.co.uk/courses/university-search/undergraduate/all/university-of-cambridge?pg=2
https://www.thecompleteuniversityguide.co.uk/courses/university-search/undergraduate/all/university-of-cambridge?pg=3
https://www.thecompleteuniversityguide.co.uk/courses/university-search/undergraduate/all/university-of-cambridge?pg=4
as I intend to scrape other uni details that may have 70+ pages...I really dont want to grab them individually!
Any help, very much appreciated
Thanks again!!!