So I have a page showing all my scores for several maps: https://osu.ppy.sh/u/10198475
I'm trying to list all my top ranks and accuracies.
Tried doing this a couple ways, here's the first try, it clicks through to the second page, but only gets the data for the first. I tried playing with the delays but it doesn't pick it up no matter how high I set it.
{"_id":"osu","startUrl":["https://osu.ppy.sh/u/10198475"],
"selectors":[{"id":"accuracy","type":"SelectorText","selector":"td > div.h","parentSelectors":["showMore"],"multiple":false,"regex":"[0-9]+\.[0-9]+","delay":"0"},{"id":"song","type":"SelectorText","selector":"td > div.h a","parentSelectors":["showMore"],"multiple":false,"regex":"","delay":"0"},{"id":"TopRanks","type":"SelectorElementClick","selector":"div#leader.expanded","parentSelectors":["_root"],"multiple":false,"delay":"2000","clickElementSelector":"td#_leader.sectionHeading","clickType":"clickOnce","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"diff","type":"SelectorText","selector":"td > div.h","parentSelectors":["showMore"],"multiple":false,"regex":"\[[a-zA-Z'., ]+\]","delay":"0"},{"id":"mods","type":"SelectorText","selector":"td > div.h b","parentSelectors":["showMore"],"multiple":false,"regex":" \+[A-Z,]+","delay":"0"},{"id":"link","type":"SelectorElementAttribute","selector":"td > div.h a","parentSelectors":["showMore"],"multiple":false,"extractAttribute":"href","delay":"0"},{"id":"showMore","type":"SelectorElementClick","selector":"div.prof-beatmap","parentSelectors":["TopRanks"],"multiple":true,"delay":"2000","clickElementSelector":"div#more-performance-0 a","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"fullPP","type":"SelectorText","selector":"div.pp-display","parentSelectors":["showMore"],"multiple":false,"regex":"[0-9]+","delay":"0"},{"id":"weight","type":"SelectorText","selector":"div.pp-display-weight","parentSelectors":["showMore"],"multiple":false,"regex":"[0-9]+%","delay":"0"}]}
Here's what I tried after reading some of the posts here, it gets all the info if I click the preview on "showMeMore" but it gives duplicates and when I try to scrape the site, it doesn't click through top ranks.
{"_id":"osu","startUrl":["https://osu.ppy.sh/u/10198475"],
"selectors":[{"id":"accuracy","type":"SelectorText","selector":"td > div.h","parentSelectors":["entry"],"multiple":false,"regex":"[0-9]+\.[0-9]+","delay":"0"},{"id":"song","type":"SelectorText","selector":"td > div.h a","parentSelectors":["entry"],"multiple":false,"regex":"","delay":"0"},{"id":"TopRanks","type":"SelectorElementClick","selector":"div#leader.loaded","parentSelectors":["_root"],"multiple":false,"delay":"1000","clickElementSelector":"td#_leader.sectionHeading","clickType":"clickOnce","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"diff","type":"SelectorText","selector":"td > div.h","parentSelectors":["entry"],"multiple":false,"regex":"\[[a-zA-Z0-9'., ]+\]","delay":"0"},{"id":"mods","type":"SelectorText","selector":"td > div.h b","parentSelectors":["entry"],"multiple":false,"regex":" \+[A-Z,]+","delay":"0"},{"id":"link","type":"SelectorElementAttribute","selector":"td > div.h a","parentSelectors":["entry"],"multiple":false,"extractAttribute":"href","delay":"0"},{"id":"showMeMore","type":"SelectorElementClick","selector":"div#leader.expanded","parentSelectors":["TopRanks","showMeMore"],"multiple":false,"delay":"2000","clickElementSelector":"div#more-performance-0 a","clickType":"clickOnce","discardInitialElements":true,"clickElementUniquenessType":"uniqueText"},{"id":"fullPP","type":"SelectorText","selector":"div.pp-display","parentSelectors":["entry"],"multiple":false,"regex":"[0-9]+","delay":"0"},{"id":"weight","type":"SelectorText","selector":"div.pp-display-weight","parentSelectors":["entry"],"multiple":false,"regex":"[0-9]+%","delay":"0"},{"id":"entry","type":"SelectorElement","selector":"div.prof-beatmap","parentSelectors":["showMeMore"],"multiple":true,"delay":"100"}]}