Describe the problem.
So I am trying to download all mod files on nexusmods and from the sitemap below I set it so I am having trouble going to the next page after completing the first page any fix? This is specifically working for monster hunter world. I know this isn't how you are supposed to use this, but hey it works. xD
Url: nexusmods.com
Sitemap:
{"_id":"nexusmods","startUrl":["https://www.nexusmods.com/"],"selectors":[{"id":"Games","type":"SelectorLink","parentSelectors":["_root"],"selector":"#game-tile-2531 a.mod-image","multiple":true,"delay":0},{"id":"MHWLinks","type":"SelectorLink","parentSelectors":["AllMods"],"selector":"div.tile-desc:nth-of-type(n+2) h3 a","multiple":true,"delay":0},{"id":"Files","type":"SelectorLink","parentSelectors":["MHWLinks"],"selector":".modtabs #mod-page-tab-files a","multiple":false,"delay":0},{"id":"Manual Downloads","type":"SelectorLink","parentSelectors":["Files"],"selector":".open li:nth-of-type(3) a","multiple":true,"delay":0},{"id":"Slow downlaod","type":"SelectorElementClick","parentSelectors":["Manual Downloads"],"selector":"button.rj-btn","multiple":false,"delay":"7000","clickElementSelector":"button.rj-btn","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"AdditionalFilesDownload","type":"SelectorLink","parentSelectors":["Manual Downloads"],"selector":".widget-mod-requirements a.btn","multiple":false,"delay":0},{"id":"Slow Download","type":"SelectorElementClick","parentSelectors":["AdditionalFilesDownload"],"selector":"button.rj-btn","multiple":false,"delay":"7000","clickElementSelector":"button.rj-btn","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"AllMods","type":"SelectorLink","parentSelectors":["Games"],"selector":"#game-mods a.view-all","multiple":false,"delay":0},{"id":"NextPage","type":"SelectorElementClick","parentSelectors":["AllMods"],"selector":".bottom-nav .next a","multiple":true,"delay":2000,"clickElementSelector":".bottom-nav .next a","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"}]}
I fixed the issue to where it crashes when it gets to the click the next page part, however whenever it presses next page it doesn't scrape it, instead it crashes. What I want the "script" to do is to loop one part of the script to another part so whenever it gets done with scraping one page it goes back to the start aka clicking the next page and then starts the scraping process
edit: I have updated the sitemap again, but I can't seem to figure out how to make it click the next arrow when it completes scraping one page + removed some selectors that weren't doing anything.