Hello. I'm trying to scrape contact information from a main webpage with a list of all the orgnizations. Each organization is a subpage that I need to go into to scrape the information I need.
The main page starts off showing 10(n) organizations, then you need to click "load more" to show an additional n+10 organizations (up to 235 organizations total).
I have used the "element click" --> "click more" click type to click the "load more" button until the full list of organizations are shown on the main page. After this, the scraper starts scraping each subpage.
THE PROBLEM: web scraper only scraps the first 10 organization's sub-pages in backwards order. The remaining 225 organizations are simply ignored. How can I either instruct the web scraper to start all the way at the bottom (after the load mores are all done), or to instruct it to go from top all the way to the bottom instead?
THANK YOU
Url: - NUSync
Sitemap:
{"_id":"test4","startUrl":["- NUSync org link","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root"],"selector":".MuiList-root a","type":"SelectorLink"},{"clickActionType":"real","clickElementSelector":"button div div","clickElementUniquenessType":"uniqueText","clickType":"clickMore","delay":2000,"discardInitialElements":"do-not-discard","id":"load more","multiple":true,"parentSelectors":["_root","load more"],"selector":"div:nth-of-type(n+234) span > div > div","type":"SelectorElementClick"},{"id":"title","multiple":false,"parentSelectors":["individual org link"],"regex":"","selector":"h1","type":"SelectorText"},{"id":"email","multiple":false,"parentSelectors":["individual org link"],"regex":"","selector":"div.MuiGrid-root:nth-of-type(2) div div div div div:nth-of-type(2)","type":"SelectorText"},{"id":"website","linkType":"linkFromHref","multiple":false,"parentSelectors":["individual org link"],"selector":"a[aria-label='Visit our site https://aiesec.org.sg/']","type":"SelectorLink"},{"id":"insta","linkType":"linkFromHref","multiple":false,"parentSelectors":["individual org link"],"selector":"a[aria-label='Visit our instagram']","type":"SelectorLink"},{"id":"linkedin","linkType":"linkFromHref","multiple":false,"parentSelectors":["individual org link"],"selector":"span.fa-linkedin-square","type":"SelectorLink"}]}