Firstly, hello fellow scraper. Just hope you can help me out a bit.
I am trying to scrape this website: SGX Investor Portal. What I am trying to do is to click on each of the stocks listed on the left hand side and pull data that loads/shows on the right hand side.
I am using the selectors for scrolling and clicking but it does not seems to work as I intend it to. (it iterates only once?). What I would want it to do is to scrape 570 or so records.
Url: SGX Investor Portal
Sitemap:
{"_id":"SGX_scrape_co_info2","startUrl":["https://investors.sgx.com/securities/stocks"],"selectors":[{"id":"list-details","parentSelectors":["_root"],"type":"SelectorElement","selector":"sgx-list-details","multiple":true},{"id":"title","parentSelectors":["_root"],"type":"SelectorText","selector":"span.widget-security-details-name","multiple":true,"regex":""},{"id":"industry","parentSelectors":["_root"],"type":"SelectorText","selector":"span.widget-security-details-general-industry","multiple":true,"regex":""},{"id":"market","parentSelectors":["_root"],"type":"SelectorText","selector":".prices-metrics-container div:nth-of-type(3) div.cmp-collapsible-two-column-item-value","multiple":true,"regex":""},{"id":"ssic_sector","parentSelectors":["_root"],"type":"SelectorText","selector":".prices-overview-container div:nth-of-type(11) div.cmp-collapsible-two-column-item-value","multiple":true,"regex":""},{"id":"scroll","parentSelectors":["_root"],"type":"SelectorElementScroll","selector":"div.list-view","multiple":true,"delay":500,"elementLimit":5000},{"id":"click first","parentSelectors":["_root"],"type":"SelectorElementClick","clickActionType":"real","clickElementSelector":"cmp-list-row-security:nth-of-type(1)","clickElementUniquenessType":"uniqueText","clickType":"clickOnce","delay":2000,"discardInitialElements":"do-not-discard","multiple":true,"selector":"cmp-list-row-security:nth-of-type(1)"}]}