Can't Scrape Website where Start URL doesn't load data

How to?
#1

I am trying to create a basic scrape for the member directory at this site: https://www.cetainternational.org/ceta-member-directory#/

In an excel, I am trying to grab the person's name, email, cellphone, and organization. However, there is an issue that the starting URL doesn't actually load the results (I have to manually click Search). Even when I delay the loading time to 5 seconds for me to manually hit search, it stops after scraping 3-4 pages of the 22.

Can anybody help me figure out why this is happening? How can I fix this?

#2

@xfong15 Hello, it appears that it is possible to scrape the desired data by targeting each of the available states(a separate sitemap would have to be created for each state).

Here's a sitemap example for the state of Alabama:

{"_id":"cetainternational-org-alabama","startUrl":["https://www.cetainternational.org/ceta-member-directory#/"],"selectors":[{"clickElementSelector":"select[id=\"5-field\"] option:contains(\"Alabama\")","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":1200,"discardInitialElements":"do-not-discard","id":"alabama-click","multiple":true,"parentSelectors":["_root"],"selector":"body","type":"SelectorElementClick"},{"clickElementSelector":"a[ng-click=\"vm.search()\"]","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":1200,"discardInitialElements":"do-not-discard","id":"search-click","multiple":true,"parentSelectors":["_root"],"selector":"body","type":"SelectorElementClick"},{"id":"wrapper","multiple":true,"parentSelectors":["_root"],"selector":"div.search-profile","type":"SelectorElement"},{"id":"name","multiple":false,"parentSelectors":["wrapper"],"regex":"","selector":"strong span.ng-scope","type":"SelectorText"},{"id":"city","multiple":false,"parentSelectors":["wrapper"],"regex":"","selector":".ds-top div:contains(\"City\")","type":"SelectorText"},{"id":"state","multiple":false,"parentSelectors":["wrapper"],"regex":"","selector":".ds-top div:contains(\"State\")","type":"SelectorText"}]}