Scrapping multiple starting pages and multiple profiles with load more buttton

How to?
#1

Describe the problem.

Hi everybody,

I am stuck on a scraping. I would like to scrap all this pages from this website: https://signal.nfx.com/

To do that I put all these url in the metadata as start urls.

My process was the following to get all the infos:

  1. I need to scrap all the profile rows but there is "load more" button so I create a selector element click.
  2. I keep my category by creating a text selector.
  3. I create an investor profile with selector link to enter in every investor profile
  4. I select the date I want inside the profile page.

Datas I want:

  • Category
  • Investor Name
  • Investor Company
  • Investor Job
  • Current investing position
  • Investing range
  • sweet spot
  • Investings on record
  • Current fund size
  • sector & stage
  • Company Website
  • Linkedin url
  • Twitter url
  • Angellist url
  • Crunchbase url

But it doesn't work and I don't understand why. After some hours in front of the tutorials videos I abdicate. If there is a nice person to help, it would be super nice. Thanks a lot !

Sitemap:
{"_id":"investor_startups","startUrl":["https://signal.nfx.com/investor-lists/top-fintech-seed-investors","https://signal.nfx.com/investor-lists/top-crypto-web3-pre-seed-investors","https://signal.nfx.com/investor-lists/top-creator-passion-economy-pre-seed-investors","https://signal.nfx.com/investor-lists/top-consumer-internet-pre-seed-investors","https://signal.nfx.com/investor-lists/top-gig-economy-pre-seed-investors","https://signal.nfx.com/investor-lists/top-fintech-pre-seed-investors","https://signal.nfx.com/investor-lists/top-web3-blockchain-pre-seed-investors","https://signal.nfx.com/investor-lists/top-social-networks-pre-seed-investors","https://signal.nfx.com/investor-lists/top-fintech-seed-investors"],"selectors":[{"id":"Investor_rows","parentSelectors":["_root"],"type":"SelectorElementClick","clickElementSelector":"button.sn-center","clickElementUniquenessType":"uniqueText","clickType":"clickMore","delay":500,"discardInitialElements":"do-not-discard","multiple":true,"selector":"tbody tr"},{"id":"investor_name","parentSelectors":["Investor_rows"],"type":"SelectorText","selector":"strong","multiple":false,"delay":0,"regex":""},{"id":"investor_company","parentSelectors":["Investor_rows"],"type":"SelectorText","selector":".sn-investor-name-wrapper > a","multiple":false,"delay":0,"regex":""},{"id":"investor_job","parentSelectors":["Investor_rows"],"type":"SelectorText","selector":"span.sn-small-link","multiple":false,"delay":0,"regex":""},{"id":"investor_sweetspot","parentSelectors":["Investor_rows"],"type":"SelectorText","selector":"div.flex-column div:nth-of-type(1)","multiple":false,"delay":0,"regex":""},{"id":"investor_range","parentSelectors":["Investor_rows"],"type":"SelectorText","selector":".flex-column div:nth-of-type(2)","multiple":false,"delay":0,"regex":""},{"id":"investor_location","parentSelectors":["Investor_rows"],"type":"SelectorText","selector":"td:nth-of-type(4) .sn-clamp div","multiple":false,"delay":0,"regex":""},{"id":"investor_investment_categories","parentSelectors":["Investor_rows"],"type":"SelectorText","selector":"td:nth-of-type(5) .sn-clamp div","multiple":false,"delay":0,"regex":""},{"id":"investor_link","parentSelectors":["Investor_rows"],"type":"SelectorLink","selector":".sn-investor-name-wrapper .flex a","multiple":false,"delay":0},{"id":"company_website","parentSelectors":["investor_link"],"type":"SelectorElementAttribute","selector":"a.iconlink:nth-of-type(1)","multiple":false,"delay":0,"extractAttribute":"href"},{"id":"investor_linkedin","parentSelectors":["investor_link"],"type":"SelectorElementAttribute","selector":"a.iconlink:nth-of-type(2)","multiple":false,"delay":0,"extractAttribute":"href"},{"id":"investor_twitter","parentSelectors":["investor_link"],"type":"SelectorElementAttribute","selector":"a.iconlink:nth-of-type(3)","multiple":false,"delay":0,"extractAttribute":"href"},{"id":"investor_angellist","parentSelectors":["investor_link"],"type":"SelectorElementAttribute","selector":"a.iconlink:nth-of-type(4)","multiple":false,"delay":0,"extractAttribute":"href"},{"id":"investor_crunchbase","parentSelectors":["investor_link"],"type":"SelectorElementAttribute","selector":"a.iconlink:nth-of-type(5)","multiple":false,"delay":0,"extractAttribute":"href"}]}