Hello everyone!
So, I have already scrapped through crowdcube's website (crowdfunding platform) and obtained the year of the funding, the amount raised and the name of the company for all the companies available.
However, now I am trying to obtain the location of those same companies. For that, the process is the same as I used before:
- Scroll all the way down
- Click on the link of each company and copy the needed info.
For the location the scrapper needs to click on the google maps link and then copy the address of each company. However, the scrapper stops (not accessing the links) after scrolling all the way to the end. I have played a bit with the selector of the ID "Company". Currently, it is "section" and with that I am able to make the scrapper scroll all the way done, but it does not access the links of the companies. If instead of having "section", as a selector for that ID, I pick only the title of each box, the scrapper does go to the link of each company and retrieves the address of each company, however it does not scroll all the way down.
So, basically I can never get all I want. Either it scrolls all the way done and does not retrieve the address or it does but not of all the companies.
Url: https://www.crowdcube.com/companies?fund_type=equity&funded_date=&raise_amount=&search=§or=
Sitemap:
{"_id":"crowdcube_location","startUrl":["https://www.crowdcube.com/companies?fund_type=equity&funded_date=&raise_amount=&search=§or="],"selectors":[{"id":"Company","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"section","multiple":true,"delay":"10000","clickElementSelector":"button.cc-pagination__load","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"Link","type":"SelectorLink","parentSelectors":["_root","Company"],"selector":"a.cc-card__link","multiple":true,"delay":0},{"id":"Company_Name","type":"SelectorText","parentSelectors":["Link"],"selector":"h1","multiple":false,"regex":"","delay":0},{"id":"Location_Link","type":"SelectorLink","parentSelectors":["Link"],"selector":".cc-company__location a","multiple":false,"delay":0},{"id":"Company_Country","type":"SelectorText","parentSelectors":["Location_Link"],"selector":"div.section-hero-header-title-description","multiple":false,"regex":"","delay":0}]}
Thank you a lot for your help!
Tiago Lopes