How to: Get 100% of results from scraper on LinkedIn Search

How to?
#1

Hello! I'm using Web Scraper to export all links from a search on LinkedIn but the crawler is not getting all the links on pages, for example:

  1. The term "ERP Solução empresarial" had 143 results (results = number of companies): https://www.linkedin.com/search/results/companies/?keywords=ERP%20Solução%20empresarial&origin=GLOBAL_SEARCH_HEADER&page=[1-15]
  2. When I run the scrape I got just 73 results (usually 50% of the results).

And it's happening for all terms.

My sitemap is: {"_id":"linkedin_patrick_novo","startUrl":["https://www.linkedin.com/search/results/companies/?keywords=ERP%20Solução%20empresarial&origin=GLOBAL_SEARCH_HEADER&page=[1-14]"],"selectors":[{"id":"linkedin_link2","type":"SelectorLink","parentSelectors":["_root"],"selector":".search-result__info a","multiple":true,"delay":0}]}

Has anyone run into the same problem?

Thank you!

#2

LinkedIn uses lazy loading, so you would need a scroller to load the bottom rows:

{"_id":"linkedin_patrick_novo-v2","startUrl":["https://www.linkedin.com/search/results/companies/?keywords=ERP%20Solução%20empresarial&origin=GLOBAL_SEARCH_HEADER&page=[1-14]"],"selectors":[{"id":"linkedin_link2","type":"SelectorLink","parentSelectors":["scroller"],"selector":"a","multiple":false,"delay":0},{"id":"scroller","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"div.search-result__info","multiple":true,"delay":"2500"}]}