Linkedin pagination

Does anybody have experience with Linkedin Search pagination links? Has anybody managed to automate their clicking? As far as I understand the situation, after the page is loaded there is no URL for pagination links, but the URL is generated dynamically. In other words, when you click on the pagination link then the website generates the URL.

I went through the Web Scraper pagination & tutorial and didn't have much success with implementing the right solution to the problem.

Also, I have the same problem while trying to scrape search results with Linkedin Sales Navigator.

At the moment, I am adding pagination links manually as a "Start URL" and it would be great to have it automated.

URL example of Linkedin Search: https://www.linkedin.com/search/results/people/?facetCurrentCompany=["1033"]&origin=FACETED_SEARCH&page=1

This is the Sitemap that I use for scraping Linkedin Sales Navigator:
{"_id":"lsn-fraudscope","startUrl":["https://www.linkedin.com/sales/search/people?company=FraudScope%3A16236221&companyTimeScope=CURRENT&logHistory=true&page=1&searchSessionId=Xxja1ikoR9e7ymqIy%2FkRHQ%3D%3D"],"selectors":[{"id":"profile","type":"SelectorLink","selector":"dt.result-lockup__name a.ember-view","parentSelectors":["profile-scrolldown"],"multiple":false,"delay":0},{"id":"profile-name","type":"SelectorText","selector":".profile-topcard-person-entity__name","parentSelectors":["profile"],"multiple":false,"regex":"","delay":0},{"id":"profile-position","type":"SelectorText","selector":"dt.profile-position__title","parentSelectors":["profile"],"multiple":false,"regex":"","delay":0},{"id":"profile-position-description","type":"SelectorText","selector":"dd.profile-position__description","parentSelectors":["profile"],"multiple":false,"regex":"","delay":0},{"id":"profile-position-location","type":"SelectorText","selector":"div.profile-topcard__location-data","parentSelectors":["profile"],"multiple":false,"regex":"","delay":0},{"id":"profile-position-when_profile_locked","type":"SelectorText","selector":".profile-topcard-person-entity__content dd","parentSelectors":["profile"],"multiple":false,"regex":"","delay":0},{"id":"profile-scrolldown","type":"SelectorElementScroll","selector":"li.search-results__result-item","parentSelectors":["_root"],"multiple":true,"delay":0}]}

1 Like

In Linkedin regular, the pagination = .next-text
In Linkedin Recruiter the pagination = .page-link:last-child

I don't have sales navigator but I would try both of these.

Be careful, Linkedin's scraping detection is pretty good and you will end up blocked or in "Linkedin-Jail" as they say.

Thanks for the replay.

I solved my problem by adding a range in my URL (example: page=[1-20]) and then adding that URL as "Start URL".

2 Likes

Hello!! I registered an account here just to say thank you!!!!

2 Likes

TYVM leaf! I'm glad that my post was of help to someone.

1 Like