Tried this cool little sitemap to make my research work, but might be missing something

How to?
#1

Hi, I've already used the platform for simple data scraping, but I can't make this trick work.

I'd like to gather data from every organisation that appears on the list provided by the EuropeanUnion website

Ex : 13000+ organisations are registered on this list, and I would like the scrapper to extract data from every company accessible by a link on the "Search the register" part.

By clicking on every link, I should land on company pages which look like this.

I want to capture the organisation name

image
image1131×52 1.02 KB

But also the person in charge for EU relations and his/her position

image
image1103×180 8.49 KB

And finally the people having access to European Parliament

image
image1122×459 19.5 KB

Here's what I tried :
{"_id":"transparencyOrganisations","startUrl":["Search the register - European Union span"},{"id":"organisationLink","parentSelectors":["pagination"],"type":"SelectorLink","selector":".ecl-content-block__title a","multiple":true,"linkType":"linkFromHref"},{"id":"orgaName","parentSelectors":["organisationLink"],"type":"SelectorText","selector":"td:nth-of-type(2) strong","multiple":false,"regex":""},{"id":"personInChargeName","parentSelectors":["organisationLink"],"type":"SelectorText","selector":"tr:contains('Person in charge of EU relations:') td:nth-of-type(2)","multiple":false,"regex":""},{"id":"personInChargePosition","parentSelectors":["organisationLink"],"type":"SelectorText","selector":"table:nth-of-type(5) tr:contains('Position:') td:nth-of-type(2)","multiple":false,"regex":""},{"id":"EPaccessName","parentSelectors":["wrapper_for_EPaccessName_EPaccess1stName"],"type":"SelectorText","selector":"td:nth-of-type(1)","multiple":false,"regex":""},{"id":"EPaccess1stName","parentSelectors":["wrapper_for_EPaccessName_EPaccess1stName"],"type":"SelectorText","selector":"td:nth-of-type(2)","multiple":false,"regex":""},{"id":"wrapper_for_EPaccessName_EPaccess1stName","parentSelectors":["organisationLink"],"type":"SelectorElement","selector":".ecl-table__cell .ecl-table__body tr","multiple":true}]}

Unfortunately it doesn't seem to work, only going through the list without capturing anything.

Does someone have the same issue ? Could someone kindly help me ?
Thank you !

#2

try this one
{"_id":"Test-2","startUrl":["https://transparency-register.europa.eu/searchregister-or-update/search-register_en?searchType=REGISTRANTS&alphaType=LATIN&letter=a&page=[1-10]#list-of-all-lobbyists"],"selectors":[{"id":"Organization","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root"],"selector":"div.ecl-u-mb-2xl:nth-of-type(3) .ecl-table__cell a","type":"SelectorLink"},{"id":"Organization Name","multiple":false,"parentSelectors":["Organization"],"regex":"","selector":"table:nth-of-type(1) tr:contains('Organisation name:') td:nth-of-type(2)","type":"SelectorText"},{"id":"Person in charge of EU relations","multiple":false,"parentSelectors":["Organization"],"regex":"","selector":"tr:contains('Person in charge of EU relations:') td:nth-of-type(2)","type":"SelectorText"},{"id":"Position","multiple":false,"parentSelectors":["Organization"],"regex":"","selector":"table:nth-of-type(5) tr:contains('Position:') td:nth-of-type(2)","type":"SelectorText"},{"columns":[{"extract":true,"header":"Surname","name":"Surname"},{"extract":true,"header":"First name","name":"First name"},{"extract":true,"header":"Start date","name":"Start date"},{"extract":true,"header":"End date","name":"End date"}],"id":"Persons accredited for access to European Parliament premises","multiple":true,"parentSelectors":["Organization"],"selector":".ecl-table__cell table.ecl-table","tableDataRowSelector":".ecl-table__body tr","tableHeaderRowSelector":".ecl-table__head tr","type":"SelectorTable"}]}