Scrap a page with "Load more" not always present

How to?
#1

Hello,

I want to scrap a set of pages but there is not always a "Load more" button. How to scrap page without the button and pages with the button in a single sitemap ?

Thank a lot.

Site map :

{"_id":"societe-com","startUrl":["https://www.societe.com/cgi-bin/liste?nom=&dirig=&pre=&ape=4110A&dep=[01-95]"],"selectors":[{"id":"link","type":"SelectorLink","parentSelectors":["bouton"],"selector":"#liste > a","multiple":true,"delay":0},{"id":"nom","type":"SelectorText","parentSelectors":["link"],"selector":"h1","multiple":false,"regex":"","delay":0},{"id":"adresse","type":"SelectorText","parentSelectors":["link"],"selector":"#rensjur tr:contains('Adresse') td:nth-of-type(2)","multiple":false,"regex":"","delay":0},{"id":"status","type":"SelectorText","parentSelectors":["link"],"selector":".CompanyIdentity__message p:nth-of-type(1)","multiple":false,"regex":"","delay":0},{"id":"bouton","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"a.levelOne","multiple":false,"delay":0,"clickElementSelector":"a.levelOne","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"}]}

It's work only when the Load more button is present

#2

The selector needs to select the items you are after and the click selector should click on the button. This should work:

{"_id":"societe-com","startUrl":["https://www.societe.com/cgi-bin/liste?nom=&dirig=&pre=&ape=4110A&dep=[01-95]"],"selectors":[{"id":"link","type":"SelectorLink","parentSelectors":["bouton"],"selector":"_parent_","multiple":true,"delay":0},{"id":"nom","type":"SelectorText","parentSelectors":["link"],"selector":"h1","multiple":false,"regex":"","delay":0},{"id":"adresse","type":"SelectorText","parentSelectors":["link"],"selector":"#rensjur tr:contains('Adresse') td:nth-of-type(2)","multiple":false,"regex":"","delay":0},{"id":"status","type":"SelectorText","parentSelectors":["link"],"selector":".CompanyIdentity__message p:nth-of-type(1)","multiple":false,"regex":"","delay":0},{"id":"bouton","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"#liste > a","multiple":true,"delay":"2000","clickElementSelector":"a.levelOne","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"}]}