Pagination issue: no data found

How to?
#1

Hello,

I have been trying to scrape this website:
https://howimetyourplanet.fr/entreprise

I need to use pagination as it is described in this video: Wpgmp_pagination: Impossibile Select Pagination Links

Pagination seems to work well (My script is running and I can see that pages are changing), however I don't get any data from each page and I can't explain why.

Update: When I use pagination in URLs it seems to be working however I get the same problem with other websites.

Thanks in advance for your help

#2

Hi,

I have the same issue. I could do it some weeks ago it seemed to work perfectly but now I cannot do it.

I tried with this website: Résultat de la recherche - Qualibat

Can you please guide us and tell why we cannot get scrapped data even though pagination seems to work?

Thanks

#3

@theolito @nathangui Hi. I tested it and everything seems to work.

{"_id":"howimetyourplanet-fr","startUrl":["https://howimetyourplanet.fr/entreprise/[1-7]"],"selectors":[{"id":"data-wrapper","type":"SelectorElement","parentSelectors":["_root"],"selector":"div.app_companies__item_wide","multiple":true,"delay":0},{"id":"name","type":"SelectorText","parentSelectors":["data-wrapper"],"selector":"a.app_companies__item_wide__name","multiple":false,"regex":"","delay":0},{"id":"location","type":"SelectorText","parentSelectors":["data-wrapper"],"selector":"div.app_companies__item_wide__location","multiple":false,"regex":"","delay":0}]}

test
test2125×1375 487 KB

I used page interval pagination defined in the start URL.

#4

Hello, thanks a lot @ViestursWS. I have been doing that as well and it is working. However it is not working everytime I use element click (as it is explained in tutorials videos), so either I am not doing it correctly either it is not working anymore. Can someone help?

#5

Hello @theolito

You can't use element-click because each page is a different link with different HTML.

You can use link selector as well like for example:

{"_id":"howimetyourplanet-fr","startUrl":["https://howimetyourplanet.fr/entreprise"],"selectors":[{"id":"data-wrapper","type":"SelectorElement","parentSelectors":["_root","link-pagination"],"selector":"div.app_companies__item_wide","multiple":true,"delay":0},{"id":"name","type":"SelectorText","parentSelectors":["data-wrapper"],"selector":"a.app_companies__item_wide__name","multiple":false,"regex":"","delay":0},{"id":"location","type":"SelectorText","parentSelectors":["data-wrapper"],"selector":"div.app_companies__item_wide__location","multiple":false,"regex":"","delay":0},{"id":"link-pagination","type":"SelectorLink","parentSelectors":["_root","link-pagination"],"selector":".app_pagination__item--next a","multiple":true,"delay":0}]}