Why isn’t the pagination taking all the data?

How to?
#1

I configured the pagination, but it only returns 60 rows of data. Is there something missing here?

Url: iFood

Sitemap:
{"_id":"ifood","startUrl":["https://www.ifood.com.br/restaurantes"],"selectors":[{"id":"pagination-merchant","parentSelectors":["_root","pagination-merchant"],"paginationType":"auto","type":"SelectorPagination","selector":"button.btn"},{"id":"merchant-wrapper","parentSelectors":["_root"],"type":"SelectorElement","selector":"div.merchant-v2","multiple":true,"scroll":false,"elementLimit":0},{"id":"name","parentSelectors":["merchant-wrapper"],"type":"SelectorText","selector":"span.merchant-v2__name","multiple":false,"regex":"","multipleType":"singleColumn"},{"id":"eta","parentSelectors":["merchant-wrapper"],"type":"SelectorText","selector":"div.merchant-v2__footer","multiple":false,"regex":"","multipleType":"singleColumn"},{"id":"fee","parentSelectors":["merchant-wrapper"],"type":"SelectorText","selector":".merchant-v2__footer span:nth-of-type(2)","multiple":false,"regex":"","multipleType":"singleColumn"},{"id":"rating","parentSelectors":["merchant-wrapper"],"type":"SelectorText","selector":"span.null","multiple":false,"regex":"","multipleType":"singleColumn"},{"id":"merchant-detail","parentSelectors":["merchant-wrapper"],"type":"SelectorText","selector":"div.merchant-v2__info","multiple":false,"regex":"","multipleType":"singleColumn"}]}

#2

Hi,

Try setting the wrapper selector as a child to the pagination:

image

1 Like
#3

Thank you very much, it works!
Is it possible to define a condition to stop the scrape?

#4

Yes. You can limit the pagination by addig something like body:not(:has(.merchant-list-v2__item-wrapper:nth-of-type(100))) to the pagination selector.

This means that the button will be clicked until the 100th merchant element is loaded in the HTML.

{"_id":"ifood","startUrl":["https://www.ifood.com.br/restaurantes"],"selectors":[{"id":"pagination-merchant","paginationType":"auto","parentSelectors":["_root","pagination-merchant"],"selector":"body:not(:has(.merchant-list-v2__item-wrapper:nth-of-type(100))) button.btn","type":"SelectorPagination"},{"elementLimit":0,"id":"merchant-wrapper","multiple":true,"parentSelectors":["pagination-merchant"],"scroll":false,"selector":"div.merchant-v2","type":"SelectorElement"},{"id":"name","multiple":false,"multipleType":"singleColumn","parentSelectors":["merchant-wrapper"],"regex":"","selector":"span.merchant-v2__name","type":"SelectorText"},{"id":"eta","multiple":false,"multipleType":"singleColumn","parentSelectors":["merchant-wrapper"],"regex":"","selector":"div.merchant-v2__footer","type":"SelectorText"},{"id":"fee","multiple":false,"multipleType":"singleColumn","parentSelectors":["merchant-wrapper"],"regex":"","selector":".merchant-v2__footer span:nth-of-type(2)","type":"SelectorText"},{"id":"rating","multiple":false,"multipleType":"singleColumn","parentSelectors":["merchant-wrapper"],"regex":"","selector":"span.null","type":"SelectorText"},{"id":"merchant-detail","multiple":false,"multipleType":"singleColumn","parentSelectors":["merchant-wrapper"],"regex":"","selector":"div.merchant-v2__info","type":"SelectorText"}]}
1 Like