Describe the problem:
I'm scraping a site that shows 60 restaurants and a 'Load More' Button at the bottom. I've managed to do a Element Click that clicks till all buttons are loaded. It works well getting the data from each restaurant page, but the problem is that only the first 60 are scraped.
My guess: it goes into specific restaurant page to grab the data, but when it goes back to the main page to get the second restaurant, it loses the 'Load More' buttons that were clicked before, ignoring all restaurants once loaded.
Am I right?
How do I fix this?
Thanks!!!
Sitemap:
{"_id":"scraping","startUrl":["https://www.ifood.com.br/lista-restaurantes"],"selectors":[{"id":"main","type":"SelectorLink","parentSelectors":["vermais"],"selector":"a.restaurant-card","multiple":true,"delay":0},{"id":"nome","type":"SelectorText","parentSelectors":["main"],"selector":".restaurant-info__title span","multiple":false,"regex":"","delay":0},{"id":"nota","type":"SelectorText","parentSelectors":["main"],"selector":"button.restaurant-rating","multiple":false,"regex":"","delay":0},{"id":"avaliações","type":"SelectorText","parentSelectors":["main"],"selector":"span.restaurant-rating__rating-count","multiple":false,"regex":"","delay":0},{"id":"categoria","type":"SelectorText","parentSelectors":["main"],"selector":".restaurant-info__subtitle span.btn__label","multiple":false,"regex":"","delay":0},{"id":"tempo_km_valor_entrega","type":"SelectorText","parentSelectors":["main"],"selector":"p.restaurant-info__subtitle","multiple":false,"regex":"","delay":0},{"id":"vermais","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.ReactVirtualized__Grid__innerScrollContainer","multiple":true,"delay":0,"clickElementSelector":"button.btn--default","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"}]}