Scraping Data After Load More

How to?
#1

I am trying to scrape restaurant details from their individual pages, after using an Element Select which clicks on the Load More button. This is after going into each region's page.

The Load More buttons are pressed with the full list of restaurants shown on the page, but no data is captured from this and no data is got from the individual pages either.

Url: https://www.kfc.co.uk/kfc-near-me

Sitemap:
{"_id":"kfc","startUrl":["https://www.kfc.co.uk/kfc-near-me"],"selectors":[{"id":"region","type":"SelectorLink","parentSelectors":["_root"],"selector":"a.q6md28-6","multiple":true,"delay":0},{"id":"name","type":"SelectorElementClick","parentSelectors":["region"],"selector":"a.fiaslf-5","multiple":true,"delay":"10000","clickElementSelector":"button.bcNAlW","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"fullname","type":"SelectorText","parentSelectors":["name"],"selector":"h1","multiple":true,"regex":"","delay":0},{"id":"address","type":"SelectorText","parentSelectors":["name"],"selector":"p.sc-gqjmRU","multiple":true,"regex":"","delay":0},{"id":"takeaways","type":"SelectorHTML","parentSelectors":["name"],"selector":"div.mo02c9-1","multiple":true,"regex":"","delay":0},{"id":"features","type":"SelectorText","parentSelectors":["name"],"selector":"div.cVEaDS","multiple":true,"regex":"","delay":0}]}

#2

Now, It's working perfectly.

Sitemap:

{"_id":"kfc","startUrl":["https://www.kfc.co.uk/kfc-near-me"],"selectors":[{"id":"region","type":"SelectorLink","parentSelectors":["_root"],"selector":"a.q6md28-6","multiple":true,"delay":0},{"id":"name","type":"SelectorElementClick","parentSelectors":["region"],"selector":"a.fiaslf-5","multiple":true,"delay":"10000","clickElementSelector":"button.bcNAlW","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"fullname","type":"SelectorText","parentSelectors":["location"],"selector":"h1","multiple":false,"regex":"","delay":0},{"id":"address","type":"SelectorText","parentSelectors":["location"],"selector":"p.sc-gqjmRU","multiple":false,"regex":"","delay":0},{"id":"takeaways","type":"SelectorHTML","parentSelectors":["location"],"selector":"div.mo02c9-1","multiple":false,"regex":"","delay":0},{"id":"features","type":"SelectorText","parentSelectors":["location"],"selector":"div.cVEaDS","multiple":false,"regex":"","delay":0},{"id":"location","type":"SelectorLink","parentSelectors":["name"],"selector":"parent","multiple":true,"delay":0}]}

#3

{"_id":"kfc","startUrl":["https://www.kfc.co.uk/kfc-near-me"],"selectors":[{"id":"region","type":"SelectorLink","parentSelectors":["_root"],"selector":"a.q6md28-6","multiple":true,"delay":0},{"id":"name","type":"SelectorElementClick","parentSelectors":["region"],"selector":"a.fiaslf-5","multiple":true,"delay":"10000","clickElementSelector":"button.bcNAlW","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"fullname","type":"SelectorText","parentSelectors":["location"],"selector":"h1","multiple":false,"regex":"","delay":0},{"id":"address","type":"SelectorText","parentSelectors":["location"],"selector":"p.sc-gqjmRU","multiple":false,"regex":"","delay":0},{"id":"takeaways","type":"SelectorHTML","parentSelectors":["location"],"selector":"div.mo02c9-1","multiple":false,"regex":"","delay":0},{"id":"features","type":"SelectorText","parentSelectors":["location"],"selector":"div.cVEaDS","multiple":false,"regex":"","delay":0},{"id":"location","type":"SelectorLink","parentSelectors":["name"],"selector":"_parent_","multiple":true,"delay":0}]}

#4

Fantastic, thanks Asad!

1 Like