How to perform infinite scroll on other scroll not the main page one

How to?
#1

Hi,

I managed to get the remaining data using another tool, though I wanted to extract the location links for all the branches on this website embedded in "show map directions". The issue is that there are 2 scrolls, a main scroll for the page and another smaller scroll for the locations beside the map, which is the one I need to perform the infinite scroll on. I used the element scroll selector, though every time I press data preview, the scraper scrolls down the main page scroll and not the smaller scroll so the other branches do not appear, even though I highlighted the area for the entire part next to the smaller scroll as the parent and the area for the information on each branch as the child.

Url: DUNKIN'

Sitemap:

{"_id":"DunkinKSA","startUrl":["DUNKIN' link","linkType":"linkFromHref","multiple":true,"parentSelectors":["Locations"],"selector":"div.action-btns:nth-of-type(9) button.btn-default","type":"SelectorLink"},{"delay":2000,"elementLimit":1000,"id":"Locations","multiple":true,"parentSelectors":["_root"],"selector":"div.cls_Sidenav_side_nav, #storeDiv div.restaurant-detail","type":"SelectorElementScroll"}]}

Thanks a lot!!

1 Like
#2

I just wanted to upvote this one because I had a similar issue recently and would like a solution to define which element gets scrolled too.

1 Like
#3

Hi,

Please post the sitemap as Preformatted text

code

#4

Sure!

HYG:
[Preformatted text](https://www.dunkinksa.com/nearby_stores)

#5

Please post the full sitemap JSON.

#6

Do you mean this?

{"_id":"DunkinKSA","startUrl":["https://www.dunkinksa.com/nearby_stores"],"selectors":[{"id":"Location link","parentSelectors":["Locations"],"type":"SelectorLink","selector":"div.action-btns:nth-of-type(9) button.btn-default","multiple":true,"linkType":"linkFromHref"},{"id":"Locations","parentSelectors":["_root"],"type":"SelectorElementScroll","selector":"div.cls_Sidenav_side_nav, #storeDiv div.restaurant-detail","multiple":true,"delay":2000,"elementLimit":1000}]}
#7

Hi,

I added the custom scroll to the sitemap, but it looks like it will not be possible to scrape the location links, since they are not rendered within the HTML and are only visible in a new window after a click on the button.

{"_id":"DunkinKSA-scroll","startUrl":["https://www.dunkinksa.com/nearby_stores"],"selectors":[{"id":"Location link","parentSelectors":["Locations"],"type":"SelectorLink","selector":"div.action-btns:nth-of-type(9) button.btn-default","multiple":true,"linkType":"linkFromHref"},{"id":"Locations","parentSelectors":["_root"],"type":"SelectorElementScroll","selector":"#storeDiv div.restaurant-detail","multiple":true,"delay":2000,"scrollElementSelector":"div.nearby_sidenav","elementLimit":1000}]}