Scroll + click + scrape product page

How to?
#1

Hi,

I've checked videos and read posts on the forum but I'm stuck.
I'm trying to scrape a site that has a scroller.

I managed to have the page being fuller loaded using the scroller element but then I need each product page to be clicked so I can scrape the data on each of these pages.

Unfortunately, once I launch the scraper it seems to work, it loads the page fully, then opens each product page but in the end I can see it only scraped the first 48 products there were showing originally and the 300+ others.

What am I missing here?
Thanks for your help

Url: Pööratud hinnad - prisma | alati odavad hinnad, telli kaup otse koju

Sitemap:
{"_id":"prisma-2-test","startUrl":["https://www.prismamarket.ee/products/collection/pooratud-hinnad"],"selectors":[{"id":"link","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root"],"selector":"a.js-link-item","type":"SelectorLink"},{"id":"EAN","multiple":false,"parentSelectors":["link"],"regex":"","selector":"span[itemprop='sku']","type":"SelectorText"},{"id":"title","multiple":false,"parentSelectors":["link"],"regex":"","selector":"h1","type":"SelectorText"},{"id":"price","multiple":false,"parentSelectors":["link"],"regex":"","selector":"div.js-details","type":"SelectorText"},{"id":"price-1","multiple":false,"parentSelectors":["link"],"regex":"","selector":"div.pricebox","type":"SelectorText"},{"delay":2000,"elementLimit":500,"id":"scroll","multiple":true,"parentSelectors":["_root"],"selector":"div.info.relative","type":"SelectorElementScroll"}]}

#2

try this sitemap:

{"_id":"prisma-2-test","startUrl":["https://www.prismamarket.ee/products/collection/pooratud-hinnad"],"selectors":[{"id":"title","multiple":false,"parentSelectors":["link"],"regex":"","selector":"h1","type":"SelectorText"},{"id":"EAN","multiple":false,"parentSelectors":["link"],"regex":"","selector":"span[itemprop='sku']","type":"SelectorText"},{"id":"price","multiple":false,"parentSelectors":["link"],"regex":"","selector":"div.js-details","type":"SelectorText"},{"id":"price-1","multiple":false,"parentSelectors":["link"],"regex":"","selector":"div.pricebox","type":"SelectorText"},{"delay":1000,"elementLimit":800,"id":"scroll","multiple":true,"parentSelectors":["_root"],"selector":"li[class*=\"js-shelf-item\"]","type":"SelectorElementScroll"},{"id":"link","linkType":"linkFromHref","multiple":false,"parentSelectors":["scroll"],"selector":"a.js-link-item","type":"SelectorLink"}]}
1 Like
#3

Holly molly, this actually works perfectly!! :open_mouth:

Thanks a lot for your help!!