Scrape copy with in a multi search page tile

How to?
#1

Hi trying to scrape the copy within a multi page search result, I need it to click into each result and scrape the retail copy? I've tried but I just can not get it to go into each search result??

Url: https://bla.com.au/shop/brand/humminbird/?et_per_page=-1

what I need from each search result "div.tab-content-scroll"

image
image1129×334 7.24 KB

Sitemap:

{"_id":"BLA-HUMMINBIRD","startUrl":["https://bla.com.au/shop/brand/humminbird/?et_per_page=-1"],"selectors":[{"id":"product-wrapper","multiple":true,"parentSelectors":["_root"],"selector":"div.wishlist-disabled","type":"SelectorElement"},{"id":"title","multiple":false,"parentSelectors":["product-wrapper"],"regex":"","selector":".product-title a","type":"SelectorText"},{"id":"rrp","multiple":false,"parentSelectors":["product-wrapper"],"regex":"","selector":"span.price","type":"SelectorText"},{"id":"pages","multiple":true,"parentSelectors":["_root"],"selector":"div.page-wrapper","type":"SelectorElement"},{"id":"title-page","multiple":false,"parentSelectors":["pages"],"regex":"","selector":"h1","type":"SelectorText"},{"id":"retial-copy","multiple":false,"parentSelectors":["pages"],"regex":"","selector":"div.tab-content-scroll","type":"SelectorText"},{"id":"bla-code","multiple":false,"parentSelectors":["pages"],"regex":"","selector":".woocommerce-product-details__short-description p","type":"SelectorText"}]}

#3

are you able to explain it to me so I can work it out for other sites? i don't get the next product section of the code?

#5

can you point me to the docs that show the click into a page and scrap stuff been looking for an hour?

#7

ok ive been having good luck, but I've had troubles with pages that have load more like here https://minnkota.com.au/minn-kota-shop-accessories/

sitemap

{"_id":"MINNKOTA-DIRECT","startUrl":["https://minnkota.com.au/minn-kota-shop-accessories/"],"selectors":[{"id":"element","multiple":true,"parentSelectors":["_root"],"selector":"div.tmb-woocommerce","type":"SelectorElement"},{"id":"link","linkType":"linkFromHref","multiple":false,"parentSelectors":["element"],"selector":".t-entry-title a","type":"SelectorLink"},{"id":"rrp","multiple":false,"parentSelectors":["element"],"regex":"","selector":"bdi","type":"SelectorText"},{"id":"title-page","multiple":false,"parentSelectors":["link"],"regex":"","selector":"h1","type":"SelectorText"},{"id":"bla-code","multiple":false,"parentSelectors":["link"],"regex":"","selector":"span[itemprop='sku']","type":"SelectorText"},{"id":"retial-copy","multiple":false,"parentSelectors":["link"],"regex":"","selector":"div.field-item","type":"SelectorText"}]}```
#8 
{"_id":"MINNKOTA-DIRECT","startUrl":["https://minnkota.com.au/minn-kota-shop-accessories/"],"selectors":[{"id":"element","parentSelectors":["_root"],"type":"SelectorElement","selector":"div.tmb-woocommerce","multiple":true},{"id":"link","parentSelectors":["element"],"type":"SelectorLink","selector":".t-entry-title a","multiple":false,"linkType":"linkFromHref"},{"id":"rrp","parentSelectors":["element"],"type":"SelectorText","selector":"bdi","multiple":false,"regex":""},{"id":"title-page","parentSelectors":["link"],"type":"SelectorText","selector":"h1","multiple":false,"regex":""},{"id":"bla-code","parentSelectors":["link"],"type":"SelectorText","selector":"span[itemprop='sku']","multiple":false,"regex":""},{"id":"retial-copy","parentSelectors":["link"],"type":"SelectorText","selector":"div.field-item","multiple":false,"regex":""},{"id":"pages","parentSelectors":["_root","element","pages"],"paginationType":"auto","selector":".text span","type":"SelectorPagination"}]}
#9

try that one think i got it yer?

#10

when i scrap it doesnt scrap all the data cause it loaded a page then click load more then scrapes where its on the page