How to scrap copy from multiple items

ricco · July 18, 2023, 10:04am

Trying to scrape copy from within a multi item website search,

Sitemap:
{"_id":"SCA_ROLA","startUrl":["https://www.supercheapauto.com.au/brands/rola?srule=top%20rated-top%20seller&sz=60"],"selectors":[{"id":"product-wrapper","multiple":true,"parentSelectors":["_root"],"selector":"div.product-tile","type":"SelectorElement"},{"id":"link","linkType":"linkFromHref","multiple":false,"parentSelectors":["product-wrapper"],"selector":"a.name-link","type":"SelectorLink"},{"extractAttribute":"title","id":"title-attribute","multiple":false,"parentSelectors":["product-wrapper"],"selector":"a","type":"SelectorElementAttribute"}]}

ricco · July 18, 2023, 11:50am

did that fix it ? soz new here

ricco · July 19, 2023, 10:18am

3HAT0K:

{"_id":"SCA_ROLA","startUrl":["https://www.supercheapauto.com.au/brands/rola?srule=top%20rated-top%20seller&sz=60"],"selectors":[{"id":"product-wrapper","multiple":true,"parentSelectors":["pages"],"selector":"div.product-tile","type":"SelectorElement"},{"id":"link","linkType":"linkFromHref","multiple":false,"parentSelectors":["product-wrapper"],"selector":"a.name-link","type":"SelectorLink"},{"extractAttribute":"title","id":"title-attribute","multiple":false,"parentSelectors":["product-wrapper"],"selector":".name-link","type":"SelectorElementAttribute"},{"id":"pages","paginationType":"auto","parentSelectors":["_root","pages"],"selector":".page-next","type":"SelectorPagination"},{"id":"price","multiple":false,"parentSelectors":["product-wrapper"],"regex":"","selector":"span.the-price","type":"SelectorText"}]}

i dont see the copy from within the page when I imported above? it lives within each of the search results and its this Rola Vortex Steel Tray - LTVX | Supercheap AutoPreformatted text

ricco · July 19, 2023, 10:45am

{"_id":"oztrail-anaconda","startUrl":["https://www.anacondastores.com/search?text=oztrail"],"selectors":[{"id":"product-wrapper","multiple":true,"parentSelectors":["_root"],"selector":"div[itemprop='itemListElement']","type":"SelectorElement"},{"id":"title","multiple":false,"parentSelectors":["product-wrapper"],"regex":"","selector":"div[itemprop='name']","type":"SelectorText"},{"id":"price-reg","multiple":false,"parentSelectors":["product-wrapper"],"regex":"","selector":".price-standard span.amount","type":"SelectorText"},{"id":"price-club","multiple":false,"parentSelectors":["product-wrapper"],"regex":"","selector":".price-vip span.amount","type":"SelectorText"},{"id":"image-url","multiple":false,"parentSelectors":["product-wrapper"],"selector":"img.productdetailimg","type":"SelectorImage"},{"id":"page","multiple":true,"parentSelectors":["_root"],"selector":".container-fluid > div.row","type":"SelectorElement"},{"id":"product-copy","multiple":false,"parentSelectors":["page"],"regex":"","selector":"p:nth-of-type(n+2), .pdp-item-list dt, dt:contains('Features') + dd","type":"SelectorText"}]}

ok I got this but I'm stuck at it going into each page from the results table and scraping the copy. I'm missing something.

ricco · July 20, 2023, 4:32am

when i copy the above in and then go data preview i dont see the copy?

ricco · July 20, 2023, 4:33am

wait looks like i gave u the different code i gave you code from my other question

eXmi11 · July 25, 2023, 3:07pm

@3HAT0K I'm facing a similar problem to @ricco . Is there a way of doing the same you are doing but at the same time iterating on every "Sort by" and "items per page" (I know it makes no sense here, but it makes sense in [my use case](https://forum.webscraper.io/t/multiple-variants-scrapping/12312)