Hello all,
I am trying to scrape from the below site and category:
Url: https://samkoandmikotoywarehouse.com/product-category/toys/
Sitemap:
{"_id":"samko_all","startUrl":["https://samkoandmikotoywarehouse.com/product-category/toys/"],"selectors":[{"id":"load_more","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"li.col-md-5ths","multiple":true,"delay":"2000","clickElementSelector":"nav.woocommerce-pagination a.button","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"product","type":"SelectorLink","parentSelectors":["load_more"],"selector":"a","multiple":false,"delay":"2000"},{"id":"upc","type":"SelectorText","parentSelectors":["product"],"selector":"div.product_meta","multiple":false,"regex":"","delay":0}]}
This page displays products, with more products displaying when "Load More" is clicked. I am able to use an Element Click selector to successful click "Load More" until all products are shown, but my child link selector will not click on the products. I need to go into the product pages to get the UPC, which I'm also having trouble selecting on it's own, but instead am grabbing additional data and cleaning up in Excel.
I've seen similar example sites like this:
Url: https://www.cablesandsensors.com
That have seemingly similar sitemaps and needs (based on the selector graph) that work fine:
Sitemap:
{"_id":"cables_and_sensors","startUrl":"https://www.cablesandsensors.com","selectors":[{"id":"load","type":"SelectorElementClick","parentSelectors":["category"],"selector":"div.collection-item","multiple":true,"delay":"","clickElementSelector":"button.btn.btn-huge.btn-light","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"ItemTitle","type":"SelectorText","parentSelectors":["item"],"selector":"div.section.section-header","multiple":false,"regex":"","delay":""},{"id":"Price","type":"SelectorText","parentSelectors":["item"],"selector":"div.col-xs-8 span.money","multiple":false,"regex":"","delay":""},{"id":"CandS_PN","type":"SelectorText","parentSelectors":["item"],"selector":"span.variant-sku","multiple":false,"regex":"","delay":""},{"id":"OEM Part Number","type":"SelectorTable","parentSelectors":["item"],"selector":"div.main-content div.col-xs-12 div table.table","multiple":true,"columns":[{"header":"Manufacturer","name":"Manufacturer","extract":true},{"header":"OEM Part #","name":"OEM Part #","extract":true}],"delay":"","tableDataRowSelector":"tr:nth-of-type(n+2)","tableHeaderRowSelector":"tr:nth-of-type(1)"},{"id":"Compatibility","type":"SelectorTable","parentSelectors":["item"],"selector":"table#compatibility.table","multiple":true,"columns":[{"header":"Manufacturer","name":"Manufacturer","extract":true},{"header":"Model","name":"Model","extract":true}],"delay":"","tableDataRowSelector":"tr:nth-of-type(n+2)","tableHeaderRowSelector":"tr.table-heading:contains('Manufacturer')"},{"id":"category","type":"SelectorLink","parentSelectors":["_root"],"selector":"a.visible-lg-block","multiple":true,"delay":""},{"id":"item","type":"SelectorLink","parentSelectors":["load"],"selector":"a","multiple":false,"delay":""}]}
I've been stuck on this for awhile - any insight would be greatly appreciated!