I have a situation where I am scraping products in a store like this:
https://www.teacherspayteachers.com/Store/Pitch-Clips/
I have it set up to visit each product and do the pagination. That works great.
However my end goal is to visit a product like this:
And get the product ID of the products found in bundle as pictured here:
In order to get the product ID, you have to visit the product page. That works pretty well EXCEPT if it has already scraped the product from the initial product scan.
So I have a list of products:
Product A
Product B
Product C
Product D
Product E
Product F
where Product F is a bundle of Product A and B
By the time it gets to scanning product F and then needs to click on the links to Product A and B, it has already scanned Product A and B so won't click on those links to get the necessary data.
Ideas?
Here is the current scraper I'm using:
{"_id":"trythis","startUrl":["https://www.teacherspayteachers.com/Store/pitch-clips"],"selectors":[{"id":"product-link","type":"SelectorLink","parentSelectors":["_root","pagination"],"selector":"#browse_list .title a","multiple":true,"delay":0},{"id":"Product URL","type":"SelectorElementAttribute","parentSelectors":["product-link"],"selector":"meta[name="url"]","multiple":false,"extractAttribute":"content","delay":0},{"id":"Product ID","type":"SelectorElementAttribute","parentSelectors":["product-link"],"selector":"meta[name="sku"]","multiple":false,"extractAttribute":"content","delay":0},{"id":"Find the Also Included in","type":"SelectorElement","parentSelectors":["product-link"],"selector":"div.ParentBundlesPrefab__content","multiple":true,"delay":0},{"id":"Also included in these bundles","type":"SelectorLink","parentSelectors":["Find the Also Included in"],"selector":"a.Anchor.Anchor--green","multiple":true,"delay":0},{"id":"Product Bundle ID","type":"SelectorElementAttribute","parentSelectors":["Also included in these bundles"],"selector":"meta[name="sku"]","multiple":false,"extractAttribute":"content","delay":0},{"id":"pagination","type":"SelectorLink","parentSelectors":["_root","pagination"],"selector":".next a","multiple":false,"delay":900}]}