Is there any way to do this? I am trying to pull Auction Data from completed auctions.
Example Completed Auction: Bone Healing System, Ring Lights, Home Items, Furniture, Plumbing Supplies, Pet Sprayers, VR Headsets, and More | Equip-Bid
Example Sitemap:
{"_id":"example","startUrl":["Current Auctions | Equip-Bid strong","multiple":true,"regex":""},{"id":"Current Bid","parentSelectors":["_root"],"type":"SelectorText","selector":"span.lot-current-bid","multiple":true,"regex":""},{"id":"Auction Close Date","parentSelectors":["_root"],"type":"SelectorText","selector":"div[title]","multiple":true,"regex":""},{"id":"Description","parentSelectors":["_root"],"type":"SelectorText","selector":"div.lot-description.description-wrap-fix","multiple":true,"regex":""}]}
Notes:
-
Auctions can be shown in 3 different formats (Short, Full, Grid). Make sure you have it set to full (in the filter options on the left side of page). I would like to pull info only available in "full".
-
I am aware of Element Wrappers. The website is designed such that each item has 3 different sections. Those 3 sections cannot be combined with a common wrapper (at least I have not been able to do so, I am hoping someone else knows how). But please do not JUST mention wrappers as a solution. If wrappers can solve this I will need a little more guidance on how to do that.
-
I am aware that by going to the individual items' pages I can pull all of this information. But doing so means increasing the number of pages I have to scrape by ~50x (since there are 50 items per page). I was hoping to pull a few thousand previous auctions, with an average of maybe 3 pages an auction. Meaning that going into each item individually would change this from scraping maybe 10,000 pages to scraping 50,000 pages.
So I am looking for any way to do this from the Auction page rather than the item page. Is there a way to group more than one wrapper? Or a way to make this work with the "Grouped" selector? Or anything else I am not thinking about?
Thanks for any help.