Hi... I am trying to scrape data from equip-bid.com. I have been able to get it to work at a basic level, but now I am trying to get more info to pull and I am having issues with doing that.
Briefly, equip-bid runs many auctions and each auction has many items. Those items are listed 50 at a time, or you can click into each item. In order to save time and hits on the website I would like to gather the data from the 50 per page location, rather than clicking into each of those 50 pages.
Here is an example auction page.
https://www.equip-bid.com/auction/19530
There are three viewing options (short, full and grid). I was able to get it to work on the grid, but I can't get all the data I want. Or I can get it to work by clicking into each of the 50 pages, but again, would prefer to avoid doing that because it takes 50x as much time. All of the data I want/need is shown on the "full" layout, so I would like to find a way to pull the items I need from the "50 at a time" page set to "full" layout.
These are the fields I would like to pull
- div.lot-description.description-wrap-fix
- span.lot-current-bid
- .col-xs-12 div:nth-of-type(2) span.pull-right
- small .pull-right span
Or, in other terms
- The grey box below each items that starts with "Description:"
- Current Bid
- Next Required Bid
- High Bidder
How can I pull all these fields into one row per auction item? I have tried using the Element selector, but these fields are grouped under separate elements, so that will not work.