How do you get next page on this site:

How to?
#1

Describe the problem.

Url: https://www.kissellmotorsports.com/default.asp?page=xPreOwnedInventory#page=xPreOwnedInventory&make=bmw

As you can see, there are 4 pages available. The way you get to them is click the right arrow. It seems to be at "div div div div.paginationWrapper button.button.next", but when I make a [link] selector, it just ignores it.

Here's the scraper code, but it doesn't work for other pages:

{"_id":"kissellbmw","startUrl":["https://www.kissellmotorsports.com/default.asp?page=xPreOwnedInventory#page=xPreOwnedInventory&make=bmw"],"selectors":[{"id":"bike","type":"SelectorElement","parentSelectors":["_root"],"selector":"div.vehicle_row","multiple":true,"delay":0},{"id":"yearmake","type":"SelectorText","parentSelectors":["bike"],"selector":"h2 a","multiple":false,"regex":"","delay":0},{"id":"mileage","type":"SelectorText","parentSelectors":["bike"],"selector":"li.InvMileage span.unitValue","multiple":false,"regex":"","delay":0},{"id":"price","type":"SelectorText","parentSelectors":["bike"],"selector":"h3 span","multiple":false,"regex":"","delay":0},{"id":"nextpg","type":"SelectorLink","parentSelectors":["_root"],"selector":"div div div div.paginationWrapper button.button.next","multiple":false,"delay":0}]}

#2

To answer my own question, I used url command line to fix this. I noticed that on pages above 1, there was a page number. Here is the URL which fixed it:

https://www.kissellmotorsports.com/default.asp?page=xPreOwnedInventory#page=xPreOwnedInventory&make=bmw&p=[1-7]

Which causes pages 1-7 to be visited. I need to add delays between page visits, but this will work fine.

1 Like