How to scrape this page with the pagination next button

How to scrape this page with the pagination next button.

I want to scrape the records (incl. price. I don´t know why it doesn´t appear?)

I copied the configuration from a post (Lawyer directory) but it does´t work. Thanks

{"_id":"deephouse","startUrl":["https://www.htfr.com/music/g/47/deep_house"],"selectors":[{"id":"pagination","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.plistingContent","multiple":true,"delay":0,"clickElementSelector":"div.paginationBox a.prev:nth-of-type(7)","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"link","type":"SelectorLink","parentSelectors":["pagination"],"selector":"div.prodTitle a.title","multiple":false,"delay":0},{"id":"record","type":"SelectorElement","parentSelectors":["link"],"selector":"div.Container:nth-of-type(1) div.plistingContent","multiple":true,"delay":0},{"id":"artist","type":"SelectorText","parentSelectors":["record"],"selector":"div.prodTitle a.title","multiple":false,"regex":"","delay":0},{"id":"title","type":"SelectorText","parentSelectors":["record"],"selector":"strong a.title","multiple":false,"regex":"","delay":0},{"id":"label","type":"SelectorText","parentSelectors":["record"],"selector":"a.labelh","multiple":false,"regex":"","delay":0},{"id":"preis","type":"SelectorText","parentSelectors":["record"],"selector":"input.price","multiple":false,"regex":"","delay":0}]}

Anyone has a solution for this one ? Thanks !

This is the "other" type of pagination where the link changes so you can't use Element-Click

For this you need a regular Link Selector, (make that a child onto it's self)
Then, in root, alongside the Pagination, add Element Selector, set to multiple that has each record highlighted - make that a child to root & Pagination
-Go inside Element Selector (child to) and put in all your selectors
-Price is tricky - you need Element Attribute set to value

{"_id":"deephouse","startUrl":["https://www.htfr.com/music/g/47/deep_house"],"selectors":[{"id":"Pag","type":"SelectorLink","parentSelectors":["_root","Pag"],"selector":"div.navigation a.prev:nth-of-type(7)","multiple":false,"delay":0},{"id":"Elemen","type":"SelectorElement","parentSelectors":["_root","Pag"],"selector":"div.Container","multiple":true,"delay":0},{"id":"Title","type":"SelectorText","parentSelectors":["Elemen"],"selector":"div.prodTitle a.title","multiple":false,"regex":"","delay":0},{"id":"Sub Title","type":"SelectorText","parentSelectors":["Elemen"],"selector":"strong a.title","multiple":false,"regex":"","delay":0},{"id":"Label","type":"SelectorText","parentSelectors":["Elemen"],"selector":"span.spacing:nth-of-type(1)","multiple":false,"regex":"","delay":0},{"id":"Price","type":"SelectorElementAttribute","parentSelectors":["Elemen"],"selector":"input.price","multiple":false,"extractAttribute":"value","delay":0},{"id":"MRNumber","type":"SelectorText","parentSelectors":["Elemen"],"selector":"span.mrNumber:nth-of-type(3)","multiple":false,"regex":"","delay":0}]}

Should you just want the data - https://docs.google.com/spreadsheets/d/1qiS1dNoMpAcWXyRGpkm_mZfIfqdSN4olnorIuqvD0Lw/edit?usp=sharing

Thanks !
Do you have more tutorials regarding pagination (especially the next button)? I really don´t understand it.

I have two examples (can´t put the URL because it requires password).
1.
URLs stay the same https://app.coobis.com/brands/orders/32684?num=11675
https://app.coobis.com/brands/orders/32684?num=11675
The next button is javascript
2.
URL changes:
https://www.publisuites.com/advertisers/newspapers/
https://www.publisuites.com/advertisers/newspapers/2/

The Next button always links to the next page f.e.

https://www.publisuites.com/advertisers/newspapers/3/

Any help on this one?

Can anybody help ? Thanks !

Hi @Gabriel,

please refer to official Tutorials section on a main website.

Hola Gabriel, estoy intentando escrapear las webs de Publisuites, pero no los periódicos, los blogs, por temáticas.... Estas no cambian la URL por lo que veo. Estoy atascado con el botón "Siguiente". Has conseguido sacarlas ?? Podrías indicarme cómo es el sitemap ?? (Solo me interesan los dominios)

Que va. Los paginations me matan

There's no real need to click on the next button cos this site can also be navigated via "pages", .e.g.
https://www.htfr.com/pages/quickSearch.php?newSearch=0&rowStart=0
https://www.htfr.com/pages/quickSearch.php?newSearch=0&rowStart=32
https://www.htfr.com/pages/quickSearch.php?newSearch=0&rowStart=64

Looks like 1 page = 32 items

So for example, if you just want the first 5 pages, you can use this as your start URL
https://www.htfr.com/pages/quickSearch.php?newSearch=0&rowStart=[0-160:32]
'cos 5 x 32 = 160

Look up range url with increment.