Pagination struggling

How to?
#1

Hey guys,

i am struggling with pagination... do you have any idea how to make it work in case like this?

Sitemap:
{"_id":"test","startUrl":["https://www.realestate.com.au/rent/in-sydney+cbd%2C+nsw/list-1?keywords=unfurnished&includeSurrounding=false&source=location-search"],"selectors":[{"id":"ele","type":"SelectorElement","parentSelectors":["_root"],"selector":"div.listingInfo","multiple":true,"delay":0},{"id":"price","type":"SelectorText","parentSelectors":["ele"],"selector":"p","multiple":false,"regex":"","delay":0},{"id":"street","type":"SelectorText","parentSelectors":["ele"],"selector":"a.name","multiple":false,"regex":"","delay":0},{"id":"bdr","type":"SelectorText","parentSelectors":["ele"],"selector":"dt:contains('Bedrooms') + dd","multiple":false,"regex":"","delay":0},{"id":"bthr","type":"SelectorText","parentSelectors":["ele"],"selector":"dt:contains('Bathrooms') + dd","multiple":false,"regex":"","delay":0},{"id":"parking place","type":"SelectorText","parentSelectors":["ele"],"selector":"dt:contains('Car Spaces') + dd","multiple":false,"regex":"","delay":0},{"id":"pagination","type":"SelectorLink","parentSelectors":["_root","pagination"],"selector":"div:nth-of-type(2) .pagination li:nth-of-type(n+2) a","multiple":true,"delay":0}]}

#2

Almost there. Your Element selector needs to have 2 parent selectors. Both root and the pagination. If you follow the logic and only select 2 parent selectors for pagination, the next page will not return any data, as there is not an Element selector present to scrape the data.

This should work:

{"_id":"test","startUrl":["https://www.realestate.com.au/rent/in-sydney+cbd%2C+nsw/list-1?keywords=unfurnished&includeSurrounding=false&source=location-search"],"selectors":[{"id":"ele","type":"SelectorElement","parentSelectors":["_root","pagination"],"selector":"div.listingInfo","multiple":true,"delay":0},{"id":"price","type":"SelectorText","parentSelectors":["ele"],"selector":"p","multiple":false,"regex":"","delay":0},{"id":"street","type":"SelectorText","parentSelectors":["ele"],"selector":"a.name","multiple":false,"regex":"","delay":0},{"id":"bdr","type":"SelectorText","parentSelectors":["ele"],"selector":"dt:contains('Bedrooms') + dd","multiple":false,"regex":"","delay":0},{"id":"bthr","type":"SelectorText","parentSelectors":["ele"],"selector":"dt:contains('Bathrooms') + dd","multiple":false,"regex":"","delay":0},{"id":"parking place","type":"SelectorText","parentSelectors":["ele"],"selector":"dt:contains('Car Spaces') + dd","multiple":false,"regex":"","delay":0},{"id":"pagination","type":"SelectorLink","parentSelectors":["_root","pagination"],"selector":"div:nth-of-type(2) .nextLink a","multiple":true,"delay":0}]}

#3

thank you very much! :slight_smile:

#4

ok...
so i tried the same method for the other website and it doesn't work, why?

Sitemap:

{"_id":"seloger_test2","startUrl":["https://www.seloger.com/list.htm?types=1%2C2&projects=1&enterprise=0&furnished=0&places=[{ci%3A690123}]&qsVersion=1.0&LISTING-LISTpg=1"],"selectors":[{"id":"pric","type":"SelectorText","parentSelectors":["div"],"selector":"span.c-pa-cprice","multiple":false,"regex":"","delay":0},{"id":"div","type":"SelectorElement","parentSelectors":["_root"],"selector":"div.c-pa-list","multiple":true,"delay":0},{"id":"bedrooms","type":"SelectorText","parentSelectors":["div"],"selector":".c-pa-criterion em:nth-child(2)","multiple":false,"regex":"","delay":0},{"id":"size","type":"SelectorText","parentSelectors":["div"],"selector":".c-pa-criterion em:nth-child(3)","multiple":false,"regex":"","delay":0},{"id":"Type","type":"SelectorText","parentSelectors":["div"],"selector":"a.c-pa-link","multiple":false,"regex":"","delay":0},{"id":"Location/Neighborhood","type":"SelectorText","parentSelectors":["div"],"selector":"div.c-pa-city","multiple":false,"regex":"","delay":0},{"id":"pagin","type":"SelectorLink","parentSelectors":["_root","pagin"],"selector":".pagination-bloc1 a","multiple":true,"delay":0}]}

#5

You need to add your 'pagin' selector as a parent of your 'div' selector. Try taking a look at the selector graph to help you visualize what is going on. It otherwise works fine, you just forgot to tell it what you want it to do with the data past the first page.