There are several pages with apartments and I need to scrape the content that is inside each page detail.
This is the main URL:
Clicking each property will open a new page with the contents I need to extract. I tried creating the pagination and element click and then a child under element click, but it is getting data only of the first page.
How to solve this?
Try this with Page load delay 5000. I tested with a smaller city so change the url as needed.
{"_id":"realtor-test","startUrl":["https://www.realtor.ca/map#ZoomLevel=7&Center=47.176948%2C-82.077671&LatitudeMax=49.04716&LongitudeMax=-76.05716&LatitudeMin=45.23846&LongitudeMin=-88.09818&view=list&Sort=6-A&PropertyTypeGroupID=1&PropertySearchTypeId=0&TransactionTypeId=2&BuildingTypeId=17&Currency=CAD"],"selectors":[{"id":"Result elements","multiple":true,"parentSelectors":["_root","Click next page button"],"selector":"div.largeListingCardCon div.cardCon","type":"SelectorElement"},{"id":"Price","multiple":false,"parentSelectors":["Result elements"],"regex":"","selector":"div.listingCardPrice","type":"SelectorText"},{"id":"Address","multiple":false,"parentSelectors":["Result elements"],"regex":"","selector":"div.listingCardAddress","type":"SelectorText"},{"id":"Bedrooms","multiple":false,"parentSelectors":["Result elements"],"regex":"","selector":"div.listingCardIconCon:nth-of-type(1) div.listingCardIconTopCon","type":"SelectorText"},{"id":"Bathrooms","multiple":false,"parentSelectors":["Result elements"],"regex":"","selector":"div.listingCardIconCon:nth-of-type(2) div.listingCardIconTopCon","type":"SelectorText"},{"id":"Realtor","multiple":false,"parentSelectors":["Result elements"],"regex":"","selector":"div.listingCardOfficeName","type":"SelectorText"},{"id":"Details page","multiple":false,"parentSelectors":["Result elements"],"selector":"a.blockLink","type":"SelectorLink"},{"clickElementSelector":"div#listViewFooter div > a[aria-label*='the next page']:not([disabled='disabled'])","clickElementUniquenessType":"uniqueHTMLText","clickType":"clickMore","delay":4000,"discardInitialElements":"discard-when-click-element-exists","id":"Click next page button","multiple":false,"parentSelectors":["_root"],"selector":"div#listCon","type":"SelectorElementClick"}]}
Thanks. Only now I had a chance to work on this. It partially works. The problem is in the details column as it contains MLS, price and address all together when they should be in separate columns