Selector is not selecting all elements / un-selects them

[Citylights]

Hi all - I would highly appreciate some help here.
I'm trying to get some appartment for rent data scraped on 10 pages.

Describe the problem.

While I can enter the first links and click multiple of them not all elements get selected. So I tried to hold shift and added the others as well, which works in first instance. But when running Web Scraper only some elements get actively taken and most of them are left out (especially on the further pages).

Pagination though seems to work as intended.

Url: Wohnung mieten in 4053 - Vergleiche 100 Inserate mit comparis.ch

Sitemap:
{{"_id":"suchfiltercomparis","startUrl":["Wohnung mieten in 4053 - Vergleiche 100 Inserate mit comparis.ch a","multiple":true,"delay":0},{"id":"Objekt","type":"SelectorLink","parentSelectors":["_root","pagination"],"selector":"div.css-ctytwt:nth-of-type(n+3) a, div:nth-of-type(n+9) a","multiple":true,"delay":0},{"id":"Titel","type":"SelectorText","parentSelectors":["Objekt"],"selector":"p.css-1e0n6hi","multiple":false,"regex":"","delay":0},{"id":"Beschreibung","type":"SelectorText","parentSelectors":["Objekt"],"selector":"div.css-16i3sck","multiple":false,"regex":"","delay":0},{"id":"Strasse + Nr","type":"SelectorText","parentSelectors":["Objekt"],"selector":"h5","multiple":false,"regex":"","delay":0}]}}

Thanks for any support/help

[Asad]

Hi,
Just add the other things you want to scrape.
Here is your new sitemap:

{"_id":"comparis","startUrl":["https://www.comparis.ch/immobilien/result/list?requestobject=%7B%22DealType%22%3A10%2C%22SiteId%22%3A0%2C%22RootPropertyTypes%22%3A%5B1%5D%2C%22PropertyTypes%22%3A%5B%5D%2C%22RoomsFrom%22%3A%223%22%2C%22RoomsTo%22%3Anull%2C%22FloorSearchType%22%3A0%2C%22LivingSpaceFrom%22%3Anull%2C%22LivingSpaceTo%22%3Anull%2C%22PriceFrom%22%3Anull%2C%22PriceTo%22%3Anull%2C%22ComparisPointsMin%22%3A0%2C%22AdAgeMax%22%3A0%2C%22AdAgeInHoursMax%22%3Anull%2C%22Keyword%22%3A%22%22%2C%22WithImagesOnly%22%3Afalse%2C%22WithPointsOnly%22%3Anull%2C%22Radius%22%3A%221%22%2C%22MinAvailableDate%22%3Anull%2C%22MinChangeDate%22%3A%221753-01-01T00%3A00%3A00%22%2C%22LocationSearchString%22%3A%224053%22%2C%22Sort%22%3A11%2C%22HasBalcony%22%3Afalse%2C%22HasTerrace%22%3Afalse%2C%22HasFireplace%22%3Afalse%2C%22HasDishwasher%22%3Afalse%2C%22HasWashingMachine%22%3Afalse%2C%22HasLift%22%3Afalse%2C%22HasParking%22%3Afalse%2C%22PetsAllowed%22%3Afalse%2C%22MinergieCertified%22%3Afalse%2C%22WheelchairAccessible%22%3Afalse%2C%22LowerLeftLatitude%22%3Anull%2C%22LowerLeftLongitude%22%3Anull%2C%22UpperRightLatitude%22%3Anull%2C%22UpperRightLongitude%22%3Anull%7D&page=0"],"selectors":[{"id":"elements","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.css-ctytwt.excbu0j5","multiple":true,"delay":2000,"clickElementSelector":"li.css-ph137m:nth-of-type(n+3) a","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"link","type":"SelectorLink","parentSelectors":["elements"],"selector":"a","multiple":false,"delay":0},{"id":"price","type":"SelectorText","parentSelectors":["link"],"selector":".css-eujsoq > font font","multiple":false,"regex":"","delay":0}]}

[Citylights]

Hi Asad

Many thanks for your reply, sounds like just an easy step.
I will try this sitemap later on today and let you know how that went

[Citylights]

Hi again

Unfortunately this didn't work out, I tried now to arrange it with elements, but I get the same issue as with links. It seems that my selected wrappers/objects get de-selected after running the scraper (output is either only 4-5 entries or 22, but should be 79).

Below is my new sitemap I used and attached are screenshots (1&2 showing how every element looks correctly selected). Screenshots 3&4 are showing what I get on the second page when using pagina (which I didn't add yet on the new sitemap).

Or could it be that the site is somehow preventing this multiple selecting?

Any help is appreciated

New sitemap:
{"_id":"elemtstry","startUrl":["3 bis 3.5 Zimmer Wohnung mieten in 4053 div.css-hs6l4r, div:nth-of-type(n+9) div.css-hs6l4r","multiple":true,"delay":0},{"id":"Price","type":"SelectorText","parentSelectors":["object-wrapper"],"selector":"div.css-1pqw1tb","multiple":false,"regex":"","delay":0},{"id":"Street","type":"SelectorText","parentSelectors":["object-wrapper"],"selector":"p.css-a7uk28","multiple":false,"regex":"","delay":0}]}

Screenshots

[ViestursWS]

@Citylights
Hi, can you copy n paste your sitemaps using preformatted text? Because I was not able to view any of them.

Are you looking to extract all of the properties?
If so... This should work. I would use page interval pagination and element-scroll because 2nd half of the listings are not discovered without it as it seems.

Sitemap example:

{"_id":"comparis","startUrl":["https://www.comparis.ch/immobilien/result/list?requestobject=%7B%22DealType%22%3A10%2C%22SiteId%22%3A0%2C%22RootPropertyTypes%22%3A%5B1%5D%2C%22PropertyTypes%22%3A%5B%5D%2C%22RoomsFrom%22%3A%223%22%2C%22RoomsTo%22%3Anull%2C%22FloorSearchType%22%3A0%2C%22LivingSpaceFrom%22%3Anull%2C%22LivingSpaceTo%22%3Anull%2C%22PriceFrom%22%3Anull%2C%22PriceTo%22%3Anull%2C%22ComparisPointsMin%22%3A0%2C%22AdAgeMax%22%3A0%2C%22AdAgeInHoursMax%22%3Anull%2C%22Keyword%22%3A%22%22%2C%22WithImagesOnly%22%3Afalse%2C%22WithPointsOnly%22%3Anull%2C%22Radius%22%3A%221%22%2C%22MinAvailableDate%22%3Anull%2C%22MinChangeDate%22%3A%221753-01-01T00%3A00%3A00%22%2C%22LocationSearchString%22%3A%224053%22%2C%22Sort%22%3A11%2C%22HasBalcony%22%3Afalse%2C%22HasTerrace%22%3Afalse%2C%22HasFireplace%22%3Afalse%2C%22HasDishwasher%22%3Afalse%2C%22HasWashingMachine%22%3Afalse%2C%22HasLift%22%3Afalse%2C%22HasParking%22%3Afalse%2C%22PetsAllowed%22%3Afalse%2C%22MinergieCertified%22%3Afalse%2C%22WheelchairAccessible%22%3Afalse%2C%22LowerLeftLatitude%22%3Anull%2C%22LowerLeftLongitude%22%3Anull%2C%22UpperRightLatitude%22%3Anull%2C%22UpperRightLongitude%22%3Anull%7D&page=[0-10]"],"selectors":[{"delay":2000,"id":"element-scroll","multiple":true,"parentSelectors":["_root"],"selector":"div#__next > div > div > div > div > div > div > div:has(h3)","type":"SelectorElementScroll"},{"delay":0,"id":"link","multiple":false,"parentSelectors":["element-scroll"],"selector":"a","type":"SelectorLink"},{"delay":0,"id":"title","multiple":false,"parentSelectors":["element-scroll"],"regex":"","selector":"h3","type":"SelectorText"}]}

Hope it helps!

[Citylights]

@ViestursWS
Sure further below is the sitemap.
Yes you're assumption (extracting all the properties) in that given specific range would be the aim of it indeed. Hm good point, it might well be that they other elements are only loaded once the user would start scrolling - I haven't thought about that

I will try out your given sitemap later on today - thanks for your support already

Sitemap using performatted text:

{"_id":"suchfiltercomparis","startUrl":["Wohnung mieten in 4053 - Vergleiche 108 Inserate mit comparis.ch a","multiple":true,"delay":0},{"id":"Objekt","type":"SelectorLink","parentSelectors":["_root","pagination"],"selector":"div.css-ctytwt:nth-of-type(n+3) a, div:nth-of-type(n+9) a","multiple":true,"delay":0},{"id":"Titel","type":"SelectorText","parentSelectors":["Objekt"],"selector":"p.css-1e0n6hi","multiple":false,"regex":"","delay":0},{"id":"Beschreibung","type":"SelectorText","parentSelectors":["Objekt"],"selector":"div.css-16i3sck","multiple":false,"regex":"","delay":0},{"id":"Strasse + Nr","type":"SelectorText","parentSelectors":["Objekt"],"selector":"h5","multiple":false,"regex":"","delay":0}]}

[ViestursWS]

[Citylights]

@ViestursWS

Hi again - many many thanks for your wonderful help, it finally worked with your given sitemap
I will now adjust all that I need and then can run it again!

Thanks so much - what a nice community

[Asad]

Sitemap:

{"_id":"comparis","startUrl":["https://www.comparis.ch/immobilien/result/list?requestobject=%7B%22DealType%22%3A10%2C%22SiteId%22%3A0%2C%22RootPropertyTypes%22%3A%5B1%5D%2C%22PropertyTypes%22%3A%5B%5D%2C%22RoomsFrom%22%3A%223%22%2C%22RoomsTo%22%3Anull%2C%22FloorSearchType%22%3A0%2C%22LivingSpaceFrom%22%3Anull%2C%22LivingSpaceTo%22%3Anull%2C%22PriceFrom%22%3Anull%2C%22PriceTo%22%3Anull%2C%22ComparisPointsMin%22%3A0%2C%22AdAgeMax%22%3A0%2C%22AdAgeInHoursMax%22%3Anull%2C%22Keyword%22%3A%22%22%2C%22WithImagesOnly%22%3Afalse%2C%22WithPointsOnly%22%3Anull%2C%22Radius%22%3A%221%22%2C%22MinAvailableDate%22%3Anull%2C%22MinChangeDate%22%3A%221753-01-01T00%3A00%3A00%22%2C%22LocationSearchString%22%3A%224053%22%2C%22Sort%22%3A11%2C%22HasBalcony%22%3Afalse%2C%22HasTerrace%22%3Afalse%2C%22HasFireplace%22%3Afalse%2C%22HasDishwasher%22%3Afalse%2C%22HasWashingMachine%22%3Afalse%2C%22HasLift%22%3Afalse%2C%22HasParking%22%3Afalse%2C%22PetsAllowed%22%3Afalse%2C%22MinergieCertified%22%3Afalse%2C%22WheelchairAccessible%22%3Afalse%2C%22LowerLeftLatitude%22%3Anull%2C%22LowerLeftLongitude%22%3Anull%2C%22UpperRightLatitude%22%3Anull%2C%22UpperRightLongitude%22%3Anull%7D&page=[0-11]"],"selectors":[{"id":"elements","type":"SelectorElement","parentSelectors":["_root"],"selector":"div.css-ctytwt.excbu0j5 ,div.css-1yybzx0.excbu0j5","multiple":true,"delay":0},{"id":"link","type":"SelectorLink","parentSelectors":["elements"],"selector":"a","multiple":false,"delay":0},{"id":"price","type":"SelectorText","parentSelectors":["link"],"selector":".css-eujsoq > font font","multiple":false,"regex":"","delay":0}]}