How: Flash/Java-content scrapable?

How to?
#1

Hi,

I try to scrape information about houseprices.
For example for the City "Crostau"; Link:

https://atlas.immobilienscout24.de/orte/deutschland/sachsen/bautzen-kreis/crostau#/preisentwicklung

Here you find a flash/Java-picture with the price-information for a couple of years (Q3 2015-Q2 2019). I need the Information for every blue point. By moving mouse over the first blue point you´ll get the Information, that the average price for houses in this town in Q3 2016 was 965 €, in Q4 2016 956€ and so on. These information I want to scrape.

My problem: Is the webscraper actually able to scrape the Informations of these points? I tried it for example with the element click selector...but it doesnt work. It scrapes always the default value of Q2 2019.

Please help me :slight_smile:
Thank you!

#2

...meanwhile ive found the information I need in the source code of the website. I tried it with the element atribute selector...but it doesnt work. Is it possible, to get these information with the element atribute selector? Oder HTML? Getting help would be fine, cause ive no idea to solve this problem :frowning:

webscraper
webscraper.png1304×286 40.3 KB

#3

Ya that is probably the best option. Grab the HTML, then do some post-conversion into the format you want. The data is well structured. To grab that block of HTML, try this scraper:

Type: HTML
Selector: body > script:contains("HOUSE_BUY")

#4

Thx for your help! But it doesnt work neither. Could you have a look on my sitemap? Where is the problem?

{"_id":"immoscout_preisentw","startUrl":["https://atlas.immobilienscout24.de/orte/deutschland/sachsen#/übersicht"],"selectors":[{"id":"kreis","type":"SelectorLink","parentSelectors":["_root"],"selector":"[data-index='3'] a","multiple":false,"delay":0},{"id":"gemeinde","type":"SelectorLink","parentSelectors":["kreis"],"selector":"[data-index='3'] a","multiple":false,"delay":0},{"id":"element_select","type":"SelectorElementClick","parentSelectors":["gemeinde"],"selector":"div.desk-four-fifths","multiple":false,"delay":0,"clickElementSelector":".margin-top-m span.Button__buttonText___2e3DK","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"html","type":"SelectorHTML","parentSelectors":["element_select"],"selector":"parent","multiple":false,"regex":"body>script:contains("HOUSE_BUY")","delay":0}]}

#5

Looks like you placed the selector string in the wrong field. Take a look at this simplified version and modify as needed. It only grabs the title and the chart's HTML.

{"_id":"immoscout_preisentw_v2","startUrl":["https://atlas.immobilienscout24.de/orte/deutschland/sachsen/bautzen-kreis/crostau#/preisentwicklung"],"selectors":[{"id":"html","type":"SelectorHTML","parentSelectors":["_root"],"selector":"body > script:contains(\"HOUSE_BUY\")","multiple":false,"regex":"","delay":0},{"id":"title","type":"SelectorText","parentSelectors":["_root"],"selector":"h2#price-history-detailed-headline","multiple":false,"regex":"","delay":0}]}

#6

Thx again!
Your job works :slight_smile: but it works just with a single link (without my pagination). Within my pagination it doesnt scrap this text. Maybe the problem is the element-click-element? If so, how should i set up the pagination correctly?

{"_id":"immoscout_preisentw","startUrl":["https://atlas.immobilienscout24.de/orte/deutschland/sachsen#/übersicht"],"selectors":[{"id":"kreis","type":"SelectorLink","parentSelectors":["_root"],"selector":"[data-index='3'] a","multiple":false,"delay":0},{"id":"gemeinde","type":"SelectorLink","parentSelectors":["kreis"],"selector":"[data-index='3'] a","multiple":false,"delay":0},{"id":"element_select","type":"SelectorElementClick","parentSelectors":["gemeinde"],"selector":"body > script:contains("HOUSE_BUY")","multiple":false,"delay":0,"clickElementSelector":".margin-top-m span.Button__buttonText___2e3DK","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"html","type":"SelectorHTML","parentSelectors":["element_select"],"selector":"body > script:contains("HOUSE_BUY")","multiple":false,"regex":"","delay":0}]}