Web Scraper version: 0.4.2
Chrome version: 83.0.4103.116
OS: Windows 10 64 Bit
Sitemap:
{"_id":"propwala","startUrl":["https://www.propertywala.com/properties/"],"selectors":[{"id":"city click","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"body","multiple":true,"delay":"2000","clickElementSelector":"select.ddlBox option[value=new_delhi],option[value=noida_uttar_pradesh],option[value=greater_noida_uttar_pradesh],option[value=gurgaon_haryana],option[value=faridabad_haryana],option[value=ghaziabad_uttar_pradesh],option[value=mumbai_maharashtra],option[value=thane_maharashtra],option[value=navi_mumbai_maharashtra],option[value=bangalore_karnataka],option[value=chennai_tamil_nadu],option[value=hyderabad_telangana],option[value=pune_maharashtra],option[value=kolkata_west_bengal],option[value=ahmedabad_gujarat],option[value=chandigarh_city],option[value=mohali_chandigarh],option[value=panchkula_chandigarh],option[value=zirakpur_chandigarh],option[value=north_goa],option[value=south_goa]","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueHTMLText"},{"id":"localities","type":"SelectorText","parentSelectors":["city click"],"selector":"select#searchLocalities option","multiple":true,"regex":"","delay":0},{"id":"city","type":"SelectorText","parentSelectors":["city click"],"selector":"select.ddlBox option[selected]","multiple":false,"regex":"","delay":0}]}
Issue:
The website has a city and a location dropdown field. On selecting a city the values of the location dropdown changes. I am trying to dump this values in the form Cities(1st Column) and Locations(2nd Column). It works fine if I try it for like 2-3 cities but as soon as I increase the amount of data, web scrapper starts to show incorrect city names with cities. And If I run it for all the cities together than it doesn't scrape anything at all. I have tried to increase timeout aswell but that doesn't help.
Extra Info : The City field has around 620 values and the Location field has around 1500 values for each City.