What am I doing wrong?

#1

I'm trying to scrape Zillow for sale by owner and I feel like I'm right there but I can not get the pagination to work like I need since zillow has windows on top of windows. I have the data scrape correct (last time I checked) You just have to have the property details open. Any help would be appreciated. I suggest using a 15000ms page reload to not get captacha.

{"_id":"zillowmain","startUrl":["https://www.zillow.com/"],"selectors":[{"id":"Details","type":"SelectorLink","parentSelectors":["Pageination","search"],"selector":"a.zsg-photo-card-overlay-link","multiple":true,"delay":"2000"},{"id":"Contact","type":"SelectorText","parentSelectors":["Details"],"selector":"div.home-details-listing-provided-by div.zsg-content-item div","multiple":false,"regex":"","delay":"0"},{"id":"Zdom","type":"SelectorText","parentSelectors":["Details"],"selector":" div.fact-group:nth-of-type(n+7)","multiple":false,"regex":"","delay":"0"},{"id":"Address","type":"SelectorText","parentSelectors":["Details"],"selector":"h1.zsg-h1 div","multiple":false,"regex":"","delay":"0"},{"id":"City-State-Zip","type":"SelectorText","parentSelectors":["Details"],"selector":"div.zsg-h2","multiple":false,"regex":"","delay":"0"},{"id":"Open-house","type":"SelectorText","parentSelectors":["Details"],"selector":"li.openhouse-card","multiple":false,"regex":"","delay":0},{"id":"Status","type":"SelectorText","parentSelectors":["Details"],"selector":"div.status","multiple":false,"regex":"","delay":0},{"id":"Price","type":"SelectorText","parentSelectors":["Details"],"selector":"span.value","multiple":false,"regex":"","delay":0},{"id":"Zestimate","type":"SelectorText","parentSelectors":["Details"],"selector":"div.zestimate div","multiple":false,"regex":"","delay":0},{"id":"Pageination","type":"SelectorLink","parentSelectors":["Pageination","duplicate"],"selector":"a.off","multiple":true,"delay":"3000"},{"id":"duplicate","type":"SelectorLink","parentSelectors":["search"],"selector":" a.zsg-photo-card-overlay-link","multiple":true,"delay":"2000"},{"id":"search","type":"SelectorLink","parentSelectors":["_root"],"selector":"ul.znav-links-user li:nth-of-type(2) li:nth-of-type(2) a.znav-link-text","multiple":false,"delay":0}]}

#2

here is the sitemap i made for you, it works.. you can add your own text selectors that you specifically want..

{"_id":"scrapper","startUrl":["https://www.zillow.com/new-harmony-ut/"],"selectors":[{"id":"aea","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"a.zsg-photo-card-overlay-link","multiple":true,"delay":"3000","clickElementSelector":"a.off","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"details","type":"SelectorLink","parentSelectors":["aea"],"selector":"parent","multiple":false,"delay":"0"},{"id":"address1","type":"SelectorText","parentSelectors":["details"],"selector":"div.zsg-h1","multiple":false,"regex":"","delay":0},{"id":"addline2","type":"SelectorText","parentSelectors":["details"],"selector":"div.zsg-h2","multiple":false,"regex":"","delay":0},{"id":"type","type":"SelectorText","parentSelectors":["details"],"selector":"h3.edit-facts-light","multiple":false,"regex":"","delay":0},{"id":"price","type":"SelectorText","parentSelectors":["details"],"selector":"div.price","multiple":false,"regex":"","delay":0},{"id":"listingagentname","type":"SelectorText","parentSelectors":["details"],"selector":"div.home-details-sticky-bal div.cf-cnt-rpt-container.zpro span.cf-rpt-display-name-text","multiple":false,"regex":"","delay":0},{"id":"listingagentphone","type":"SelectorText","parentSelectors":["details"],"selector":"div.home-details-sticky-bal div.cf-cnt-rpt-container.zpro span.cf-phone","multiple":false,"regex":"","delay":0},{"id":"specs","type":"SelectorText","parentSelectors":["details"],"selector":"div.character-count-truncated","multiple":false,"regex":"","delay":0}]}

sample data:
https://sheet.zoho.com/sheet/editor.do?doc=398d93a2eb56bc8b16385058108a48efb8abf8254819b500ef22d30bb952a5ffb8b35a4e9bd593a417ff57714d0abefc2fac6e1472cbd87338a3da01df3f377b

[Album] Imgur

#3

Hey thank you so much your awesome!

When I copied it I had to tweak a few minor things for mine to function but it was easy. If i don't need to do something let me know, but i made the following changes (besides the meta data). Changed aea to single click instead of multiple clicks for pagination (multiple clicks I believe would work for the a.off selector). Details parent selector added underscores so it is parent. I repathed the phone and agent from the previous too .cf-cnt-rpt-container span.cf-phone. May be a regional difference or pro acc difference? also I use span.listing-field instead of the container because I use it for Fsbo and the light box won't include the owners name if listed. but I'm so happy you solved my biggest issue so quick.

#4

also do you know if I can convert the numbers into href="Tel:" for a spreadsheet and how I would do that? Also the URLs possibly but I won't get greedy with those lol.

Thanks!

#5

glad i was of some help :slight_smile:

I'm still learning the basics of this plugin.. I just work based on trail and error method. So complex terminology I wouldn't understand.

#6

I didn't understand your question properly, if you can explain with an image, I think I can help you.

#7

Can you explain these corrections? I'm not following what you did to fix it. Also, how would I modify the set up to run only on a filtered set of properties?