Imposible scrapping this page with Pagination

How to?
#1

Hello, I try to scrap data on this link. I need to paginate this

Url: https://www.inforesidencias.com/centros/buscador/residencias/madrid

Sitemap:

{"_id":"residencias_madrid","startUrl":["https://www.inforesidencias.com/centros/buscador/residencias/madrid"],"selectors":[{"id":"nombre","type":"SelectorText","parentSelectors":["Residencias"],"selector":"h2.span7 a","multiple":false,"regex":"","delay":0},{"id":"direccion","type":"SelectorText","parentSelectors":["Residencias"],"selector":"address","multiple":false,"regex":"","delay":0},{"id":"Residencias","type":"SelectorElement","parentSelectors":["_root"],"selector":"div.result","multiple":true,"delay":"0"},{"id":"clickTel","type":"SelectorElementClick","parentSelectors":["Residencias"],"selector":"address","multiple":false,"delay":0,"clickElementSelector":"span.verTelefono a","clickType":"clickOnce","discardInitialElements":true,"clickElementUniquenessType":"uniqueText"},{"id":"tel","type":"SelectorText","parentSelectors":["clickTel"],"selector":"span.verTelefono","multiple":true,"regex":"","delay":0},{"id":"clickWeb","type":"SelectorElementClick","parentSelectors":["Residencias"],"selector":"address","multiple":false,"delay":0,"clickElementSelector":"span.verWeb a","clickType":"clickOnce","discardInitialElements":true,"clickElementUniquenessType":"uniqueText"},{"id":"web","type":"SelectorText","parentSelectors":["clickWeb"],"selector":"span.verWeb","multiple":false,"regex":"","delay":0}]}

Thanks

#2

Stumped here @iconoclast?

#3

I need your help @iconoclast

Please...

#4

Here is a work around. Download the chrome extension Instant Data Scraper for chrome. In about 98 seconds in will scrape all results (careful, it loops the last page over and over.) It won't click on any links or crawl in anyway but it will get you name, url and address.

You can then use the URL to make a giant sitemap

Here are the results: https://goo.gl/WZGhxb

If you go to sheet 2, i've compiled all the URLS into a sitemap for webscraper.io. You just need to import, navigate to one of the URL and build the selectors.

#5

Uh Sorry guys someone must've called me on the phone when I was looking at this topic.
Let me know If help is still needed.

#6

No workaround is needed.
It seems it's important to have at least 3 seconds for pagination to cycle through, in case you have bad connection you may increase delay for Element Click.

Your sitemap:

{"_id":"residencias_madrid2","startUrl":["https://www.inforesidencias.com/centros/buscador/residencias/madrid"],"selectors":[{"id":"clickTel","type":"SelectorElementClick","parentSelectors":["Links2Places"],"selector":"span.verTelefono","multiple":false,"delay":0,"clickElementSelector":"span.verTelefono","clickType":"clickOnce","discardInitialElements":true,"clickElementUniquenessType":"uniqueText"},{"id":"tel","type":"SelectorText","parentSelectors":["clickTel"],"selector":"_parent_","multiple":false,"regex":"","delay":0},{"id":"Pagination","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.span9:nth-of-type(3)","multiple":true,"delay":"3000","clickElementSelector":"a:contains('Siguiente »')","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"Links2Places","type":"SelectorLink","parentSelectors":["Pagination"],"selector":"h2.span a","multiple":true,"delay":""},{"id":"clickWeb","type":"SelectorElementClick","parentSelectors":["Links2Places"],"selector":"span#verWeb.verWeb","multiple":false,"delay":0,"clickElementSelector":"span#verWeb.verWeb a","clickType":"clickOnce","discardInitialElements":true,"clickElementUniquenessType":"uniqueText"},{"id":"website","type":"SelectorText","parentSelectors":["clickWeb"],"selector":"a","multiple":false,"regex":"","delay":0},{"id":"address","type":"SelectorText","parentSelectors":["Links2Places"],"selector":"address","multiple":false,"regex":"","delay":0}]}
#7

Really? A delay? I was racking my brain on why it wasn't working and it was all a delay issue.

1 Like
#8

Thank you, you are superheroes