Pagination is not working

How to?
#1

I have tried to use url page brackets, to select the next buttons in several different ways, but for some reason, I am only able to crawl the first page

Url: https://www.century21.pt/consultor-imobiliario/?=&page=1&numberOfElements=12&l=&n=&nodeId=47718&agencyId=

Sitemap:
{"_id":"century21-agemts","startUrl":["https://www.century21.pt/consultor-imobiliario/?=&page=1&numberOfElements=12&l=&n=&nodeId=47718&agencyId="],"selectors":[{"id":"agent-page","type":"SelectorElement","parentSelectors":["_root"],"selector":"article > div","multiple":true,"delay":0},{"id":"name","type":"SelectorText","parentSelectors":["agent-page"],"selector":"h2","multiple":false,"regex":"","delay":0},{"id":"company","type":"SelectorText","parentSelectors":["agent-page"],"selector":"h3","multiple":false,"regex":"","delay":0},{"id":"phone","type":"SelectorText","parentSelectors":["agent-page"],"selector":"h4 a","multiple":false,"regex":"","delay":0},{"id":"next","type":"SelectorLink","parentSelectors":["_root","next"],"selector":"li.next a","multiple":true,"delay":0}]}

can anybody help me out?

Thanks in advance

Help to scrape this pages
#2

Hey there!

What you're trying to do can be easily done with using Element Click selector.

Please try this one out:

{"_id":"century21-agents","startUrl":["https://www.century21.pt/consultor-imobiliario/?=&page=1&numberOfElements=12&l=&n=&nodeId=47718&agencyId="],"selectors":[{"id":"agent-page","type":"SelectorElementClick","selector":"article > div","parentSelectors":["_root"],"multiple":true,"delay":"3000","clickElementSelector":"li.next a","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"name","type":"SelectorText","selector":"h2","parentSelectors":["agent-page"],"multiple":false,"regex":"","delay":0},{"id":"company","type":"SelectorText","selector":"h3","parentSelectors":["agent-page"],"multiple":false,"regex":"","delay":0},{"id":"phone","type":"SelectorText","selector":"h4 a","parentSelectors":["agent-page"],"multiple":false,"regex":"","delay":0}]}

UPD: fixed wrong name

#3

@iconoclast

First of all thanks for your prompt support, but I really need some "Webscrepper for Dummies" support :slight_smile:

I am trying to get the name and the phone number from those agents, but independently of the way I try to do it either I get a black csv file, or it blocks right after the first page.

Could you share with me your hints

Thanks again

#4

Manuel, it didn't give any results due to a mistake I've made inside a sitemap when I was sending it to you. Should work properly now (please re-import it).

Element Click selector acts exactly as an Element selector, but also Clicks. In order to get data from it, you have to add text selectors inside it to pick the data encapsulated/wrapped inside the element.

For more information, I suggest you to view tutorials on main website here:
https://www.webscraper.io/tutorials

Much can be found within docs as well:
https://www.webscraper.io/documentation