Simple but annoying pagination problem

Hi,

I'm sure this is easy to solve but I'm really struggling with it after my 10th try.... Please help :slight_smile:

The issue is that I have a list of profiles that I want to scrape, every profile has to opened, content scraped and then back to the list, next profile, so on and so on. At the end of the page the "next page" button should be clicked and the whole process repeated: Open profile link -> scrape data -> back to profile list -> open profile link -> ..... -> next page.

The main problem that I cannot solve is that it has to open every profile individually....

Url: http://www.oldtimer.net/oldtimer-firmensuche.html?filter=1&action=search&branditem=&katitem=35&land_id=&stichwort=Stichwort...&ab=Adressen+duchsuchen

Sitemap:
{"_id":"test","startUrl":["http://www.oldtimer.net/oldtimer-firmensuche.html?filter=1&action=search&branditem=&katitem=35&land_id=&stichwort=Stichwort...&ab=Adressen+duchsuchen"],"selectors":[{"id":"name","type":"SelectorLink","selector":"strong.adressName a","parentSelectors":["_root","pages"],"multiple":true,"delay":0},{"id":"address","type":"SelectorText","selector":"p#adress","parentSelectors":["name"],"multiple":false,"regex":"","delay":0},{"id":"phone","type":"SelectorText","selector":"div div div p:nth-of-type(3)","parentSelectors":["name"],"multiple":false,"regex":"","delay":0},{"id":"web","type":"SelectorText","selector":"a.inetStat","parentSelectors":["name"],"multiple":false,"regex":"","delay":0},{"id":"email","type":"SelectorText","selector":"span.textButton","parentSelectors":["name"],"multiple":false,"regex":"","delay":0},{"id":"info","type":"SelectorText","selector":"div div div div div p:nth-of-type(2)","parentSelectors":["name"],"multiple":false,"regex":"","delay":0},{"id":"pages","type":"SelectorLink","selector":"td a","parentSelectors":["_root","pages"],"multiple":false,"delay":0}]}

Hi

I have the same issue I think, I asked here weeks ago and no one answers.... makes the scrapper tool usuless sadly.

Change your "page" selector to td a:contains("weiter") and check multiple option.

Thanks a lot for your help! The part with ("weiter") causes the JSON to be invalid. Is the writing td a:contains("weiter") correct?

Appreciate the input a lot!

If you are pasting it directly into the json then you need to backslash quotes like: (\"weiter\") .