Pagination not working on semfirm.com

Hi,
i can't get the pagination to work
it only scrapes the 1st page
can you pls help?

Url: http://semfirms.com

Sitemap:
{"_id":"semfirms","startUrl":["https://www.semfirms.com/seo-firms/"],"selectors":[{"id":"company","type":"SelectorText","selector":"h2 a","parentSelectors":["pagination"],"multiple":false,"regex":"","delay":0},{"id":"pagination","type":"SelectorElementClick","selector":"div.listing-page-inner li","parentSelectors":["_root"],"multiple":true,"delay":0,"clickElementSelector":"ul.pagination li:nth-of-type(11) a","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"location","type":"SelectorText","selector":"div.col-sm-4 p","parentSelectors":["pagination"],"multiple":false,"regex":"","delay":0},{"id":"description","type":"SelectorText","selector":"div.company-details p:nth-of-type(1)","parentSelectors":["pagination"],"multiple":false,"regex":"","delay":0},{"id":"services","type":"SelectorText","selector":"p:nth-of-type(2)","parentSelectors":["pagination"],"multiple":false,"regex":"","delay":0},{"id":"link","type":"SelectorLink","selector":"a.go-profile","parentSelectors":["pagination"],"multiple":false,"delay":0}]}

Hi!

You can use pagination brackets [1-...] on this website, as URL contains page number.

I'd recommend you to first set Results per page to 100, so you'll have to scrape just 16 pages.

Please try this url as a sitemap for your scrape:

https://www.semfirms.com/seo-firms/[1-16]/?rec_perpage=100

cool thanks iconoclast!
it worked like charm! :smiley:
do you have any idea how to solve this also? Twitter auto loading

I am currently at my job, i'll try to help everyone when i'm home :slight_smile:

thanks a lot!
you :rocket:
have a good one!

1 Like