{"_id":"jewelers_org","startUrl":["https://www.jewelers.org/buying-jewelry/shop-with-a-member-jeweler/find-a-jeweler?filter_company_searchbox=&filter_company_specialty=&filter_location_searchbox=&filter_location_specialty=&filter_location_distance=5&filter_online_specialty=&r=1&tab=company"],"selectors":[{"id":"company","parentSelectors":["store"],"type":"SelectorText","selector":".padding h1","multiple":false,"regex":""},{"id":"address","parentSelectors":["store"],"type":"SelectorText","selector":"div.retlc","multiple":false,"regex":""},{"id":"specialties","parentSelectors":["store"],"type":"SelectorText","selector":"div.profile-specialties","multiple":false,"regex":""},{"id":"website","parentSelectors":["store"],"type":"SelectorLink","selector":".title a","multiple":false,"linkType":"linkFromHref"},{"id":"established","parentSelectors":["store"],"type":"SelectorText","selector":"div.profile-year","multiple":false,"regex":""},{"id":"pagination","parentSelectors":["_root","pagination"],"paginationType":"auto","type":"SelectorPagination","selector":"div.dataTables_paginate"},{"id":"store","parentSelectors":["pagination"],"type":"SelectorLink","selector":".name a","multiple":true,"linkType":"linkFromHref"},{"id":"website_link","parentSelectors":["store"],"type":"SelectorLink","selector":".retlc a:nth-of-type(1)","multiple":false,"linkType":"linkFromHref"}]}
Hi,
Please try targeting only the 'next' button instead of the whole pagination section.
See reference sitemap below:
{"_id":"jewelers_org","startUrl":["https://www.jewelers.org/buying-jewelry/shop-with-a-member-jeweler/find-a-jeweler?filter_company_searchbox=&filter_company_specialty=&filter_location_searchbox=&filter_location_specialty=&filter_location_distance=5&filter_online_specialty=&r=1&tab=company"],"selectors":[{"id":"company","multiple":false,"parentSelectors":["store"],"regex":"","selector":".padding h1","type":"SelectorText"},{"id":"address","multiple":false,"parentSelectors":["store"],"regex":"","selector":"div.retlc","type":"SelectorText"},{"id":"specialties","multiple":false,"parentSelectors":["store"],"regex":"","selector":"div.profile-specialties","type":"SelectorText"},{"id":"website","linkType":"linkFromHref","multiple":false,"parentSelectors":["store"],"selector":".title a","type":"SelectorLink"},{"id":"established","multiple":false,"parentSelectors":["store"],"regex":"","selector":"div.profile-year","type":"SelectorText"},{"id":"pagination","paginationType":"auto","parentSelectors":["_root","pagination"],"selector":"a.next","type":"SelectorPagination"},{"id":"store","linkType":"linkFromHref","multiple":true,"parentSelectors":["pagination"],"selector":".name a","type":"SelectorLink"},{"id":"website_link","linkType":"linkFromHref","multiple":false,"parentSelectors":["store"],"selector":".retlc a:nth-of-type(1)","type":"SelectorLink"}]}
Thanks Janis. I tried that but still no success. Now it paginates but doesn't scrape anything. Before it scrapped the first page perfectly and then stopped.
The scraper will start returning results only after iterating over all pages, this is by design.