I am unable to extract the complete data from the website due to a pagination issue.
The target URL is:
- Search Page: https://www.zaubacorp.com/companysearchresults/NOIDA
-
Sitemap Snippet:
{id: "sitemap code"}
{"_id":"ZAUBA2","startUrl":["https://www.zaubacorp.com/company-by-address/NOIDA"],"selectors":[{"id":"NEXT PAGE","parentSelectors":["_root","NEXT PAGE"],"paginationType":"auto","type":"SelectorPagination","selector":"div:nth-of-type(3) div.dt-layout-end"},{"id":"SITEMAPLINKS","parentSelectors":["_root"],"type":"SelectorLink","selector":"td:nth-of-type(2) a","multiple":true,"version":2,"linkType":"linkFromHref"},{"id":"COMPANY NAME","parentSelectors":["SITEMAPLINKS"],"type":"SelectorText","selector":"#title strong","multiple":false,"regex":"","multipleType":"singleColumn","version":2},{"id":"DIRECTOR LINK","parentSelectors":["SITEMAPLINKS"],"type":"SelectorLink","selector":"li:nth-of-type(2) a","multiple":false,"version":2,"linkType":"linkFromHref"},{"id":"CONTACT INFORMATION","parentSelectors":["SITEMAPLINKS"],"type":"SelectorLink","selector":"li:nth-of-type(3) a","multiple":false,"version":2,"linkType":"linkFromHref"},{"id":"DIRECTOR NAME","parentSelectors":["DIRECTOR LINK"],"type":"SelectorText","selector":"div.table-responsive:nth-of-type(1) .table-condensed tr:nth-of-type(1) td:nth-of-type(2)","multiple":false,"regex":"","multipleType":"singleColumn","version":2},{"id":"EMAILID","parentSelectors":["CONTACT INFORMATION"],"type":"SelectorText","selector":".col-md-6 span:nth-of-type(1)","multiple":false,"regex":"","multipleType":"singleColumn","version":2},{"id":"WEBSITE","parentSelectors":["CONTACT INFORMATION"],"type":"SelectorText","selector":".col-md-6 span:nth-of-type(2)","multiple":false,"regex":"","multipleType":"singleColumn","version":2},{"id":"ADDRESS","parentSelectors":["CONTACT INFORMATION"],"type":"SelectorText","selector":"span:nth-of-type(4)","multiple":false,"regex":"","multipleType":"singleColumn","version":2}]}
The website loads additional results across multiple pages, but the pagination is either dynamically loaded, hidden behind JavaScript, or not easily accessible to my scraper. As a result, my extraction script only retrieves data from the first page and does not navigate to subsequent pages. I need a solution that allows me to detect, access, and iterate through all paginated pages to extract the full dataset.