Simple scrape only works for first page

How to?
#1

Describe the problem.
I am trying to capture some pdfs. Ideally, only a few specific pages. My attempt at getting the scraper to capture the pdfs works for the first page then fails and closes. It seems when I use https://webscraper.io/ the page presented by the site is slightly different, as the pagination buttons have moved to the bottom of the screen. Reading other posts capturing pdfs is a challenge, if that's the case then a list of URLs would be a step forward.

Url: FamilySearch.org [You will need to create a free account to access this page]

Sitemap:
{"_id":"pdf_save","startUrl":["FamilySearch.org a","clickElementUniquenessType":"uniqueText","clickType":"clickOnce","delay":2000,"discardInitialElements":"do-not-discard","id":"pdf_dl","multiple":false,"parentSelectors":["_root","next_page"],"selector":".buttonList a","type":"SelectorElementClick"},{"id":"next_page","paginationType":"auto","parentSelectors":["_root","next_page"],"selector":"span.next","type":"SelectorPagination"}]}

#2

@packwood Hello. Try to use the 'SelectorPagination' - span.next set as a 'parent' to an 'Element attribute' selector - .buttonList a with an 'Attribute name' - href.

Sitemap example:

{"_id":"pdf_save","startUrl":["https://www.familysearch.org/ark:/61903/3:1:3Q9M-CS8J-QWSN-H?cat=2270757"],"selectors":[{"id":"pdf_dl","paginationType":"auto","parentSelectors":["_root","pdf_dl"],"selector":"span.next","type":"SelectorPagination"},{"delay":0,"extractAttribute":"href","id":"image-link","multiple":false,"parentSelectors":["pdf_dl"],"selector":".buttonList a","type":"SelectorElementAttribute"}]}

#3

Thanks, that solves the pagination, but how can I stop it after say 20 pages and still see the results? The only way I seem to be able to stop it is to close the browser tab, but then it doesn't save the URLs captured.

#4

@packwood Hi, yes it is possible to limit the pagination. For example, till it discovers an image which URL contains - 'CS8J-QWSF-C'.

That would be: html:not(:has([href*="CS8J-QWSF-C"])) span.next

Sitemap example:

{"_id":"pdf_save","startUrl":["https://www.familysearch.org/ark:/61903/3:1:3Q9M-CS8J-QWSN-H?cat=2270757"],"selectors":[{"id":"pdf_dl","paginationType":"auto","parentSelectors":["_root","pdf_dl"],"selector":"html:not(:has([href*=\"CS8J-QWSF-C\"])) span.next","type":"SelectorPagination"},{"delay":0,"extractAttribute":"href","id":"image-link","multiple":false,"parentSelectors":["pdf_dl"],"selector":".buttonList a","type":"SelectorElementAttribute"}]}

#5

Thank you. That works perfectly. It's just a case of knowing the syntax of course!

#6

@packwood Yes, indeed. If you are interested to learn more about it, be sure to check the following link: Top 5 CSS Selectors You Need to Know.