I have a website full of devotional pages and I need to scrape the meta title and description from each page. How do I do that consistently across pages that were created under different people and have different tags for metadata in each year section?
Url: President Henry J. Eyring - “Prophets, Plans, and Persistence” - BYU-Idaho (an example of a page I need the title and description from)
Speeches (the main page they are all connected to. I need to click the little video icon for each devotional to get to the page with the title)
Sitemap:
{"_id":"devo","startUrl":["Error option:not(:contains("All Years"))","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":1200,"discardInitialElements":"discard-when-click-element-exists","multiple":true,"selector":"body"},{"id":"wrapper","parentSelectors":["years"],"type":"SelectorElement","selector":"li.ng-scope","multiple":true},{"id":"speaker","parentSelectors":["wrapper"],"type":"SelectorText","selector":"div.speaker","multiple":false,"regex":""},{"id":"title","parentSelectors":["wrapper"],"type":"SelectorText","selector":"div.speechTitle","multiple":false,"regex":""},{"id":"date","parentSelectors":["wrapper"],"type":"SelectorText","selector":"div.speechDate","multiple":false,"regex":""}]}