I am trying to scrap this site - using search terms - 315 items over 16 pages
I can scrape most of the data I need but want to get website and email addresses
To get this I need to click on each listing scrape that data from individual page
When I add a Click Element and go to the new page and try and select web address or email address I see an error message "Parent does not contain selected element".
How do I fix this ?
Also, on the secondary page in order to get email I need to click "show" - is there an eay way to do this.
My SItemap
{"_id":"solicitors-3","startUrl":["https://solicitors.lawsociety.org.uk/search/results?Pro=False&UmbrellaLegalIssue=LIUMCP&LegalIssue=LILPH&LegalIssue=LILPP&Page=[1-5]"],"selectors":[{"id":"company","multiple":false,"parentSelectors":["boxes-2"],"regex":"","selector":"h2 a","type":"SelectorText"},{"id":"address","multiple":false,"parentSelectors":["boxes-2"],"regex":"","selector":"dt:contains('Address:') + dd","type":"SelectorText"},{"id":"phone","multiple":false,"parentSelectors":["boxes-2"],"regex":"","selector":"dd.hidden-phone","type":"SelectorText"},{"id":"no-of-offices","multiple":false,"parentSelectors":["boxes-2"],"regex":"","selector":".hidden-phone > a strong","type":"SelectorText"},{"id":"no-of-solicitors","multiple":false,"parentSelectors":["boxes-2"],"regex":"","selector":".related strong","type":"SelectorText"},{"id":"boxes-2","multiple":true,"parentSelectors":["_root"],"selector":"section","type":"SelectorElement"},{"clickElementSelector":"h2 a","clickElementUniquenessType":"uniqueText","clickType":"clickOnce","delay":4000,"discardInitialElements":"do-not-discard","id":"click","multiple":false,"parentSelectors":["boxes-2"],"selector":"_parent_","type":"SelectorElementClick"}]}