Hi,
I'm using the free Chrome extension to parse content from this URL
root page: Alternative Investment Funds Laws and Regulations Report 2024
test page for the data wrapper: Alternative Investment Funds Laws and Regulations Report 2024 Andorra
The root has a list of countries through which I iterate. It is in those pages where I try to get the data from. Basically, those pages have multiple sections, each of them with multiple questions and their answers. It was a bit tricky to extract the data because of the html structure but I managed to do it.
My problem is.
- If I use text selectors with the Multiple checkbox enabled, all data from all countries (linked pages) is identified but each field ends up in a different row when I export the data.
i.e.:
category in one row
question 1 in next row
paragraph 1 (first DIV) of answer 1 in next row
paragraph 2 (second DIV) of answer 1 in next row
... etc
This is the sitemap
{"_id":"ICLG_New","startUrl":["Legal Guides, Business Reports and Events | ICLG a","multiple":true,"linkType":"linkFromHref"},{"id":"Data","parentSelectors":["CountryList"],"type":"SelectorText","selector":"h3.chapterh2, .question strong, .question + p, .question + p ~ p:not(.question), .question + ul, .question + p ~ ul, .question + ol, .question + p ~ ol","multiple":true,"regex":""}]}
- If I configure a wrapper of type Element with the Multiple checkbox enabled, and then the section/question/answer fields (Text type) with the Multiple checkbox disabled, then the only data found (and exported) is the very first pair of question/answer. All other Q&A pairs are not identified.
This is the sitemap
{"_id":"ICLGv3","startUrl":["https://iclg.com/practice-areas/data-protection-laws-and-regulations"],"selectors":[{"id":"CountryList","parentSelectors":["_root"],"type":"SelectorLink","selector":".country-list a","multiple":true,"linkType":"linkFromHref"},{"id":"wrapper","parentSelectors":["CountryList"],"type":"SelectorElement","selector":"article.chapter","multiple":true},{"id":"Section","parentSelectors":["wrapper"],"type":"SelectorText","selector":"h3.chapterh2","multiple":false,"regex":""},{"id":"Question","parentSelectors":["wrapper"],"type":"SelectorText","selector":".question strong","multiple":false,"regex":""},{"id":"Answer","parentSelectors":["wrapper"],"type":"SelectorText","selector":".question + p, .question + p ~ p:not(.question), .question + ul, .question + p ~ ul, .question + ol, .question + p ~ ol","multiple":false,"regex":""}]}
I have also played around with the Grouped selector type, following suggestions from some posts in this forum, but I haven't been able to make it work. Kind of works, but It creates huge texts that force Excel to fix the file when you open it, and the way data is reported there is not manageable at all.
My expectation is to have a data export where I can see each Q&A pair in a single row.
Does someone have an idea on how to adapt my sitemaps or my selectors to make it work?.
Thanks in advance for your time.