Scraping nested div

How to?
#1

How would I accomplish selecting the following situation:

<div id="test_div">
    <div id="sub_element">Some unimportant text</div>
    A very important multi-line text to scrape
</div>

If I select it with "div#text_div", perform a scrape, it would show in preview the following text:

Some unimportant text
A very important multi-line text to scrape

But when I export it to csv it just shows as:

Some unimportant text

It is missing the text that is inside the #text_div, but outside of any sub div.
jQuery can perform it when selecting a text node of the #text_div, but not in the primary selector. How would I be able to select only "A very important multi-line text to scrape" or at least everything inside #text_div, AND to be able to export it to csv?

#2

Are you sure that it is not exported to csv? Have you tried clicking on cell that should contain this text? It is possible that it is only displaying the first line and the rest expands when you click on the cell. Depends on what software you are using to browse your csv file.

1 Like
#3

You were right. Only the first line was showing. I have eventually scraped that field as HTML and later on trimmed it to size in excel. But now I know it was not necessery and that the scraper did a good job.
Thank you for the help.