Exclude scraping child div

How to?
#1

I want to scrape only the original text, not any quoted text from each post in this forum. For example, I only want to scrape the following from the first post on the URL below:

That was not Mr. Bogle's position. I don't believe he ever said that US investors would be richly rewarded by shunning int'l diversification. He often said that ex-US was unnecessary.

Int'l diversification started becoming popular in the early 1990's.

I do not want the following:

splat789 wrote: US investors who ignored the siren song of the Ex-US stock hype and stayed the course with US stock have been richly rewarded just as Jack Bogle said they would be.

I can't figure out how to exclude the quoted text (the blockquote). I have tried :not(blockquote) and > div, but neither of these work.

How can I do this?

Url: International (Non-US) versus US Equities (The "Arguments") - Page 2 - Bogleheads.org

{"_id":"bogleheads3","startUrl":["International (Non-US) versus US Equities (The "Arguments") - Page 2 - Bogleheads.orgparent:not(blockquote)","type":"SelectorText"}]}

#2

This one is tricky, because they are both part of the same div. It's probably easier to just block the unwanted bits first with an ad blocker such as uBlock Origin or Adblock Plus. They both use CSS selectors, similar to WS. Exanples below with uBlock Origin.

image
image1248×808 61 KB

image
image938×446 29 KB

image
image1091×823 73.6 KB

#3

Thanks for the reply. I was able to block the unwanted elements using uBlock Origin, but when I scrape the site, those unwanted elements are still getting scraped. It's almost as if webscraper is ignoring those filtering rules from uBlock Origin. Do you know why that is or how to fix it?

#4

Hmm that didn't work as I expected. Perhaps because WS is reading the HTML before the adblocker does.

Maybe you can try scraping both the post and its blockquote at the same time (two scrapers). Then with post processing, you would search for the blockquote text within the post text, and replace with nothing (delete). Excel might be able to do this, or you can try Python.