How To Scrape List If It Includes Specific Text

How to?
#1

Hello all,

First of all I want to say thank you for this great tool! It's really helped me achieve what I was trying to do.

I'm using it a little differently than most in that I don't have it scrape lots of listings from a page -- instead I plug in 1 page at a time, and have it scrape information from that one page. The reason is that I first have to do some research, and then I'd plug in the info into a spreadsheet -- through Web Scraper I do the research and then it pulls all the info automatically, so I can just copy and paste the results into my spreadsheet. But here is my issue.

The type of pages I'm scraping have a section with a list (ul and li). Let's call it "General Features". I am only interested in whether the page I'm scraping has a specific feature. If it does, then it would be listed in one of the

  • of the list. For example, let's call it "Special Price".

    So it would be something like

      General Features
    • Feature 1
    • Feature 2
    • Special Price
    • Feature 3

        • All I need is basically a YES/NO -- does it have the Special Price feature in the list, or does it not? I don't even need the rest of the data. Is there a way to get that answer somehow?

        And one more (simpler) question -- is there a way for me to load in a few URLs at the same time in some sort of a list? The way I do it right now is that I created a Sitemap with the URL of 1 page, and then I edit that URL and re-run it every time. It would be great if I could gather up 10 URLs and run them all at once.

        Thank you all in advance!

    • #2

    Sure, you can use the CSS contains option. e.g. for standard <ul> it would be:

    ul:contains('Special Price')

    which would return Null if ul doesn't have Special Price. For the inverse condition, you can use.

    ul:not(":contains('Special Price')")