Yelp do not include advertised links

How to?
#1

Hi,
I have this sitemap for yelp pagination

{"_id":"yelppaginationbostonburgers","startUrl":["https://www.yelp.com/search?cflt=burgers&find_loc=02108","https://www.yelp.com/search?cflt=burgers&find_loc=02109"],"selectors":[{"id":"lien-page","type":"SelectorLink","parentSelectors":["_root","pagination"],"selector":"h4 a","multiple":true,"delay":0},{"id":"pagination","type":"SelectorLink","parentSelectors":["_root","pagination"],"selector":"a.next-link","multiple":true,"delay":0}]}

how can i extract only the links under "All results" without extracting the ads locations?
i tried with
h4 a:contains("/biz/)
but it doesnt work

any idea?
thanks!

#2

If you're talking about avoiding the sponsored links, yea I faced that issue. Fortunately, all the sponsored ones have "/adredir?ad_business" in their URL, so you can avoid them with a :not condition:

Type: Link
Selector: div > ul > li div[class*='businessName'] a:not([href*='adredir'])

#3

Thanks a lot leemeng! :rocket:

#4

Hi everyone! Im trying to solve a similar issue and im wondering if anyone else reading this can advise.

Please bare in mind I literally have zero experience using web scraper, this is my first project, so please be very specific about how to follow any solution you may offer. Ive worked out the basics of creating a site map and assigned links to grab data, but my issue is just one step more in terms of complexity.

I am trying to scrape information from product pages. I would like to scrape the first NON-SPONSORED search result that comes up. I noticed that all sponsored URLS contain "sspa?child" . How and where do I add the "not" statement as leemeng mentioned above? ^

which box/setting allows me to add this in?

thank you in advance
Owen

#5

Hard to diagnose without website, but if they are just standard HTML links, you can try something like

Type: Link
Selector: a:not([href*='sspa?child'])