Scraping TripAdvisor

How to?
#1

Hi, does anybody help me with scraping TripAdvisor comments (full)?
Can you sent me a sitemap?

THNX
Marek

#2

Yea, that's not exactly how it works. If you want help you need to be more specific and tell us which trip adviser pages you want to scrape.

#3

Thank for fast response,
I'd like to scrap "Things to do category" for example that one:
https://www.tripadvisor.co.uk/Attraction_Review-g274812-d287878-Reviews-Rynek-Wroclaw_Lower_Silesia_Province_Southern_Poland.html
Title and full opinions

#4

You want to scrape things to do in Rynek (Details page) or the search it came from

which lists many activities?

#5

There are 1715 reviews in English about Rynek in Wroclaw and I'd like to scrape this 1715 reviews.

#6

You can try this. It seems to crap out after around 700.. Not entirely sure why but I don't have the time to triage. Might need to adjust delay a bit

{"_id":"trip-a-example","startUrl":["https://www.tripadvisor.com/Attraction_Review-g274812-d287878-Reviews-Rynek-Wroclaw_Lower_Silesia_Province_Southern_Poland.html"],"selectors":[{"id":"Element Select - Page Changer","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.rev_wrap","multiple":true,"delay":"3000","clickElementSelector":"div.mobile-more a.nav.next","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueCSSSelector"},{"id":"Name","type":"SelectorText","parentSelectors":["Element Select - Page Changer"],"selector":"div.info_text div:nth-of-type(1)","multiple":false,"regex":"","delay":0},{"id":"Location","type":"SelectorText","parentSelectors":["Element Select - Page Changer"],"selector":"div.userLoc","multiple":false,"regex":"","delay":0},{"id":"Date of stay ","type":"SelectorText","parentSelectors":["Element Select - Page Changer"],"selector":"div.prw_rup.prw_reviews_stay_date_hsx","multiple":false,"regex":"","delay":0},{"id":"Main Quote","type":"SelectorText","parentSelectors":["Element Select - Page Changer"],"selector":"span.noQuotes","multiple":false,"regex":"","delay":0},{"id":"Long-Quote","type":"SelectorText","parentSelectors":["Element Select - Page Changer"],"selector":"p.partial_entry","multiple":false,"regex":"","delay":0},{"id":"Bubble Rating","type":"SelectorText","parentSelectors":["Element Select - Page Changer"],"selector":"span.ui_bubble_rating","multiple":false,"regex":"","delay":0}]}
1 Like
#7

Thank you very much Bret!
It works very well,
I've scrapped all 1716 reviews!
I'd like to thank you in the scientific paper which I will write using this data.
Do you agree?

Marek

1 Like
#8

Sure thing l. Glad I was able to help

#11