Tripadvisor categories on the left side

How to?
#1

Hi,
im trying to scrape all the categories on the left side of tripadvisor excluding restaurants.
So i will have to click on "show more" and select all the categories here
https://monosnap.com/file/3zRvsfCMoI8zlkN6BNd9l91AGZsGGm

Url: https://www.tripadvisor.com/Restaurants-g60878-Seattle_Washington.html

i tried this, but the pagination is not working, the categories i added are correctly selected, but then the pop up shuts down

{"_id":"tripadvisorpaginationwithcat","startUrl":["https://www.tripadvisor.com/Restaurants-g60878-Seattle_Washington.html"],"selectors":[{"id":"deselectrestaurants","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"label[for='checkbox_3']","multiple":false,"delay":2000,"clickElementSelector":"label[for='checkbox_3']","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"showmore","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div._3m8WY12V:nth-of-type(2) span._3ncH7U-p","multiple":false,"delay":2000,"clickElementSelector":"div._3m8WY12V:nth-of-type(2) span._3ncH7U-p","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"quickbites","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"label[for='checkbox_4']","multiple":false,"delay":2000,"clickElementSelector":"label[for='checkbox_4']","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"dessert","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"label[for='checkbox_5']","multiple":false,"delay":2000,"clickElementSelector":"label[for='checkbox_5']","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"coffee","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"label[for='checkbox_6']","multiple":false,"delay":2000,"clickElementSelector":"label[for='checkbox_6']","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"name","type":"SelectorLink","parentSelectors":["_root"],"selector":"a._15_ydu6b","multiple":true,"delay":0},{"id":"pagination","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"a.nav","multiple":true,"delay":2000,"clickElementSelector":"a.nav","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"}]}

any idea?
thanks

1 Like
#2

Hi @eldoland. Click would work but you had chosen "click once" instead of "click more"... Anyway you can actually get through with the link selector here which is a bit friendlier approach.

Pagination
Pagination1994×220 49.4 KB

Here's my version:
{"_id":"tripadvisorpaginationwithcat","startUrl":["https://www.tripadvisor.com/Restaurants-g60878-Seattle_Washington.html"],"selectors":[{"id":"deselectrestaurants","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"label[for='checkbox_3']","multiple":false,"delay":2000,"clickElementSelector":"label[for='checkbox_3']","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"showmore","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div._3m8WY12V:nth-of-type(2) span._3ncH7U-p","multiple":false,"delay":2000,"clickElementSelector":"div._3m8WY12V:nth-of-type(2) span._3ncH7U-p","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"quickbites","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"label[for='checkbox_4']","multiple":false,"delay":2000,"clickElementSelector":"label[for='checkbox_4']","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"dessert","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"label[for='checkbox_5']","multiple":false,"delay":2000,"clickElementSelector":"label[for='checkbox_5']","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"coffee","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"label[for='checkbox_6']","multiple":false,"delay":2000,"clickElementSelector":"label[for='checkbox_6']","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"name","type":"SelectorLink","parentSelectors":["wrapper"],"selector":"a._15_ydu6b","multiple":true,"delay":0},{"id":"pagination","type":"SelectorLink","parentSelectors":["_root","pagination"],"selector":"a:contains(\"Next\")","multiple":true,"delay":0},{"id":"wrapper","type":"SelectorElement","parentSelectors":["_root","pagination"],"selector":"[data-test-target=\"restaurants-list\"] > div:not(:contains(\"Sponsored\"))","multiple":true,"delay":0}]}

Hope it helps.

#3

For Tripadvisor, you can sniff out the URL in the Network tab to pre-select types, for example, this URL will load with Dessert, Coffee n Tea and Bakeries types already selected:

2021-03-23_051942
2021-03-23_0519421294×744 194 KB

right-click, Copy -> Copy link address
You would need to edit the URL first and remove &ajax=1 or you will get a weird-looking page. Then you can use it as Start URL.

#4

Hi @ViestursWS thanks a lot!
i tried your sitemap, it correctly select the categories, however, when it starts scraping it goes back to "restaurants" with all the results
https://monosnap.com/file/lfhOCgNeqyFU4uhPfE845XKqndkvjY

and yes thanks @leemeng
i checked on the inspector and actually i was able to find all the links for each and every category

dessert THE BEST Dessert in Seattle - Tripadvisor
quick bites THE 10 BEST Restaurants in Seattle - Updated March 2021 - Tripadvisor
offe & tea THE BEST Coffee & Tea in Seattle - Tripadvisor
bakeries THE 10 BEST Bakeries in Seattle - Tripadvisor
bars & pubs THE BEST Bars & Pubs in Seattle - Tripadvisor
local chef THE BEST Restaurants in Seattle - Updated March 2021 - Tripadvisor
specialty food THE 10 BEST Restaurants in Seattle - Updated March 2021 - Tripadvisor
delivery only THE BEST Restaurants in Seattle - Updated March 2021 - Tripadvisor

thanks guys, really appreciate it!
:smiley: