Hi all!
I'm working on a project for my corpus class and getting extremely confused with how to scrape websites.
The website I'm scraping is https://finance.yahoo.com/quote/S/community?p=S
I'm trying to just get the texts from the comments and have it scroll down/click to at least 6 months of comments. I'm getting random comments from a wide variety of dates and I wanted to have it sorted out chronologically. I've gotten a decent amount of comments but need to know how to click on a dropdown list and sort by newest reactions.
I also have to explain this in front of a panel, so any explanation helps!
Thanks!
{"_id":"sprint","startUrl":["https://finance.yahoo.com/quote/S/community?p=S"],"selectors":[{"id":"comment","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"li.comment","multiple":true,"delay":0,"clickElementSelector":"button.Ff\(ss\).Mb\(20px\) span","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"text","type":"SelectorText","parentSelectors":["comment"],"selector":"div.C\(\$c-fuji-grey-l\)","multiple":true,"regex":"","delay":0},{"id":"time","type":"SelectorText","parentSelectors":["comment"],"selector":"span.C\(\$c-fuji-grey-g\) span","multiple":false,"regex":"","delay":0}]}