Hello,
I've been searching a lot on scraping bookmarked tweets, but I can't seem to find any solution or free tools to do it. I came to find Web Scraper but I think I'm not getting the "Selector" correctly.
Ideally I wanted to export csv with columns of twitter user ID, timestamp, body text, url to media (if attached), url to tweet etc if possible, but at this point if I could get just a list of tweets with body text at minimum that would be great...
Here is the sitemap I have so far, it seems to capture random numbers of tweets (5-6 tweets) from the bottom of the page after it finishes scrolling everything.
{
"_id": "bookmarked_tweets",
"startUrl": ["https://mobile.twitter.com/i/bookmarks"],
"selectors": [{
"id": "Tweet",
"type": "SelectorElementScroll",
"selector": "div:nth-of-type(-n+5000)",
"parentSelectors": ["_root"],
"multiple": true,
"delay": "3000"
}, {
"id": "text",
"type": "SelectorText",
"selector": "article.css-1dbjc4n.r-1loqt21.r-1udh08x.r-o7ynqc.r-1j63xyz",
"parentSelectors": ["Tweet"],
"multiple": true,
"regex": "",
"delay": 0
}]
}
I have so many tweets saved in my bookmark and all I wanted to do is to get a list and holistic view...
Scroll loading seems to be the tricky part when getting the Selector for the "id": "text"...
Your help would be really appreciated! Thank you in advance!