Scraping youtube still possible?

How to?
#1

Hi there,

First of all, I am a newbie that has run trough all tutorials found and could not find the solution. I have almost no technical background so I am struggling a lot.

Describe the problem.
I would like to extract the url, title and description box of all the videos from a youtube channel. I have seen in the forum some solution for the scroll problem but nothing seems to work anymore. Nothng seems to work anymore. I also can't figure how to get the description directly on the html, without clicking. I would really appreciate some help from the great minds in this forum, I have been trying that for 4 hours and I feel a little bit silly.

Thanks for the support

Url: JOYCA - YouTube

Sitemap:
{"_id":"another-fail","startUrl":["https://www.youtube.com/@Joyca/videos"],"selectors":[{"id":"videolink","parentSelectors":["_root"],"type":"SelectorLink","selector":"div#contents.ytd-rich-grid-renderer","multiple":true},{"id":"showmore","parentSelectors":["videolink"],"type":"SelectorElementClick","clickElementSelector":"yt-formatted-string.more-button","clickElementUniquenessType":"uniqueText","clickType":"clickOnce","delay":3000,"discardInitialElements":"do-not-discard","multiple":false,"selector":"yt-formatted-string.more-button"},{"id":"description","parentSelectors":["videolink"],"type":"SelectorElementAttribute","selector":"#attributed-snippet-text > span","multiple":false,"extractAttribute":"role"},{"id":"title","parentSelectors":["videolink"],"type":"SelectorText","selector":"h1 yt-formatted-string.ytd-watch-metadata","multiple":false,"regex":""}]}

#2

There is a known limitation in Chrome where Google limits all extensions' ability to get data from google's websites (Chrome and YouTube are both owned by google). This probably affects Web Scraper too.

You can probably get around this by using Firefox and the FF version of Web Scraper:

1 Like
#3

Thanks for your help!

Unfortunatly, it still not work. I guess my sitemap is just bad.. I did try to do it with chatgpt but it's a big fail again..

Do you have an idea what's the problem with that sitemap?

{"_id":"youtube_videos",
"startUrl":["JOYCA - YouTube"],
"selectors":[
{"id":"video_container",
"type":"SelectorElementScroll",
"parentSelectors":["_root"],
"selector":"ytd-grid-video-renderer",
"multiple":true,
"delay":"2000",
"scrollElement":"body",
"scrollDelay":"2000",
"terminateScrolling":true,
"timeout":"10000"},
{"id":"title",
"type":"SelectorText",
"parentSelectors":["video_container"],
"selector":"#video-title",
"regex":"",
"delay":0},
{"id":"url",
"type":"SelectorLink",
"parentSelectors":["video_container"],
"selector":"#video-title",
"multiple":false,
"delay":0},
{"id":"description",
"type":"SelectorText",
"parentSelectors":["video_container"],
"selector":"yt-formatted-string#description-text",
"regex":"",
"delay":0}
]}

Thx u so much

#4

@random Hi, you should be able to extract the video links, title & description using the following sitemap:

{"_id":"youtube-test","startUrl":["https://www.youtube.com/@Joyca/videos"],"selectors":[{"id":"videolink","parentSelectors":["_root"],"type":"SelectorLink","selector":"a.focus-on-expand.ytd-rich-grid-media","multiple":true},{"id":"showmore","parentSelectors":["videolink"],"type":"SelectorElementClick","clickElementSelector":"tp-yt-paper-button#expand","clickElementUniquenessType":"uniqueText","clickType":"clickOnce","delay":3000,"discardInitialElements":"do-not-discard","multiple":false,"selector":"body"},{"id":"description","parentSelectors":["videolink"],"type":"SelectorText","selector":"ytd-text-inline-expander","multiple":false,"regex":""},{"id":"title","parentSelectors":["videolink"],"type":"SelectorText","selector":"h1 yt-formatted-string.ytd-watch-metadata","multiple":false,"regex":""}]}

1 Like
#5

it work perfectly thank you so much!

you are awesome!