Thanks to visit codestin.com
Credit goes to github.com

Skip to content

plugins.youtube: rewrite and fix matchers#5137

Merged
bastimeyer merged 1 commit into
streamlink:masterfrom
bastimeyer:plugins/youtube/rewrite-and-fix-matchers
Feb 6, 2023
Merged

plugins.youtube: rewrite and fix matchers#5137
bastimeyer merged 1 commit into
streamlink:masterfrom
bastimeyer:plugins/youtube/rewrite-and-fix-matchers

Conversation

@bastimeyer

@bastimeyer bastimeyer commented Feb 1, 2023

Copy link
Copy Markdown
Member

Fixes #5136

This splits up the ugly verbose matcher regex into separate matchers for each URL type, and it adds the new @-URLs and /live/video-id URLs.

Before merging, please help me validate all the supported URLs, because splitting up the matchers required some refactoring, and not all self.match[key] keys are available anymore, which can lead to an unintended IndexError being raised.

Btw, this unrelated issue with the muxed adaptive streams and missing audio bitrates needs separate fixing:

$ streamlink 'https://www.youtube.com/embed/aqz-KE-bpKQ'
[cli][info] Found matching plugin youtube for URL https://www.youtube.com/embed/aqz-KE-bpKQ
Traceback (most recent call last):
  File "/home/basti/venv/streamlink-311/bin/streamlink", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/basti/repos/streamlink/src/streamlink_cli/main.py", line 933, in main
    handle_url()
  File "/home/basti/repos/streamlink/src/streamlink_cli/main.py", line 549, in handle_url
    streams = fetch_streams(plugin)
              ^^^^^^^^^^^^^^^^^^^^^
  File "/home/basti/repos/streamlink/src/streamlink_cli/main.py", line 443, in fetch_streams
    return plugin.streams(stream_types=args.stream_types,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/basti/repos/streamlink/src/streamlink/plugin/plugin.py", line 420, in streams
    ostreams = self._get_streams()
               ^^^^^^^^^^^^^^^^^^^
  File "/home/basti/repos/streamlink/src/streamlink/plugins/youtube.py", line 376, in _get_streams
    streams.update(self._create_adaptive_streams(adaptive_formats))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/basti/repos/streamlink/src/streamlink/plugins/youtube.py", line 236, in _create_adaptive_streams
    if best_audio_itag is None or self.adp_audio[itag] > self.adp_audio[best_audio_itag]:
                                  ~~~~~~~~~~~~~~^^^^^^
KeyError: 328

@bastimeyer bastimeyer added the plugin issue A Plugin does not work correctly label Feb 1, 2023
@bastimeyer

bastimeyer commented Feb 1, 2023 •
edited
Loading

Copy link
Copy Markdown
Member Author

Apparently, youtube.com/user/... doesn't seem to work anymore. This should be removed then, I guess. Can anyone confirm (different channels)?

default matcher

$ streamlink 'youtube.com/live/21X5lGlDOfg'
[cli][info] Found matching plugin youtube for URL youtube.com/live/21X5lGlDOfg
Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)

$ streamlink 'youtube.com/v/21X5lGlDOfg'
[cli][info] Found matching plugin youtube for URL youtube.com/v/21X5lGlDOfg
Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)

$ streamlink 'youtube.com/watch?v=21X5lGlDOfg'
[cli][info] Found matching plugin youtube for URL youtube.com/watch?v=21X5lGlDOfg
Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)

channel matcher

$ streamlink 'youtube.com/NASA'
[cli][info] Found matching plugin youtube for URL youtube.com/@NASA
Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)

$ streamlink 'youtube.com/NASA/live'
[cli][info] Found matching plugin youtube for URL youtube.com/@NASA/live
Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)

$ streamlink 'youtube.com/@NASA'
[cli][info] Found matching plugin youtube for URL youtube.com/@NASA
Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)

$ streamlink 'youtube.com/@NASA/live'
[cli][info] Found matching plugin youtube for URL youtube.com/@NASA/live
Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)

$ streamlink 'youtube.com/c/NASA'
[cli][info] Found matching plugin youtube for URL youtube.com/c/NASA
Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)

$ streamlink 'youtube.com/c/NASA/live'
[cli][info] Found matching plugin youtube for URL youtube.com/c/NASA
Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)

$ streamlink 'youtube.com/channel/UCLA_DiR1FfKNvjuUpBHmylQ'
[cli][info] Found matching plugin youtube for URL youtube.com/channel/UCLA_DiR1FfKNvjuUpBHmylQ
Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)

$ streamlink 'youtube.com/channel/UCLA_DiR1FfKNvjuUpBHmylQ/live'
[cli][info] Found matching plugin youtube for URL youtube.com/channel/UCLA_DiR1FfKNvjuUpBHmylQ
Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)

embed matcher

$ streamlink 'youtube.com/embed/21X5lGlDOfg'
[cli][info] Found matching plugin youtube for URL youtube.com/embed/21X5lGlDOfg
Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)

$ streamlink 'youtube.com/embed/live_stream?channel=UCLA_DiR1FfKNvjuUpBHmylQ'
[cli][info] Found matching plugin youtube for URL youtube.com/embed/live_stream?channel=UCLA_DiR1FfKNvjuUpBHmylQ
Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)

shorthand matcher

$ streamlink 'youtu.be/21X5lGlDOfg'
[cli][info] Found matching plugin youtube for URL youtu.be/21X5lGlDOfg
Available streams: 144p (worst), 240p, 360p, 480p, 720p (best)

@bastimeyer bastimeyer mentioned this pull request Feb 1, 2023
Closed
4 tasks
@bastimeyer bastimeyer force-pushed the plugins/youtube/rewrite-and-fix-matchers branch from 6293401 to 1cfc047 Compare February 2, 2023 08:16
@bastimeyer bastimeyer mentioned this pull request Feb 2, 2023
Merged
@bastimeyer bastimeyer force-pushed the plugins/youtube/rewrite-and-fix-matchers branch from 1cfc047 to a3fe96f Compare February 2, 2023 12:36
@gravyboat

gravyboat commented Feb 2, 2023

Copy link
Copy Markdown
Member

Do you want people to test this prior to merge?

@bastimeyer

bastimeyer commented Feb 2, 2023

Copy link
Copy Markdown
Member Author

If you take a look at the diff of the force-pushes, you can see that I already removed the /user/... URLs.

@gravyboat

gravyboat commented Feb 2, 2023

Copy link
Copy Markdown
Member

I was talking about this portion:

Before merging, please help me validate all the supported URLs, because splitting up the matchers required some refactoring, and not all self.match[key] keys are available anymore, which can lead to an unintended IndexError being raised.

since you asked the reporter to take a look in #5136 (comment)

@bastimeyer

bastimeyer commented Feb 2, 2023

Copy link
Copy Markdown
Member Author

Changes should be fine, I guess. If there are no reviews, then so be it, and if there are any issues after merging, then those will need to get fixed afterwards.

@gravyboat

gravyboat commented Feb 3, 2023

Copy link
Copy Markdown
Member

Let's give the original reporter one more day to test then consider this good. I tried a few URLs last night for the various matchers and they were fine.

@bastimeyer bastimeyer merged commit 9fa2d82 into streamlink:master Feb 6, 2023
@bastimeyer bastimeyer deleted the plugins/youtube/rewrite-and-fix-matchers branch February 6, 2023 21:41
@vstavrinov

vstavrinov commented Feb 8, 2023

Copy link
Copy Markdown
Contributor

@bastimeyer :

Apparently, youtube.com/user/... doesn't seem to work anymore. This should be removed then, I guess. Can anyone confirm (different channels)?

These are still working:

https://www.youtube.com/user/Bloomberg
https://www.youtube.com/user/FRANCE24English
https://www.youtube.com/user/channelnewsasia
https://www.youtube.com/user/aljazeeraenglish
https://www.youtube.com/user/twit

This list could be continued. Some /user/ and @ endpoints with the same id point to the same channel while others to different. Some where I saw redirection. What endpoints works depends on channel. So the migration could take a long time if not endless

@bastimeyer

bastimeyer commented Feb 8, 2023

Copy link
Copy Markdown
Member Author

The @-URLs are not mapped to /user/-URLs, because @-URLs had a first-come-first-serve registration phase. I was under the impression that /user/-URLs had been completely removed, because the channels I had tested were not working anymore, but apparently that's not the case for all channels. This is typical YouTube inconsistency. A revert of the removal is trivial though because it only affects the matcher regex and nothing else. I'll open a PR in a bit.

@bastimeyer bastimeyer mentioned this pull request Feb 8, 2023
Merged
@jenlunds

jenlunds commented Feb 18, 2023

Copy link
Copy Markdown

I was under the impression that /user/-URLs had been completely removed, because the channels I had tested were not working anymore, but apparently that's not the case for all channels. This is typical YouTube inconsistency.

How did you decide the correct /user/-URL for the channels you tested? The NASA one for example is not /user/nasa but /user/NASAtelevision. Can you share the URLs of the other channels that seemingly didn't work?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

plugin issue A Plugin does not work correctly

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

plugins.youtube: New URL formats are unsupported

4 participants

@bastimeyer @gravyboat @vstavrinov @jenlunds