stream.hls: remove Sequence named tuple wrapper#5526
Merged
bastimeyer merged 4 commits intoOct 7, 2023
Conversation
8e23005 to
f2b4b9c
Compare
14 tasks
6f17f38 to
4bb8cc1
Compare
4bb8cc1 to
134dc65
Compare
- Turn `Segment` and `Playlist` into dataclasses, so they can be subclassed - Update `TwitchSegment` and inherit from the default HLS `Segment` - Update `TwitchM3U8Parser` accordingly - Fix segment URL updates in `LTVHLSStreamWorker`
- Add `num` attribute to `Segment` and initialize with `-1`
- Move sequence number logic from `HLSStreamWorker` to `M3U8Parser`:
Update the `num` attribute of each playlist `Segment` after parsing
instead of wrapping it in a new `Sequence` named tuple
- Remove `Sequence` named tuple wrapper
- Update various `Sequence` references in `HLSStream{Writer,Worker}`
- Update generic typevar of `HLSStream{Writer,Worker,Reader}`
- Update HLS subclasses accordingly in various plugins
- Update tests
- Update method and attribute names after the removal of `Sequence`
- Update method signatures
- Change argument/keyword names from sequence(s) to segment(s)
- Remove `sequences` arg from `HLSStreamWorker.process_segments()`
as it's not needed anymore after the sequence number logic was moved
to the parser
- Keep "Sequence" names and log messages for everything that's about
the order of segments in the playlist or the position in the stream
- Update HLS subclasses accordingly in various plugins
- Update tests
- Rename `stream.hls_playlist.load` to `stream.hls_playlist.parse_m3u8`
- Replace `HLSStreamWorker._reload_playlist()`
and `HLSStream._get_variant_playlist()`
with direct `parse_m3u8()` calls, using dependency injection
and the `HLSStream.__parser__` reference
- Turn `M3U8` into a generic class and add covariant type-vars
for the `Segment` and `Playlist` dataclasses
- Turn `M3U8Parser` into a generic class and add covariant type-vars
for the `M3U8` class, as well as for `Segment` and `Playlist`
- Update `M3U8Parser.get_{segment,playlist}()`
and add `M3U8Parser.__{segment,playlist}__` references, to avoid
having to copy data when subclassing `Segment`/`Playlist` dataclasses
- Update `TwitchHLSStream` and related classes:
- Set concrete types for `TwitchM3U8`
- Set concrete types for `TwitchM3U8Parser`
- Set `TwitchM3U8Parser.__m3u8__` to `TwitchM3U8`
- Set `TwitchM3U8Parser.__segment__` to `TwitchSegment`
- Set `TwitchHLSStream.__parser__` to `TwitchM3U8Parser`
Note:
Some typing issues are still unresolved due to shortcomings of current
PEP revisions and typing implementations.
134dc65 to
640b668
Compare
Member
Author
|
Rebased to As explained, some HLS internals got renamed. Here's a quick summary of the relevant stuff which could potentially be accessed by 3rd party plugins:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ref #5498
Turn
SegmentandPlaylistinto dataclasses, so they can be subclassedTwitchSegmentinherit fromSegmentTwitchM3U8Parseraccordinglyltv_lsm_lvplugin (previously cloned theSegmentnamed tuple, now simply updates theuriattribute)Remove
Sequencenamed tuple wrapper, soHLSStream{Writer,Worker}operate on theSegmentdirectlynumattribute toSegmentand initialize with-1HLSStreamWorkertoM3U8ParserSequencereferences inHLSStream{Writer,Worker}HLSStream{Writer,Worker,Reader}I was thinking about adding a
segmentcompatibility property with a deprecation warning toSegment, so third-party plugin implementations which subclassHLSStream{Writer,Worker}don't break when they still implement logic for theSequencewrapper (where the actual segment was accessed via thesegmentattribute), but I decided against it, because (3) changes the names and signatures of some methods anyway. As said in Refactor segmented streams and use a common BaseSegment #5498, these are not public interfaces, so I don't care.Update class attributes and method names/signatures of
HLSStream{Writer,Worker}after the removal ofSequencesequencesarg fromHLSStreamWorker.process_segments()as it's not needed anymore after the sequence number logic was moved to the parserRefactor M3U8 playlist loading and update/fix typing
stream.hls_playlist.loadtostream.hls_playlist.parse_m3u8HLSStreamWorker._reload_playlist()andHLSStream._get_variant_playlist()with directparse_m3u8()calls, using dependency injection and theHLSStream.__parser__referenceM3U8into a generic class and add covariant type-vars for theSegmentandPlaylistdataclassesM3U8Parserinto a generic class and add covariant type-vars for theM3U8class, as well asSegmentandPlaylistThere are still some typing issues left, but I didn't manage to fix it (yet). I believe there are currently some restrictions (PEP 696 - default values for
TypeVars), so unless the whole design of the parser and stuff gets rewritten, this can't be improved further. But I'm not 100% sure. Should be good enough though.I was also thinking about finally moving the HLS and DASH stuff into their own sub-packages, so we don't have multiple modules lying around in the
streampackage. This would even make more sense with my plans for theBaseSegmentstuff, so the HLS and DASH parsers don't have to import the segmented stream implementations, as theBaseSegmentdefinition would have to go there otherwise. Those changes don't belong in this PR though.Btw, I'm not expecting any reviews here, because it requires knowledge of the details of the HLS classes. I will merge this when I feel ready.