Thanks to visit codestin.com
Credit goes to github.com

Skip to content

FFmpeg validation#4843

Closed
bastimeyer wants to merge 2 commits into
streamlink:masterfrom
bastimeyer:ffmpeg-validation
Closed

FFmpeg validation#4843
bastimeyer wants to merge 2 commits into
streamlink:masterfrom
bastimeyer:ffmpeg-validation

Conversation

@bastimeyer

Copy link
Copy Markdown
Member

We've recently added warning log messages to the FFMPEGMuxer's resolver when FFmpeg could not be found, so that the user can be informed that muxing is not supported. However, there are still a couple of issues left which I think should be addressed:

  1. The chosen/resolved FFmpeg executable does not get validated/verified:
    This is a problem when the user has set an invalid FFmpeg executable, or when it is broken and doesn't work. In case of invalid executables, the output and stream muxing are pretty much undefined, and broken executables lead to empty stream outputs, and unless verbose logging is enabled, no error messages get printed, making the user believe there's an issue with the input stream.
  2. The FFmpeg version does not get logged:
    Logging the FFmpeg version makes debugging issues easier. The recent issue which comes to mind is:
    stream.ffmpegmux: stream with independent segments (audio+video) cannot be remuxed #4825
  3. Certain streams might require a certain FFmpeg version in order to correctly mux streams:
    This PR doesn't add any version checks, but this could be added in the future if it's needed, based on this work.
    hls-multi from arte.tv has seconds of hanging video #4520 (comment)

Opening the PR as a draft, as it's unfinished work.


The first commit implements a generic ProcessOutput utility class for asynchronously spawning a subprocess and reading its stdout/stderr streams.

This is necessary, because using APIs like subprocess.run(capture_output=True) or asyncio.create_subprocess_exec().communicate() read the entire stdout/stderr streams into memory until the process terminates on its own. This is bad when executing unknown binaries. And reading streams of a spawned subprocess in separate threads as well as dealing with timeouts is unnecessarily difficult to implement in regards to making everything thread-safe (especially when writing tests, just see the messy HLS stream tests for example).

Asynchronously iterating output streams and using asyncio.sleep() makes this fairly easy and straightforward to implement, but testing stuff unfortunately requires adding two more dev-requirements:

  1. pytest-asyncio - already installed as a transitive dependency, so no changes here
  2. mock - compatibility backport of the unittest.mock stdlib implementation for py37 (AsyncMock is missing)
    mock was already removed from the dev-requirements in d04767f, and the only gotchas were the args/kwargs properties in Mock.call_args which are missing on py37, but that doesn't justify adding compatibility imports everywhere

Another issue I ran into while writing tests was freezegun, which currently affects the internals of asyncio in its most recent version, and there's an open PR on their repo which will change that. I managed to get the tests working (after lots of issues with asyncio.sleep()), but I'm not sure how the next freezegun release(s) will affect the tests. We'll see...


The second commit adds FFmpeg executable validation and version logging. The validation can be turned off by setting --ffmpeg-no-validation.

Still need to update and fix tests.

I've included some links in the code to the FFmpeg version string definitions, which have been stable for 12 years now.

New utility class for executing a subprocess and asynchronously reading
its stdout and stderr streams line by line while the process is running,
with an optional timeout which kills the process. The output streams
and exit code get passed to callback methods which can control the
process execution and final output result.

This is useful for validating unknown executables, e.g. when querying
a version string output, as it avoids reading the entire stdout/stderr
at once and also avoids waiting for the process to terminate on its own.
and log FFmpeg version output on the debug logging level

WIP: update and fix tests
@bastimeyer

Copy link
Copy Markdown
Member Author

I'll open a new PR in a couple of days once I've finished updating the tests.

@bastimeyer bastimeyer closed this Sep 30, 2022
@bastimeyer bastimeyer deleted the ffmpeg-validation branch September 30, 2022 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant