[video_player] Adds audio track metadata fetching and audio track selection feature #9925

nateshmbhat · 2025-08-29T08:09:37Z

Description

This PR adds comprehensive audio track retrieval and selection support to the video_player package, enabling developers to access detailed information about available audio tracks and switch between them during playback.

Changes Made

Core Features

Added VideoAudioTrack model with comprehensive metadata fields: id, label, language, isSelected, bitrate, sampleRate, channelCount, codec
Added getAudioTracks() method to retrieve all available audio tracks with real metadata
Added selectAudioTrack() method to switch between audio tracks during playback
Updated VideoPlayerController to expose audio track functionality

Platform Implementations

Android:
- Real metadata extraction using ExoPlayer's getCurrentTracks() API
- Robust track selection using TrackSelectionOverride with proper error handling
- Support for multiple audio formats (AAC, AC3, EAC3, MP3, etc.)
iOS:
- Metadata extraction from AVFoundation using AVAssetTrack for regular videos
- HLS stream support using AVMediaSelectionGroup for adaptive streams
- Proper track selection for both asset tracks and media selection options

Technical Infrastructure

Updated Pigeon interfaces for both Android and iOS with new data structures:
- AudioTrackMessage, ExoPlayerAudioTrackData, AssetAudioTrackData, MediaSelectionAudioTrackData, NativeAudioTrackData
Enhanced platform interface with new methods and data classes
Has native unit tests for both Android and iOS platforms
Created demo screen showcasing audio track functionality with interactive UI

Demo Features

Interactive video player with audio track selection
Real-time metadata display (bitrate, sample rate, channels, codec)
Support for multiple video sources including HLS streams
Visual indicators for currently selected tracks

Related Issues

#59437 - Add audio track metadata support to video_player

Testing

Added native unit tests for both Android and iOS
Tested with various video formats and HLS streams
Verified backward compatibility with existing functionality
Demo screen widget for manual testing and validation

Breaking Changes

None - all changes are additive and backward compatible.

Pre-Review Checklist

…roid

…lity

gemini-code-assist

Code Review

This pull request introduces a valuable feature for fetching and selecting audio tracks in the video_player package. The changes are comprehensive, touching the platform interface, Android (ExoPlayer) and iOS (AVFoundation) implementations, and adding a new demo screen.

My review has identified several critical and high-severity issues that should be addressed:

The new native unit tests for both Android and iOS appear to be broken due to type mismatches and incorrect method calls, and will not compile in their current state.
The Dart implementation code for both Android and iOS contains unsafe null assertions on data coming from the native side, which could lead to runtime crashes.
The Android implementation has been updated to use a milestone version of Gradle, which is not recommended for production packages.
The new audio track selection feature has not been implemented for the web platform, which will lead to UnimplementedError on web.
There is a minor issue in the new example app where context is used after an await without checking if the widget is still mounted.

Addressing these points will significantly improve the robustness and completeness of this new feature.

...deo_player_android/android/src/test/java/io/flutter/plugins/videoplayer/AudioTracksTest.java

packages/video_player/video_player_avfoundation/darwin/RunnerTests/AudioTracksTests.m

packages/video_player/video_player_android/android/gradle/wrapper/gradle-wrapper.properties

packages/video_player/video_player_android/lib/src/android_video_player.dart

packages/video_player/video_player_avfoundation/lib/src/avfoundation_video_player.dart

packages/video_player/video_player/example/lib/audio_tracks_demo.dart

…oPlayer async updates

… fix format builders

…o develop

nateshmbhat · 2025-08-30T12:33:22Z

I updated native andorid tests and ran the tests and verified that all native android tests are passing.

…deoPlayerTests.m

…tracks demo

fix(ios): fixed tests

nateshmbhat · 2025-09-01T07:04:20Z

@stuartmorgan-g @ash2moon can you please comment on this PR and lemme know if there are any concerns ?

nateshmbhat · 2025-09-09T07:44:05Z

any update on this PR ?

stuartmorgan-g · 2025-09-09T11:44:02Z

From triage: ping @tarrinneal @ash2moon @LongCatIsLooong @hellohuanlin for review

@nateshmbhat Apologies for the delay; things have been unusually busy recently. A couple of high-level notes while you are waiting for full reviews:

The video_player_web changes can all be reverted; it looks like you probably ran the auto-formatter in the package without running pub get, so applied the wrong formatter version.
It appears that one of the new tests is crashing on macOS.
If this feature isn't going to be implemented on all platforms, it needs to have a support query so clients know if they can call it.

LongCatIsLooong · 2025-09-08T20:02:45Z

packages/video_player/video_player_avfoundation/CHANGELOG.md

@@ -1,5 +1,6 @@
 ## NEXT

+* Implements `getAudioTracks()` and `selectAudioTrack()` methods for iOS/macOS using AVFoundation.


uber nit: the package is called video_player_avfoundation so I think the ios/macOS and AVFoundation part seems redundant.

LongCatIsLooong · 2025-09-08T20:06:07Z

packages/video_player/video_player/example/lib/audio_tracks_demo.dart

+
+  // Sample video URLs with multiple audio tracks
+  final List<String> _sampleVideos = <String>[
+    'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',


Would it be possible to use existing videos hosted in the flutter/samples repo (see the other demos), or do they not have tracks? I'm not sure what licenses these samples use.

This can be a const right?

LongCatIsLooong · 2025-09-08T20:08:16Z

packages/video_player/video_player/example/lib/audio_tracks_demo.dart

+    try {
+      await _controller?.dispose();
+
+      _controller = VideoPlayerController.networkUrl(


nit: here and everywhere (especially in _loadAudioTracks): consider using a local var so you don't have to use ! to force unwrap the controller every time.

LongCatIsLooong · 2025-09-08T20:14:03Z

packages/video_player/video_player/example/lib/audio_tracks_demo.dart

+
+      // Add a small delay to allow ExoPlayer to process the track selection change
+      // This is needed because ExoPlayer's track selection update is asynchronous
+      await Future<void>.delayed(const Duration(milliseconds: 100));


Hmm this could be a major source of flakiness. Shouldn't selectAudioTrack completes the future only after the new tracks becomes ready?

Also, a major source of user errors if this isn't documented (still I would prefer that we hide the ugliness within our implementation so the user doesn't have to worry about that).

LongCatIsLooong · 2025-09-08T20:16:49Z

packages/video_player/video_player/example/lib/audio_tracks_demo.dart

+      // Reload tracks to update selection status
+      await _loadAudioTracks();
+
+      if (!mounted) {


This check is also needed whenever you want to call setState I think?

LongCatIsLooong · 2025-09-12T18:42:25Z

...undation/darwin/video_player_avfoundation/Sources/video_player_avfoundation/FVPVideoPlayer.m

+      AVMediaSelectionOption *option = audioGroup.options[i];
+
+      // Skip nil options
+      if (!option || [option isKindOfClass:[NSNull class]]) {


nit: the first nil check doesn't seem to be needed?

LongCatIsLooong · 2025-09-12T18:43:17Z

...undation/darwin/video_player_avfoundation/Sources/video_player_avfoundation/FVPVideoPlayer.m

+
+      NSString *displayName = option.displayName;
+      if (!displayName || displayName.length == 0) {
+        displayName = [NSString stringWithFormat:@"Audio Track %ld", (long)(i + 1)];


Why does the implementation have to provide a fallback name?

LongCatIsLooong · 2025-09-12T18:44:05Z

...undation/darwin/video_player_avfoundation/Sources/video_player_avfoundation/FVPVideoPlayer.m

+
+      NSString *languageCode = @"und";
+      if (option.locale) {
+        languageCode = option.locale.languageCode ?: @"und";


nit: NSString* languageCode = option.locale.languageCode ?: @"und";, without the if?

LongCatIsLooong · 2025-09-12T18:47:08Z

...undation/darwin/video_player_avfoundation/Sources/video_player_avfoundation/FVPVideoPlayer.m

+      }
+
+      NSString *commonMetadataTitle = nil;
+      for (AVMetadataItem *item in option.commonMetadata) {


the API documentation recommends filtering using a different method: https://developer.apple.com/documentation/avfoundation/avmediaselectionoption/commonmetadata?language=objc#Discussion

LongCatIsLooong · 2025-09-12T18:57:55Z

...undation/darwin/video_player_avfoundation/Sources/video_player_avfoundation/FVPVideoPlayer.m

+
+    // Only attempt format description parsing in production (non-test) environments
+    // Skip entirely if we detect any mock objects or test environment indicators
+    NSString *trackClassName = NSStringFromClass([track class]);


Why is the behavior different in production and tests? I think that defeats the purpose of testing if the logic is different for tests?

nateshmbhat added 4 commits August 29, 2025 11:32

feat(video_player): add audio track selection support for iOS and And…

c8b0071

…roid

feat(android): implement audio track selection in video player

4e4dc8c

feat(video_player): Format Entire Codebase

25de26c

feat(video_player): add audio track selection and retrieval functiona…

31f9030

…lity

nateshmbhat requested review from LongCatIsLooong, hellohuanlin, ash2moon, ditman and tarrinneal as code owners August 29, 2025 08:09

github-actions bot added platform-android platform-ios platform-macos platform-web p: video_player labels Aug 29, 2025

nateshmbhat mentioned this pull request Aug 29, 2025

[video_player] Add audio track metadata support (bitrate, sample rate, channels, codec) #9782

Closed

11 tasks

gemini-code-assist bot reviewed Aug 29, 2025

View reviewed changes

nateshmbhat added 6 commits August 29, 2025 14:19

test(video_player): add tests for audio track selection and management

8f711b5

fix(video): address PR review comments

e0f6d65

Merge branch 'main' into develop

fc30013

fix(video_player): add delay after audio track selection to handle Ex…

8a68e76

…oPlayer async updates

test(video_player): update audio tracks test to use ImmutableList and…

894f516

… fix format builders

Merge branch 'develop' of github.com:nateshmbhat/flutter_packages int…

644e08e

…o develop

nateshmbhat added 4 commits August 30, 2025 22:03

refactor(tests): move audio track tests from AudioTracksTests.m to Vi…

fdde6f8

…deoPlayerTests.m

fix(ios,android): fixed test failure cases (linting and warnings)

4291609

style(audio_tracks): improve code style and add type safety in audio …

8dfd8e3

…tracks demo

chore(android): bump compileSdk from 34 to 35 for video player plugin

a892a5e

fix(ios): fixed tests

stuartmorgan-g added triage-ios Should be looked at in iOS triage triage-android Should be looked at in Android triage labels Sep 2, 2025

nateshmbhat added 2 commits September 4, 2025 21:48

Merge branch 'main' into develop

1537778

Merge branch 'main' into develop

c222584

This comment was marked as duplicate.

Sign in to view

LongCatIsLooong reviewed Sep 12, 2025

View reviewed changes

		@@ -1,5 +1,6 @@
		## NEXT

		* Implements `getAudioTracks()` and `selectAudioTrack()` methods for iOS/macOS using AVFoundation.

[video_player] Adds audio track metadata fetching and audio track selection feature #9925

Are you sure you want to change the base?

[video_player] Adds audio track metadata fetching and audio track selection feature #9925

Uh oh!

Conversation

nateshmbhat commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes Made

Core Features

Platform Implementations

Technical Infrastructure

Demo Features

Related Issues

Testing

Breaking Changes

Pre-Review Checklist

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nateshmbhat commented Aug 30, 2025

Uh oh!

nateshmbhat commented Sep 1, 2025

Uh oh!

nateshmbhat commented Sep 9, 2025

Uh oh!

stuartmorgan-g commented Sep 9, 2025

Uh oh!

This comment was marked as duplicate.

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LongCatIsLooong Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nateshmbhat commented Aug 29, 2025 •

edited

Loading

LongCatIsLooong Sep 12, 2025 •

edited

Loading