feat/video-recording #2983
Merged
feat/video-recording #2983
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: Nick Sweeting <[email protected]>
…unt/browser-use into feat/video-recording
- Introduced `record_video_framerate` and `record_video_size` parameters to the BrowserSession class for enhanced video recording capabilities. - Updated example in video_recording.py to reflect changes, switching from BrowserSession to Browser for video recording setup. This update allows users to customize video recording settings more effectively.
| self.logger.warning('Cannot start video recording: viewport size could not be determined.') | ||
| return | ||
|
|
||
| video_format = getattr(profile, 'record_video_format', 'mp4').strip('.') |
There was a problem hiding this comment.
Bug: Undefined Video Format Setting
The record_video_format field is accessed from the browser profile but isn't defined in the BrowserProfile class. While getattr prevents a crash by defaulting to 'mp4', this means users can't configure the video format. This is inconsistent with other video recording settings like record_video_framerate and record_video_size which are configurable.
Agent Task Evaluation Results: 4/4 (100%)View detailed results
Check the evaluate-tasks job for detailed task execution logs. |
| record_video_size: ViewportSize | None = Field( | ||
| default=None, description='Video frame size. If not set, it will use the viewport size.' | ||
| ) | ||
| record_video_framerate: int = Field(default=30, description='The framerate to use for the video recording.') |
There was a problem hiding this comment.
Framerate lacks a lower bound; 0 or negative values would be invalid for recording and can cause encoder/runtime errors. Add a minimum of 1 via Field(ge=1).
(Based on your team's feedback about AI suggestions matching the project's tab indentation style.)
Prompt for AI agents
Address the following comment on browser_use/browser/profile.py at line 628:
<comment>Framerate lacks a lower bound; 0 or negative values would be invalid for recording and can cause encoder/runtime errors. Add a minimum of 1 via Field(ge=1).
(Based on your team's feedback about AI suggestions matching the project's tab indentation style.)</comment>
<file context>
@@ -615,6 +615,18 @@ class BrowserProfile(BrowserConnectArgs, BrowserLaunchPersistentContextArgs, Bro
+ record_video_size: ViewportSize | None = Field(
+ default=None, description='Video frame size. If not set, it will use the viewport size.'
+ )
+ record_video_framerate: int = Field(default=30, description='The framerate to use for the video recording.')
+
# TODO: finish implementing extension support in extensions.py
</file context>
Suggested change
| record_video_framerate: int = Field(default=30, description='The framerate to use for the video recording.') | |
| record_video_framerate: int = Field(default=30, ge=1, description='The framerate to use for the video recording.') |
| ## Recording & Debugging | ||
| - `record_video_dir`: Directory to save video recordings as `.webm` files | ||
| - `record_video_dir`: Directory to save video recordings as `.mp4` files | ||
| - `record_video_size` (default: `ViewportSize`): The frame size (width, height) of the video recording. |
There was a problem hiding this comment.
Documented default is incorrect; the field defaults to None and falls back to the current viewport size when unspecified.
Prompt for AI agents
Address the following comment on docs/customize/browser/all-parameters.mdx at line 77:
<comment>Documented default is incorrect; the field defaults to None and falls back to the current viewport size when unspecified.</comment>
<file context>
@@ -73,7 +73,9 @@ mode: "wide"
## Recording & Debugging
-- `record_video_dir`: Directory to save video recordings as `.webm` files
+- `record_video_dir`: Directory to save video recordings as `.mp4` files
+- `record_video_size` (default: `ViewportSize`): The frame size (width, height) of the video recording.
+- `record_video_framerate` (default: `30`): The framerate to use for the video recording.
- `record_har_path`: Path to save network trace files as `.har` format
</file context>
Suggested change
| - `record_video_size` (default: `ViewportSize`): The frame size (width, height) of the video recording. | |
| - `record_video_size` (default: `None`): The frame size (width, height) of the video recording. Uses current viewport size when not set. |
| self._recorder = VideoRecorderService(output_path=output_path, size=size, framerate=profile.record_video_framerate) | ||
| self._recorder.start() | ||
|
|
||
| if not self._recorder._is_active: |
There was a problem hiding this comment.
Accessing VideoRecorderService._is_active directly breaks encapsulation; prefer a public accessor or use service methods to infer state.
Prompt for AI agents
Address the following comment on browser_use/browser/watchdogs/recording_watchdog.py at line 52:
<comment>Accessing VideoRecorderService._is_active directly breaks encapsulation; prefer a public accessor or use service methods to infer state.</comment>
<file context>
@@ -0,0 +1,126 @@
+ self._recorder = VideoRecorderService(output_path=output_path, size=size, framerate=profile.record_video_framerate)
+ self._recorder.start()
+
+ if not self._recorder._is_active:
+ self._recorder = None
+ return
</file context>
| await cdp_session.cdp_client.send.Page.startScreencast( | ||
| params={ | ||
| 'format': 'png', | ||
| 'quality': 90, |
There was a problem hiding this comment.
Passing 'quality' with format='png' is unnecessary; remove it to match CDP expectations and avoid confusion.
Prompt for AI agents
Address the following comment on browser_use/browser/watchdogs/recording_watchdog.py at line 63:
<comment>Passing 'quality' with format='png' is unnecessary; remove it to match CDP expectations and avoid confusion.</comment>
<file context>
@@ -0,0 +1,126 @@
+ await cdp_session.cdp_client.send.Page.startScreencast(
+ params={
+ 'format': 'png',
+ 'quality': 90,
+ 'maxWidth': size['width'],
+ 'maxHeight': size['height'],
</file context>
| self.logger.warning('Cannot start video recording: viewport size could not be determined.') | ||
| return | ||
|
|
||
| video_format = getattr(profile, 'record_video_format', 'mp4').strip('.') |
There was a problem hiding this comment.
Configurable video_format only changes the file extension while encoder is fixed to H.264; unsupported combos (e.g., .webm) may fail at runtime. Validate/limit formats or map codecs per format.
Prompt for AI agents
Address the following comment on browser_use/browser/watchdogs/recording_watchdog.py at line 45:
<comment>Configurable video_format only changes the file extension while encoder is fixed to H.264; unsupported combos (e.g., .webm) may fail at runtime. Validate/limit formats or map codecs per format.</comment>
<file context>
@@ -0,0 +1,126 @@
+ self.logger.warning('Cannot start video recording: viewport size could not be determined.')
+ return
+
+ video_format = getattr(profile, 'record_video_format', 'mp4').strip('.')
+ output_path = Path(profile.record_video_dir) / f'{uuid7str()}.{video_format}'
+
</file context>
| """ | ||
| Synchronous handler for incoming screencast frames. | ||
| """ | ||
| if not self._recorder: |
There was a problem hiding this comment.
Frames are not acked when recorder is None and screencast is never stopped, causing stalled screencast/backpressure.
Prompt for AI agents
Address the following comment on browser_use/browser/watchdogs/recording_watchdog.py at line 100:
<comment>Frames are not acked when recorder is None and screencast is never stopped, causing stalled screencast/backpressure.</comment>
<file context>
@@ -0,0 +1,126 @@
+ """
+ Synchronous handler for incoming screencast frames.
+ """
+ if not self._recorder:
+ return
+ self._recorder.add_frame(event['data'])
</file context>
| self._dom_watchdog = None | ||
| self._screenshot_watchdog = None | ||
| self._permissions_watchdog = None | ||
| self._recording_watchdog = None |
There was a problem hiding this comment.
_watchdogs_attached flag not reset on reset()/stop()/kill() prevents reattaching handlers to the new EventBus, breaking subsequent starts.
Prompt for AI agents
Address the following comment on browser_use/browser/session.py at line 407:
<comment>_watchdogs_attached flag not reset on reset()/stop()/kill() prevents reattaching handlers to the new EventBus, breaking subsequent starts.</comment>
<file context>
@@ -401,6 +404,7 @@ async def reset(self) -> None:
self._dom_watchdog = None
self._screenshot_watchdog = None
self._permissions_watchdog = None
+ self._recording_watchdog = None
def model_post_init(self, __context) -> None:
</file context>
| """ | ||
| if not self._recorder: | ||
| return | ||
| self._recorder.add_frame(event['data']) |
There was a problem hiding this comment.
Possible race: self._recorder can become None between the check and add_frame call; use a local reference to avoid TOCTOU on the attribute.
Prompt for AI agents
Address the following comment on browser_use/browser/watchdogs/recording_watchdog.py at line 102:
<comment>Possible race: self._recorder can become None between the check and add_frame call; use a local reference to avoid TOCTOU on the attribute.</comment>
<file context>
@@ -0,0 +1,126 @@
+ """
+ if not self._recorder:
+ return
+ self._recorder.add_frame(event['data'])
+ asyncio.create_task(self._ack_screencast_frame(event, session_id))
+
</file context>
| self._recorder = None | ||
| return | ||
|
|
||
| self.browser_session.cdp_client.register.Page.screencastFrame(self.on_screencastFrame) |
There was a problem hiding this comment.
Registered screencast listener is never unregistered and Page.stopScreencast is not called during shutdown, risking resource leaks and unnecessary frame handling.
Prompt for AI agents
Address the following comment on browser_use/browser/watchdogs/recording_watchdog.py at line 56:
<comment>Registered screencast listener is never unregistered and Page.stopScreencast is not called during shutdown, risking resource leaks and unnecessary frame handling.</comment>
<file context>
@@ -0,0 +1,126 @@
+ self._recorder = None
+ return
+
+ self.browser_session.cdp_client.register.Page.screencastFrame(self.on_screencastFrame)
+
+ try:
</file context>
| """ | ||
| if not self._recorder: | ||
| return | ||
| self._recorder.add_frame(event['data']) |
There was a problem hiding this comment.
Calling add_frame synchronously in the event callback blocks on a subprocess, stalling the event loop and risking frame drops/timeouts. Offload frame processing to a thread executor.
Prompt for AI agents
Address the following comment on browser_use/browser/watchdogs/recording_watchdog.py at line 102:
<comment>Calling add_frame synchronously in the event callback blocks on a subprocess, stalling the event loop and risking frame drops/timeouts. Offload frame processing to a thread executor.</comment>
<file context>
@@ -0,0 +1,126 @@
+ """
+ if not self._recorder:
+ return
+ self._recorder.add_frame(event['data'])
+ asyncio.create_task(self._ack_screencast_frame(event, session_id))
+
</file context>
|
@EnzoFanAccount would you say any of the bot comments is relevant? |
|
@EnzoFanAccount Can you also make it work for multiple tabs? E.g. you could listen to on_TabCreatedEvent and on_SwitchTabEvent and take whatever tab is currently active in add_frame |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Auto-generated PR for branch: feat/video-recording
Summary by cubic
Adds built-in MP4 video recording for browser sessions using CDP screencasting. Enable by setting record_video_dir; recordings auto-save when the session ends.
New Features
Dependencies