-
Notifications
You must be signed in to change notification settings - Fork 755
Clip source offsetting #1003
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Clip source offsetting #1003
Conversation
WalkthroughAdds per-clip source offsets (camera/mic/system) across UI, project config, rendering, and audio. Threads ClipOffsets through editor, playback, and rendering pipelines, updates audio renderer to use per-track offsets, and adds UI controls to edit offsets in milliseconds stored as seconds. Minor preset and selection tweaks. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant UI as ConfigSidebar (UI)
participant Store as Editor Store
participant Render as Playback/Renderer
participant Dec as RecordingSegmentDecoders
Note over UI,Store: User edits offsets (ms) → store saves seconds
UI->>Store: setOffset(type, ms) -> store.seconds
Store-->>Render: Project updated (clips)
Render->>Dec: get_frames(segment_time, needs_camera, clip_offsets)
Note right of Dec: uses offsets.camera to fetch camera frame
Dec-->>Render: Decoded frames
sequenceDiagram
autonumber
participant EAudio as Editor Audio
participant AR as Audio Renderer
participant Buf as Output Buffer
Note over EAudio,AR: Build tracks with per-clip offsets
EAudio->>AR: render_audio([AudioRendererTrack{data,gain,stereo_mode,offset}], ...)
AR->>AR: Mix per-track samples using track.offset
AR-->>Buf: Write interleaved stereo samples
sequenceDiagram
autonumber
participant Track as ClipTrack
participant Store as Editor Store
Note over Track: Mouseup on clip segment
Track->>Store: select clip at index (always open config)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Pre-merge checks (2 passed, 1 warning)❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
Poem
Tip 👮 Agentic pre-merge checks are now available in preview!Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.
Please see the documentation for more information. Example: reviews:
pre_merge_checks:
custom_checks:
- name: "Undocumented Breaking Changes"
mode: "warning"
instructions: |
Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal). Please share your feedback with us on this Discord post. ✨ Finishing touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 4
🧹 Nitpick comments (9)
apps/desktop/src/utils/createPresets.ts (1)
31-31
: Preset configs should exclude clip data at the type level.Instead of mutating config.clips to undefined, omit it in the type to prevent accidental inclusion and avoid runtime mutation.
Apply:
-export type CreatePreset = { - name: string; - config: Omit<ProjectConfiguration, "timeline">; - default: boolean; -}; +export type CreatePreset = { + name: string; + config: Omit<ProjectConfiguration, "timeline" | "clips">; + default: boolean; +}; ... - // @ts-expect-error we reeeally don't want the timeline in the preset - config.timeline = undefined; - config.clips = undefined; + // @ts-expect-error we reeeally don't want the timeline in the preset + config.timeline = undefined;apps/desktop/src/routes/editor/Timeline/ClipTrack.tsx (1)
358-374
: Confirm UX: always opening clip configuration on mouseup.This removes the previous “single-segment no-open” behavior. If intentional, LGTM; if not, gate on segments().length > 1 as before.
- setEditorState("timeline", "selection", { - type: "clip", - index: i(), - }); +// If desired to keep prior UX: +// if (segments().length > 1) { +setEditorState("timeline", "selection", { type: "clip", index: i() }); +// }crates/rendering/src/lib.rs (1)
139-149
: Guard camera time against negatives.If offsets.camera can be negative, request a frame at < 0 may return None or waste work. Safer to clamp to 0.
- .map(|d| d.get_frame(segment_time + offsets.camera))) + .map(|d| { + let camera_time = (segment_time + offsets.camera).max(0.0); + d.get_frame(camera_time) + }))crates/editor/src/playback.rs (1)
253-258
: Remove now-unused prev_audio_config.After always calling set_playhead, prev_audio_config is written but never read.
- let mut prev_audio_config = project.borrow().audio.clone(); ... - audio_renderer.set_playhead( + audio_renderer.set_playhead( playhead + elapsed as f64 / output_info.sample_rate as f64, &project, ); - prev_audio_config = project.audio.clone();apps/desktop/src/routes/editor/ConfigSidebar.tsx (1)
2426-2479
: Keep NumberField state in sync with props and avoid wraparound on empty/NaN input.
- Local
value
doesn’t react ifprops.value
changes (switching clips). Add a sync effect.- Use an explicit empty/NaN check on blur;
!rawValue()
also triggers for 0.Apply this diff:
const rawValue = () => Math.round((props.value ?? 0) * 1000); const [value, setValue] = createSignal(rawValue().toString()); +createEffect(() => { + setValue(rawValue().toString()); +}); @@ - onBlur={() => { - if (!rawValue() || value() === "" || Number.isNaN(rawValue())) { - setValue("0"); - props.onChange(0); - } - }} + onBlur={() => { + const v = value(); + const n = Number.parseInt(v, 10); + if (Number.isNaN(n) || v.trim() === "") { + setValue("0"); + props.onChange(0); + } + }}crates/audio/src/renderer.rs (1)
24-30
: Consider boundingsamples
by the maximum positive per-track offset.Tracks delayed by a positive
track.offset
reduce the readable window from the sharedoffset
. Without accounting for this, the mixer may emit trailing silence where some tracks haven’t reached valid data yet.You can precompute
max_right_shift = tracks.iter().map(|t| t.offset.max(0)).max().unwrap_or(0) as usize
and capsamples
by(frames_per_track - offset - max_right_shift)
to better reflect the tightest window.crates/project/src/configuration.rs (3)
600-606
: Make offsets tolerant to omissionIf a clip entry is created with only an index, deserialization will currently fail because offsets is required. Default it to zeroed offsets.
#[derive(Type, Serialize, Deserialize, Clone, Debug, Default)] #[serde(rename_all = "camelCase")] pub struct ClipConfiguration { pub index: u32, - pub offsets: ClipOffsets, + #[serde(default)] + pub offsets: ClipOffsets, }
603-604
: Consider a stable clip identifier instead of positional indexIndices can shift on reorder/delete; a stable id (e.g., UUID or recording segment id) avoids misapplying offsets after edits.
590-598
: Clarify units/sign and harden (alias + struct-level serde default) for ClipOffsets
- Add short field docs (seconds; positive = delay, negative = advance), move per-field serde defaults to a struct-level #[serde(default)], and accept camelCase input for system_audio via #[serde(alias = "systemAudio")]. Repo search shows snake_case "system_audio" is used widely, so alias is backward-compatible.
-#[derive(Type, Serialize, Deserialize, Clone, Copy, Debug, Default)] -pub struct ClipOffsets { - #[serde(default)] - pub camera: f32, - #[serde(default)] - pub mic: f32, - #[serde(default)] - pub system_audio: f32, -} +#[derive(Type, Serialize, Deserialize, Clone, Copy, Debug, Default)] +#[serde(default)] +pub struct ClipOffsets { + /// Offset in seconds. Positive delays the source relative to screen; negative advances it. + pub camera: f32, + /// Offset in seconds. Positive delays the source relative to screen; negative advances it. + pub mic: f32, + /// Offset in seconds. Positive delays the source relative to screen; negative advances it. + #[serde(alias = "systemAudio")] + pub system_audio: f32, +}
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (11)
apps/desktop/src/routes/editor/ConfigSidebar.tsx
(5 hunks)apps/desktop/src/routes/editor/Timeline/ClipTrack.tsx
(1 hunks)apps/desktop/src/utils/createPresets.ts
(1 hunks)apps/desktop/src/utils/tauri.ts
(2 hunks)crates/audio/src/renderer.rs
(3 hunks)crates/editor/src/audio.rs
(2 hunks)crates/editor/src/editor_instance.rs
(1 hunks)crates/editor/src/playback.rs
(2 hunks)crates/project/src/configuration.rs
(2 hunks)crates/rendering/src/lib.rs
(4 hunks)crates/scap-direct3d/src/lib.rs
(0 hunks)
💤 Files with no reviewable changes (1)
- crates/scap-direct3d/src/lib.rs
🧰 Additional context used
📓 Path-based instructions (7)
**/tauri.ts
📄 CodeRabbit inference engine (CLAUDE.md)
Never edit auto-generated IPC bindings file: tauri.ts
Do not edit auto-generated file: tauri.ts
Files:
apps/desktop/src/utils/tauri.ts
apps/desktop/**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
apps/desktop/**/*.{ts,tsx}
: In the desktop app, rely on unplugin-icons auto-imports; do not manually import icon modules
Use generated tauri_specta commands/events (commands, events) in the desktop frontend; listen to generated events directly
Use @tanstack/solid-query for server state in the desktop app
Files:
apps/desktop/src/utils/tauri.ts
apps/desktop/src/utils/createPresets.ts
apps/desktop/src/routes/editor/ConfigSidebar.tsx
apps/desktop/src/routes/editor/Timeline/ClipTrack.tsx
{apps/desktop,packages/ui-solid}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
Component naming (Solid): components in PascalCase; hooks/utilities in camelCase starting with 'use' where applicable
Files:
apps/desktop/src/utils/tauri.ts
apps/desktop/src/utils/createPresets.ts
apps/desktop/src/routes/editor/ConfigSidebar.tsx
apps/desktop/src/routes/editor/Timeline/ClipTrack.tsx
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
Use strict TypeScript and avoid any; prefer shared types from packages
**/*.{ts,tsx}
: Use Biome to format/lint TypeScript with a 2-space indent
TypeScript file names should be kebab-case (e.g., user-menu.tsx)
Files:
apps/desktop/src/utils/tauri.ts
apps/desktop/src/utils/createPresets.ts
apps/desktop/src/routes/editor/ConfigSidebar.tsx
apps/desktop/src/routes/editor/Timeline/ClipTrack.tsx
{crates,apps/cli,apps/desktop/src-tauri}/**/*.rs
📄 CodeRabbit inference engine (AGENTS.md)
{crates,apps/cli,apps/desktop/src-tauri}/**/*.rs
: Rust code must be formatted with rustfmt and adhere to workspace clippy lints
Rust module files should use snake_case names
Files:
crates/editor/src/editor_instance.rs
crates/rendering/src/lib.rs
crates/editor/src/audio.rs
crates/project/src/configuration.rs
crates/audio/src/renderer.rs
crates/editor/src/playback.rs
{crates,apps/cli,apps/desktop/src-tauri}/**/{src,tests}/**/*.rs
📄 CodeRabbit inference engine (AGENTS.md)
Rust tests should live in src (inline/unit) or tests (integration) directories
Files:
crates/editor/src/editor_instance.rs
crates/rendering/src/lib.rs
crates/editor/src/audio.rs
crates/project/src/configuration.rs
crates/audio/src/renderer.rs
crates/editor/src/playback.rs
**/*.{tsx,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
React/Solid components should be named using PascalCase
Files:
apps/desktop/src/routes/editor/ConfigSidebar.tsx
apps/desktop/src/routes/editor/Timeline/ClipTrack.tsx
🧠 Learnings (1)
📚 Learning: 2025-09-08T16:48:20.727Z
Learnt from: CR
PR: CapSoftware/Cap#0
File: AGENTS.md:0-0
Timestamp: 2025-09-08T16:48:20.727Z
Learning: Applies to **/tauri.ts : Do not edit auto-generated file: tauri.ts
Applied to files:
apps/desktop/src/utils/tauri.ts
🧬 Code graph analysis (6)
crates/rendering/src/lib.rs (2)
apps/desktop/src/utils/tauri.ts (1)
ClipOffsets
(353-353)crates/rendering/src/decoder/mod.rs (1)
get_frame
(32-38)
crates/editor/src/audio.rs (2)
crates/audio/src/lib.rs (1)
cast_f32_slice_to_bytes
(31-33)crates/editor/src/playback.rs (1)
start
(43-127)
crates/project/src/configuration.rs (2)
apps/desktop/src/utils/tauri.ts (5)
CaptionsData
(349-349)CaptionSegment
(347-347)CaptionSettings
(348-348)ClipOffsets
(353-353)ClipConfiguration
(352-352)apps/desktop/src/store/captions.ts (1)
CaptionSettings
(12-25)
apps/desktop/src/routes/editor/ConfigSidebar.tsx (4)
apps/desktop/src/utils/tauri.ts (1)
ClipOffsets
(353-353)crates/editor/src/editor_instance.rs (1)
meta
(98-100)apps/desktop/src/routes/editor/context.ts (1)
meta
(308-310)apps/desktop/src/routes/editor/ui.tsx (1)
Field
(25-47)
crates/audio/src/renderer.rs (2)
apps/desktop/src/utils/tauri.ts (1)
StereoMode
(441-441)crates/editor/src/audio.rs (3)
tracks
(186-208)data
(54-56)gain
(58-60)
crates/editor/src/playback.rs (1)
crates/rendering/src/lib.rs (1)
get_frames
(135-159)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Clippy
- GitHub Check: Build Desktop (x86_64-pc-windows-msvc, windows-latest)
- GitHub Check: Build Desktop (aarch64-apple-darwin, macos-latest)
- GitHub Check: Analyze (rust)
🔇 Additional comments (7)
apps/desktop/src/utils/tauri.ts (1)
416-416
: Contract change acknowledged: ProjectConfiguration now includes clips.Looks good as long as it’s generated. Please ensure UI and backend handle absence (undefined) and default to zero offsets.
crates/editor/src/editor_instance.rs (1)
219-220
: Passing offsets through to decoding: LGTM.Forwarding per-clip offsets to get_frames is correct.
crates/rendering/src/lib.rs (1)
223-236
: Per-clip offset threading: LGTM.Lookup + defaulting and passing into get_frames is correct.
crates/editor/src/playback.rs (1)
86-92
: Clip lookup: verify index semantics.Same as editor_instance.rs — ensure segment_i corresponds to the clip index intended for offsets, not just the current timeline segment index.
crates/editor/src/audio.rs (1)
189-206
: ```shell
#!/bin/bash
set -euo pipefail
echo "PWD: $(pwd)"
echo "---- rg for AudioSegmentTrack ----"
rg -n -S --hidden --no-ignore -C3 'AudioSegmentTrack' || true
echo "---- show crates/editor/src/audio.rs (first 400 lines) ----"
if [ -f crates/editor/src/audio.rs ]; then
sed -n '1,400p' crates/editor/src/audio.rs
else
echo "file not found: crates/editor/src/audio.rs"
fi
echo "---- rg for ClipOffsets struct and system_audio ----"
rg -n -S --hidden --no-ignore -P -C3 'struct\s+ClipOffsets' || true
rg -n -S --hidden --no-ignore 'ClipOffsets' -C3 || true
rg -n -S --hidden --no-ignore 'system_audio' -C3 || true
rg -n -S --hidden --no-ignore 'mic' -C3 || true
echo "---- rg for AudioRendererTrack ----"
rg -n -S --hidden --no-ignore 'AudioRendererTrack' -C3 || true</blockquote></details> <details> <summary>crates/project/src/configuration.rs (2)</summary><blockquote> `583-589`: **LGTM: CaptionsData now implements Default** This makes missing captions deserialize cleanly (empty segments + default settings). --- `607-622`: **LGTM: clips added with serde(default) — backward compatible** Empty Vec on missing field preserves old projects; deriving Default on ProjectConfiguration is consistent with load()/write() behavior. </blockquote></details> </blockquote></details> </details> <!-- This is an auto-generated comment by CodeRabbit for review status -->
const clipConfig = () => project.clips?.[props.segmentIndex]; | ||
const offsets = () => clipConfig()?.offsets || {}; | ||
|
||
function setOffset(type: keyof ClipOffsets, offset: number) { | ||
if (Number.isNaN(offset)) return; | ||
|
||
setProject( | ||
produce((proj) => { | ||
const clips = (proj.clips ??= []); | ||
let clip = clips.find( | ||
(clip) => clip.index === (props.segment.recordingSegment ?? 0), | ||
); | ||
if (!clip) { | ||
clip = { index: 0, offsets: {} }; | ||
clips.push(clip); | ||
} | ||
|
||
clip.offsets[type] = offset / 1000; | ||
}), | ||
); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🛠️ Refactor suggestion
Fix clip lookup and wrong index when creating a new clip config (offsets currently attach to clip 0).
- offsets() reads
project.clips?.[props.segmentIndex]
(timeline index) but setOffset writes to the clip whoseindex
equalsprops.segment.recordingSegment ?? 0
. These indices are different concepts; using array indexing is incorrect. - Creating a missing clip with
index: 0
for every segment is a bug; it misapplies offsets to clip 0.
Update both the reader and writer to consistently resolve by recordingSegment
:
Apply this diff:
- const clipConfig = () => project.clips?.[props.segmentIndex];
- const offsets = () => clipConfig()?.offsets || {};
+ const clipIndex = props.segment.recordingSegment ?? 0;
+ const clipConfig = () => project.clips?.find(c => c.index === clipIndex);
+ const offsets = () => clipConfig()?.offsets ?? {};
@@
- setProject(
- produce((proj) => {
- const clips = (proj.clips ??= []);
- let clip = clips.find(
- (clip) => clip.index === (props.segment.recordingSegment ?? 0),
- );
- if (!clip) {
- clip = { index: 0, offsets: {} };
- clips.push(clip);
- }
- clip.offsets[type] = offset / 1000;
- }),
- );
+ setProject(produce((proj) => {
+ const clips = (proj.clips ??= []);
+ let clip = clips.find((c) => c.index === clipIndex);
+ if (!clip) {
+ clip = { index: clipIndex, offsets: {} };
+ clips.push(clip);
+ }
+ clip.offsets[type] = offset / 1000;
+ }));
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
const clipConfig = () => project.clips?.[props.segmentIndex]; | |
const offsets = () => clipConfig()?.offsets || {}; | |
function setOffset(type: keyof ClipOffsets, offset: number) { | |
if (Number.isNaN(offset)) return; | |
setProject( | |
produce((proj) => { | |
const clips = (proj.clips ??= []); | |
let clip = clips.find( | |
(clip) => clip.index === (props.segment.recordingSegment ?? 0), | |
); | |
if (!clip) { | |
clip = { index: 0, offsets: {} }; | |
clips.push(clip); | |
} | |
clip.offsets[type] = offset / 1000; | |
}), | |
); | |
} | |
const clipIndex = props.segment.recordingSegment ?? 0; | |
const clipConfig = () => project.clips?.find(c => c.index === clipIndex); | |
const offsets = () => clipConfig()?.offsets ?? {}; | |
function setOffset(type: keyof ClipOffsets, offset: number) { | |
if (Number.isNaN(offset)) return; | |
setProject(produce((proj) => { | |
const clips = (proj.clips ??= []); | |
let clip = clips.find((c) => c.index === clipIndex); | |
if (!clip) { | |
clip = { index: clipIndex, offsets: {} }; | |
clips.push(clip); | |
} | |
clip.offsets[type] = offset / 1000; | |
})); | |
} |
🤖 Prompt for AI Agents
In apps/desktop/src/routes/editor/ConfigSidebar.tsx around lines 2326 to 2346,
the reader uses project.clips?.[props.segmentIndex] (timeline index) while the
writer finds/creates clips by clip.index === (props.segment.recordingSegment ??
0), causing offsets to attach to clip 0; update both to consistently resolve
clips by recordingSegment: change clipConfig() and offsets() to use
props.segment.recordingSegment (with safe fallback) when indexing/reading, and
in setOffset when creating a missing clip set its index to the recordingSegment
value (not 0) so offsets are stored on the correct clip; keep Number.isNaN guard
and the same offset division by 1000.
apps/desktop/src/utils/tauri.ts
Outdated
export type ClipConfiguration = { index: number; offsets: ClipOffsets } | ||
export type ClipOffsets = { camera?: number; mic?: number; system_audio?: number } | ||
export type CommercialLicense = { licenseKey: string; expiryDate: number | null; refresh: number; activatedOn: number } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Verification agent
🧩 Analysis chain
Do not manually edit this generated file; ensure these types come from tauri-specta codegen.
If these additions weren’t produced by codegen, please revert and re-generate. Also verify Rust-side shapes: TS shows optional fields (ClipOffsets.camera/mic/system_audio and ProjectConfiguration.clips?), which must match Rust (Option vs plain). Mismatch will desync IPC contracts.
To confirm, please check the Rust definitions:
🏁 Script executed:
#!/bin/bash
# Verify Rust shapes for ProjectConfiguration.clips and ClipOffsets
fd -t f configuration.rs crates | xargs -I{} rg -n -C3 'struct\s+ProjectConfiguration|clips\s*:' {}
fd -t f configuration.rs crates | xargs -I{} rg -n -C3 'struct\s+ClipOffsets|impl\s+Default\s+for\s+ClipOffsets' {}
# Also locate get_segment_time to confirm index semantics
rg -n 'fn\s+get_segment_time' -C3
Length of output: 1593
Do not edit generated tauri.ts — regenerate from tauri-specta; Rust↔TS shapes mismatch
- ClipOffsets mismatch: Rust (crates/project/src/configuration.rs, ~lines 590–596) defines camera/mic/system_audio as non-optional f32 with #[serde(default)]; TS currently has these as optional (camera?: number, etc.) — regenerate so TS fields are non-optional numbers.
- ProjectConfiguration.clips mismatch: Rust (crates/project/src/configuration.rs, ~lines 606–622) has pub clips: Vec (#[serde(default)]); TS should use ClipConfiguration[] (not optional).
- Index semantics: get_segment_time signature (crates/project/src/configuration.rs, lines 511 & 640) is fn get_segment_time(&self, frame_time: f64) -> Option<(f64, u32)> — segment index is a numeric u32; ensure ClipConfiguration.index is numeric and aligned.
- Action: if apps/desktop/src/utils/tauri.ts was edited manually, revert and re-run tauri-specta codegen to resync IPC contracts.
🤖 Prompt for AI Agents
In apps/desktop/src/utils/tauri.ts around lines 352 to 354, the generated
TypeScript shapes no longer match the Rust definitions: make ClipOffsets fields
non-optional numbers (camera: number; mic: number; system_audio: number), ensure
ClipConfiguration.index is a numeric number (not a string/optional) and
ProjectConfiguration.clips is ClipConfiguration[] (not optional), then revert
any manual edits to this file and re-run the tauri-specta codegen to regenerate
the file so IPC/Rust↔TS contracts are in sync.
crates/audio/src/renderer.rs
Outdated
// dbg!(i); | ||
let i = i.wrapping_add_signed(dbg!(track.offset)); | ||
// dbg!(i); | ||
|
||
let data = track.data; | ||
let gain = gain_for_db(track.gain); | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🛠️ Refactor suggestion
Remove dbg! and fix negative-offset indexing (current wrapping underflows and silences tracks).
wrapping_add_signed
on usize
makes negative offsets wrap to huge indices, so samples never play until wraparound (practically never). Also dbg!
spams logs per sample.
Apply this diff:
- for track in tracks {
- // dbg!(i);
- let i = i.wrapping_add_signed(dbg!(track.offset));
- // dbg!(i);
+ for track in tracks {
+ // translate frame index by per-track offset in signed space
+ let ti = i as isize + track.offset;
+ if ti < 0 {
+ continue; // not yet reached this track’s first sample
+ }
+ let ti = ti as usize;
let data = track.data;
let gain = gain_for_db(track.gain);
@@
- if data.channels() == 1 {
- if let Some(sample) = data.samples().get(offset + i) {
+ if data.channels() == 1 {
+ if let Some(sample) = data.samples().get(offset + ti) {
...
- } else if data.channels() == 2 {
- let base_idx = offset * 2 + i * 2;
+ } else if data.channels() == 2 {
+ let base_idx = (offset + ti) * 2;
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
// dbg!(i); | |
let i = i.wrapping_add_signed(dbg!(track.offset)); | |
// dbg!(i); | |
let data = track.data; | |
let gain = gain_for_db(track.gain); | |
for track in tracks { | |
// translate frame index by per-track offset in signed space | |
let ti = i as isize + track.offset; | |
if ti < 0 { | |
continue; // not yet reached this track’s first sample | |
} | |
let ti = ti as usize; | |
let data = track.data; | |
let gain = gain_for_db(track.gain); | |
if data.channels() == 1 { | |
if let Some(sample) = data.samples().get(offset + ti) { | |
... | |
} | |
} else if data.channels() == 2 { | |
let base_idx = (offset + ti) * 2; | |
... | |
} | |
} |
🤖 Prompt for AI Agents
In crates/audio/src/renderer.rs around lines 37 to 43, remove the dbg! calls and
stop using usize::wrapping_add_signed which wraps negative offsets into huge
indices; instead compute a signed sample position by adding the track offset to
the current sample index in a signed integer, check bounds (if pos < 0 or pos >=
data.len() treat as silence), and only index data when pos is in-range (pos as
usize). Keep gain_for_db(track.gain) as-is.
crates/editor/src/editor_instance.rs
Outdated
let clip_config = project.clips.iter().find(|v| v.index == segment_i); | ||
let clip_offsets = clip_config.map(|v| v.offsets).unwrap_or_default(); | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Verification agent
🧩 Analysis chain
Clip lookup: verify index space.
You’re matching on segment_i from get_segment_time; confirm this aligns with the “clip index” (recording segment index), not timeline segment index. If timeline segments can map to the same recording segment, consider using that mapping for a stable clip index.
🏁 Script executed:
#!/bin/bash
# Inspect get_segment_time return to confirm which index it yields
rg -n 'fn\s+get_segment_time\s*\(' -C4 crates
# Find where TimelineSegment.recordingSegment is used to map timeline->recording
rg -n 'recordingSegment' -C3
Length of output: 12663
🏁 Script executed:
#!/bin/bash
set -euo pipefail
file="crates/project/src/configuration.rs"
echo "=== Matches for pub fn get_segment_time in $file ==="
rg -n "pub fn get_segment_time" "$file" || true
echo
echo "=== Show context around each get_segment_time ==="
while IFS=: read -r f ln rest; do
start=$((ln - 30)); [ $start -lt 1 ] && start=1
end=$((ln + 140))
echo
echo "---- $f:$ln (showing lines $start-$end) ----"
sed -n "${start},${end}p" "$f"
done < <(rg -n "pub fn get_segment_time" "$file" || true)
echo
echo "=== Search for TimelineSegment / recording_segment / recordingSegment across repo ==="
rg -nP 'struct\s+TimelineSegment' crates || true
rg -n 'recording_segment' crates || true
rg -n 'recordingSegment' -S || true
Length of output: 351
🏁 Script executed:
#!/bin/bash
set -euo pipefail
cfg="crates/project/src/configuration.rs"
ei="crates/editor/src/editor_instance.rs"
echo "=== file existence ==="
[ -f "$cfg" ] && echo "$cfg: exists" || { echo "$cfg: MISSING"; exit 0; }
[ -f "$ei" ] && echo "$ei: exists" || echo "$ei: MISSING"
echo
echo "=== rg matches for get_segment_time (full repo) ==="
rg -n "get_segment_time" -S || true
echo
echo "=== Matches in $cfg for pub fn get_segment_time ==="
rg -n "pub fn get_segment_time" "$cfg" || true
matches=$(rg -n "pub fn get_segment_time" "$cfg" || true)
if [ -n "$matches" ]; then
echo
echo "=== Show context around each get_segment_time ==="
echo "$matches" | while IFS=: read -r file ln rest; do
start=$((ln - 40)); [ $start -lt 1 ] && start=1
end=$((ln + 180))
echo
echo "---- $file:$ln (showing lines $start-$end) ----"
sed -n "${start},${end}p" "$file" || true
done
else
echo "No matches in $cfg"
fi
echo
echo "=== Show surrounding types/structs in $cfg (lines 1-220,220-520,520-880) ==="
sed -n '1,220p' "$cfg" || true
echo "----"
sed -n '220,520p' "$cfg" || true
echo "----"
sed -n '520,880p' "$cfg" || true
echo
echo "=== Search for TimelineConfiguration / TimelineSegment / SceneSegment in repo ==="
rg -n "TimelineConfiguration|TimelineSegment|SceneSegment|recordingSegment|recording_segment" -S || true
echo
echo "=== Show editor_instance.rs around lines 180-260 and full file if small ==="
if [ -f "$ei" ]; then
sed -n '1,120p' "$ei" || true
echo "----"
sed -n '121,300p' "$ei" || true
echo "---- full file size (lines):"
wc -l "$ei"
fi
echo
echo "=== Search for usages of get_segment_time and where its result is consumed ==="
rg -n "get_segment_time\(" -S || true
echo
echo "=== Search for 'clip_config' and 'clips.iter().find' occurrences ==="
rg -n "clips.iter().find|clip_config" -S || true
echo
echo "=== Search for fields named 'index' inside clip type definitions (rust/ts) ==="
rg -n "struct .*Clip|type .*Clip|clips: Vec|clips:" -S || true
Length of output: 1367
Use the timeline→recording mapping when looking up clips (timeline index != recording index).
get_segment_time returns a timeline-segment index (index into timeline.segments); project.clips are keyed by recording-segment indexes, so using segment_i directly can pick the wrong clip when multiple timeline segments map to the same recording segment. Derive the recording index from the timeline segment's recording_segment/recordingSegment field and use that to find the clip (fallback to segment_i only if the mapping is absent).
Location: crates/editor/src/editor_instance.rs (around lines 214–216).
🤖 Prompt for AI Agents
In crates/editor/src/editor_instance.rs around lines 214 to 216, the code looks
up a clip by using the timeline segment index (segment_i) but project.clips are
keyed by recording-segment indexes; change the lookup to first obtain the
timeline segment at segment_i and read its recording_segment/recordingSegment
field to derive the recording index, then use that recording index to find the
clip in project.clips; if the timeline segment has no recording mapping, fall
back to using segment_i as before; keep the existing unwrap_or_default behavior
for offsets.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
♻️ Duplicate comments (1)
crates/audio/src/renderer.rs (1)
36-41
: Fix negative-offset wrapping and deref bugs (silence due to wrap; compile risk on&f32
ops).
wrapping_add_signed
onusize
underflows for negative offsets, producing huge indices and silencing tracks.samples().get(...)
returns&f32
; multiplying references without deref is invalid; use patternSome(&x)
or*ref
.Apply:
- for track in tracks { - let i = i.wrapping_add_signed(track.offset); - - let data = track.data; - let gain = gain_for_db(track.gain); + for track in tracks { + // translate frame index by per-track offset in signed space + let ti = i as isize + track.offset; + if ti < 0 { + continue; // not yet reached this track’s first sample + } + let ti = ti as usize; + + let data = track.data; + let gain = gain_for_db(track.gain); @@ - if data.channels() == 1 { - if let Some(sample) = data.samples().get(offset + i) { - left += sample * 0.707 * gain; - right += sample * 0.707 * gain; - } + if data.channels() == 1 { + if let Some(&sample) = data.samples().get(offset + ti) { + left += sample * 0.707 * gain; + right += sample * 0.707 * gain; + } } else if data.channels() == 2 { - let base_idx = offset * 2 + i * 2; - let Some(l_sample) = data.samples().get(base_idx) else { + let base_idx = (offset + ti) * 2; + let Some(&l_sample) = data.samples().get(base_idx) else { continue; }; - let Some(r_sample) = data.samples().get(base_idx + 1) else { + let Some(&r_sample) = data.samples().get(base_idx + 1) else { continue; };Also applies to: 46-58, 60-73
🧹 Nitpick comments (3)
crates/audio/src/renderer.rs (3)
24-30
: Clamp output length using per-track positive offsets.Current cap uses only
frames - offset
, ignoring positive per-track offsets that shorten usable frames. Incorporatemax(0, track.offset)
(in frames) so we don’t overrun logical track ends and waste cycles mixing silence.- let samples = samples.min( - tracks - .iter() - .flat_map(|t| (t.data.samples().len() / t.data.channels() as usize).checked_sub(offset)) - .max() - .unwrap_or(0), - ); + let samples = samples.min( + tracks + .iter() + .flat_map(|t| { + let frames = t.data.samples().len() / t.data.channels() as usize; + let poff = if t.offset > 0 { t.offset as usize } else { 0 }; + frames.checked_sub(offset + poff) + }) + .max() + .unwrap_or(0), + );
9-14
: Document offset units and sign semantics.Add a short docstring to prevent unit drift (seconds vs samples) elsewhere.
pub struct AudioRendererTrack<'a> { pub data: &'a AudioData, pub gain: f32, pub stereo_mode: StereoMode, - pub offset: isize, + /// Offset in sample frames. Positive = delay start; negative = start earlier (silence padded). + pub offset: isize, }
39-41
: Micro-opt: avoid recomputing linear gain per sample.
gain_for_db(track.gain)
is invariant acrossi
; precompute once per track before the inner loop or store linear gain inAudioRendererTrack
.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
apps/desktop/src/routes/editor/ConfigSidebar.tsx
(5 hunks)crates/audio/src/renderer.rs
(3 hunks)crates/editor/src/audio.rs
(4 hunks)crates/editor/src/segments.rs
(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- apps/desktop/src/routes/editor/ConfigSidebar.tsx
🧰 Additional context used
📓 Path-based instructions (2)
{crates,apps/cli,apps/desktop/src-tauri}/**/*.rs
📄 CodeRabbit inference engine (AGENTS.md)
{crates,apps/cli,apps/desktop/src-tauri}/**/*.rs
: Rust code must be formatted with rustfmt and adhere to workspace clippy lints
Rust module files should use snake_case names
Files:
crates/editor/src/segments.rs
crates/audio/src/renderer.rs
crates/editor/src/audio.rs
{crates,apps/cli,apps/desktop/src-tauri}/**/{src,tests}/**/*.rs
📄 CodeRabbit inference engine (AGENTS.md)
Rust tests should live in src (inline/unit) or tests (integration) directories
Files:
crates/editor/src/segments.rs
crates/audio/src/renderer.rs
crates/editor/src/audio.rs
🧬 Code graph analysis (3)
crates/editor/src/segments.rs (1)
crates/editor/src/audio.rs (4)
new
(44-56)new
(84-93)new
(242-261)new
(314-334)
crates/audio/src/renderer.rs (2)
apps/desktop/src/utils/tauri.ts (1)
StereoMode
(441-441)crates/editor/src/audio.rs (4)
tracks
(194-216)offset
(70-72)data
(58-60)gain
(62-64)
crates/editor/src/audio.rs (2)
apps/desktop/src/utils/tauri.ts (5)
StereoMode
(441-441)AudioConfiguration
(324-324)ClipOffsets
(353-353)ProjectConfiguration
(416-416)TimelineConfiguration
(444-444)crates/editor/src/playback.rs (1)
start
(43-127)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Clippy
- GitHub Check: Build Desktop (x86_64-pc-windows-msvc, windows-latest)
- GitHub Check: Analyze (rust)
🔇 Additional comments (4)
crates/editor/src/audio.rs (3)
40-40
: LGTM! New offset parameter properly integrated.The
get_offset
function parameter is correctly added to theAudioSegmentTrack
struct and constructor, enabling per-clip offset functionality.Also applies to: 48-48, 54-54
70-72
: LGTM! Clean offset delegation method.The
offset
method properly delegates to the stored closure, maintaining consistency with the existinggain
andstereo_mode
methods.
197-215
: LGTM! Proper offset computation and integration.The offset lookup from clip configuration is handled correctly with a sensible default, and the conversion to samples using the sample rate is appropriate. The multiplication by
SAMPLE_RATE
and cast toisize
properly converts from seconds to sample offsets.crates/editor/src/segments.rs (1)
20-21
: LGTM: per-clip offsets are correctly threaded into mic/system tracks.Non-capturing closures coerce to
fn
, so this matchesAudioSegmentTrack::new
’s new signature. Please confirm ClipOffsets default to zero seconds to avoid accidental shifts when offsets are absent, and double-check unit consistency (UI ms vs stored seconds).Also applies to: 28-29
Allows customising the timing of camera/mic/system audio relative to the screen capture using configuration that applies to all segments that depend on a given clip. Could probably be improved with sliders or something but at least the backend works.

Summary by CodeRabbit
New Features
Improvements
Bug Fixes