Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

Brendonovich
Copy link
Member

@Brendonovich Brendonovich commented Sep 11, 2025

Allows customising the timing of camera/mic/system audio relative to the screen capture using configuration that applies to all segments that depend on a given clip. Could probably be improved with sliders or something but at least the backend works.
image

Summary by CodeRabbit

  • New Features

    • Per-clip offset editing for System Audio, Microphone, and Camera with millisecond input and quick +/- controls.
  • Improvements

    • Clip configuration now opens on mouse release for quicker access.
    • Preview and playback honor per-clip offsets for improved A/V sync.
    • Presets no longer include timeline or clip data, preventing unintended preset state.
  • Bug Fixes

    • More consistent playhead updates during audio playback, reducing jitter.

Copy link
Contributor

coderabbitai bot commented Sep 11, 2025

Walkthrough

Adds per-clip source offsets (camera/mic/system) across UI, project config, rendering, and audio. Threads ClipOffsets through editor, playback, and rendering pipelines, updates audio renderer to use per-track offsets, and adds UI controls to edit offsets in milliseconds stored as seconds. Minor preset and selection tweaks.

Changes

Cohort / File(s) Summary
Editor UI: Clip offsets & selection
apps/desktop/src/routes/editor/ConfigSidebar.tsx, apps/desktop/src/routes/editor/Timeline/ClipTrack.tsx
Add SourceOffsetField UI, helpers (clipConfig, offsets, setOffset), store offsets (UI ms → seconds), ensure clip object exists when saving offsets. Timeline now always selects/opens clip config on mouseup.
Desktop utils & types
apps/desktop/src/utils/createPresets.ts, apps/desktop/src/utils/tauri.ts
Presets: clear clips when creating preset. Tauri types: add ClipOffsets and ClipConfiguration; extend ProjectConfiguration with optional clips.
Project configuration model
crates/project/src/configuration.rs
Add ClipOffsets and ClipConfiguration types; add clips: Vec<ClipConfiguration> to ProjectConfiguration; unify Default derives and apply serde camelCase where applicable.
Rendering pipeline (video/frames)
crates/rendering/src/lib.rs, crates/editor/src/editor_instance.rs, crates/editor/src/playback.rs
Thread per-segment ClipOffsets into RecordingSegmentDecoders::get_frames (signature now takes offsets); callers compute per-segment clip offsets and pass them; playback uses offsets when requesting frames; playhead update timing adjusted.
Audio engine & editor audio integration
crates/audio/src/renderer.rs, crates/editor/src/audio.rs, crates/editor/src/segments.rs
Introduce AudioRendererTrack { data, gain, stereo_mode, offset }; render_audio now accepts &[AudioRendererTrack] and writes into provided output buffer; editor builds tracks with offsets derived from ClipOffsets; AudioSegmentTrack gains offset-aware constructor and method.
Misc formatting
crates/scap-direct3d/src/lib.rs
Remove an extra blank line (no functional change).

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant UI as ConfigSidebar (UI)
  participant Store as Editor Store
  participant Render as Playback/Renderer
  participant Dec as RecordingSegmentDecoders
  Note over UI,Store: User edits offsets (ms) → store saves seconds
  UI->>Store: setOffset(type, ms) -> store.seconds
  Store-->>Render: Project updated (clips)
  Render->>Dec: get_frames(segment_time, needs_camera, clip_offsets)
  Note right of Dec: uses offsets.camera to fetch camera frame
  Dec-->>Render: Decoded frames
Loading
sequenceDiagram
  autonumber
  participant EAudio as Editor Audio
  participant AR as Audio Renderer
  participant Buf as Output Buffer
  Note over EAudio,AR: Build tracks with per-clip offsets
  EAudio->>AR: render_audio([AudioRendererTrack{data,gain,stereo_mode,offset}], ...)
  AR->>AR: Mix per-track samples using track.offset
  AR-->>Buf: Write interleaved stereo samples
Loading
sequenceDiagram
  autonumber
  participant Track as ClipTrack
  participant Store as Editor Store
  Note over Track: Mouseup on clip segment
  Track->>Store: select clip at index (always open config)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Pre-merge checks (2 passed, 1 warning)

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title "Clip source offsetting" is a short, single-line summary that directly reflects the PR’s primary change — adding per-clip offsets for camera, microphone, and system audio across UI, config types, and rendering/playback code. It is concise, specific to the main functionality shown in the diffs (ClipOffsets/ClipConfiguration, SourceOffsetField, and downstream rendering updates), and avoids noisy wording. A reviewer scanning history will understand the primary intent from this title alone.

Poem

In timelines I nibble, hop-hop with delight,
Offsets in milliseconds tucked away just right.
Camera, mic, system—each clip finds its beat,
Tracks line up neatly, audio and video meet.
A rabbit's small blessing for frames now in sync. 🥕

Tip

👮 Agentic pre-merge checks are now available in preview!

Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.

  • Built-in checks – Quickly apply ready-made checks to enforce title conventions, require pull request descriptions that follow templates, validate linked issues for compliance, and more.
  • Custom agentic checks – Define your own rules using CodeRabbit’s advanced agentic capabilities to enforce organization-specific policies and workflows. For example, you can instruct CodeRabbit’s agent to verify that API documentation is updated whenever API schema files are modified in a PR. Note: Upto 5 custom checks are currently allowed during the preview period. Pricing for this feature will be announced in a few weeks.

Please see the documentation for more information.

Example:

reviews:
  pre_merge_checks:
    custom_checks:
      - name: "Undocumented Breaking Changes"
        mode: "warning"
        instructions: |
          Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).

Please share your feedback with us on this Discord post.

✨ Finishing touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch editor-clip-offset

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (9)
apps/desktop/src/utils/createPresets.ts (1)

31-31: Preset configs should exclude clip data at the type level.

Instead of mutating config.clips to undefined, omit it in the type to prevent accidental inclusion and avoid runtime mutation.

Apply:

-export type CreatePreset = {
-  name: string;
-  config: Omit<ProjectConfiguration, "timeline">;
-  default: boolean;
-};
+export type CreatePreset = {
+  name: string;
+  config: Omit<ProjectConfiguration, "timeline" | "clips">;
+  default: boolean;
+};
...
-      // @ts-expect-error we reeeally don't want the timeline in the preset
-      config.timeline = undefined;
-      config.clips = undefined;
+      // @ts-expect-error we reeeally don't want the timeline in the preset
+      config.timeline = undefined;
apps/desktop/src/routes/editor/Timeline/ClipTrack.tsx (1)

358-374: Confirm UX: always opening clip configuration on mouseup.

This removes the previous “single-segment no-open” behavior. If intentional, LGTM; if not, gate on segments().length > 1 as before.

- setEditorState("timeline", "selection", {
-   type: "clip",
-   index: i(),
- });
+// If desired to keep prior UX:
+// if (segments().length > 1) {
+setEditorState("timeline", "selection", { type: "clip", index: i() });
+// }
crates/rendering/src/lib.rs (1)

139-149: Guard camera time against negatives.

If offsets.camera can be negative, request a frame at < 0 may return None or waste work. Safer to clamp to 0.

- .map(|d| d.get_frame(segment_time + offsets.camera)))
+ .map(|d| {
+     let camera_time = (segment_time + offsets.camera).max(0.0);
+     d.get_frame(camera_time)
+ }))
crates/editor/src/playback.rs (1)

253-258: Remove now-unused prev_audio_config.

After always calling set_playhead, prev_audio_config is written but never read.

-        let mut prev_audio_config = project.borrow().audio.clone();
...
-                audio_renderer.set_playhead(
+                audio_renderer.set_playhead(
                     playhead + elapsed as f64 / output_info.sample_rate as f64,
                     &project,
                 );
-                prev_audio_config = project.audio.clone();
apps/desktop/src/routes/editor/ConfigSidebar.tsx (1)

2426-2479: Keep NumberField state in sync with props and avoid wraparound on empty/NaN input.

  • Local value doesn’t react if props.value changes (switching clips). Add a sync effect.
  • Use an explicit empty/NaN check on blur; !rawValue() also triggers for 0.

Apply this diff:

 const rawValue = () => Math.round((props.value ?? 0) * 1000);
 
 const [value, setValue] = createSignal(rawValue().toString());
+createEffect(() => {
+  setValue(rawValue().toString());
+});
@@
-            onBlur={() => {
-              if (!rawValue() || value() === "" || Number.isNaN(rawValue())) {
-                setValue("0");
-                props.onChange(0);
-              }
-            }}
+            onBlur={() => {
+              const v = value();
+              const n = Number.parseInt(v, 10);
+              if (Number.isNaN(n) || v.trim() === "") {
+                setValue("0");
+                props.onChange(0);
+              }
+            }}
crates/audio/src/renderer.rs (1)

24-30: Consider bounding samples by the maximum positive per-track offset.

Tracks delayed by a positive track.offset reduce the readable window from the shared offset. Without accounting for this, the mixer may emit trailing silence where some tracks haven’t reached valid data yet.

You can precompute max_right_shift = tracks.iter().map(|t| t.offset.max(0)).max().unwrap_or(0) as usize and cap samples by (frames_per_track - offset - max_right_shift) to better reflect the tightest window.

crates/project/src/configuration.rs (3)

600-606: Make offsets tolerant to omission

If a clip entry is created with only an index, deserialization will currently fail because offsets is required. Default it to zeroed offsets.

 #[derive(Type, Serialize, Deserialize, Clone, Debug, Default)]
 #[serde(rename_all = "camelCase")]
 pub struct ClipConfiguration {
     pub index: u32,
-    pub offsets: ClipOffsets,
+    #[serde(default)]
+    pub offsets: ClipOffsets,
 }

603-604: Consider a stable clip identifier instead of positional index

Indices can shift on reorder/delete; a stable id (e.g., UUID or recording segment id) avoids misapplying offsets after edits.


590-598: Clarify units/sign and harden (alias + struct-level serde default) for ClipOffsets

  • Add short field docs (seconds; positive = delay, negative = advance), move per-field serde defaults to a struct-level #[serde(default)], and accept camelCase input for system_audio via #[serde(alias = "systemAudio")]. Repo search shows snake_case "system_audio" is used widely, so alias is backward-compatible.
-#[derive(Type, Serialize, Deserialize, Clone, Copy, Debug, Default)]
-pub struct ClipOffsets {
-    #[serde(default)]
-    pub camera: f32,
-    #[serde(default)]
-    pub mic: f32,
-    #[serde(default)]
-    pub system_audio: f32,
-}
+#[derive(Type, Serialize, Deserialize, Clone, Copy, Debug, Default)]
+#[serde(default)]
+pub struct ClipOffsets {
+    /// Offset in seconds. Positive delays the source relative to screen; negative advances it.
+    pub camera: f32,
+    /// Offset in seconds. Positive delays the source relative to screen; negative advances it.
+    pub mic: f32,
+    /// Offset in seconds. Positive delays the source relative to screen; negative advances it.
+    #[serde(alias = "systemAudio")]
+    pub system_audio: f32,
+}
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d9c4961 and 516d0f1.

📒 Files selected for processing (11)
  • apps/desktop/src/routes/editor/ConfigSidebar.tsx (5 hunks)
  • apps/desktop/src/routes/editor/Timeline/ClipTrack.tsx (1 hunks)
  • apps/desktop/src/utils/createPresets.ts (1 hunks)
  • apps/desktop/src/utils/tauri.ts (2 hunks)
  • crates/audio/src/renderer.rs (3 hunks)
  • crates/editor/src/audio.rs (2 hunks)
  • crates/editor/src/editor_instance.rs (1 hunks)
  • crates/editor/src/playback.rs (2 hunks)
  • crates/project/src/configuration.rs (2 hunks)
  • crates/rendering/src/lib.rs (4 hunks)
  • crates/scap-direct3d/src/lib.rs (0 hunks)
💤 Files with no reviewable changes (1)
  • crates/scap-direct3d/src/lib.rs
🧰 Additional context used
📓 Path-based instructions (7)
**/tauri.ts

📄 CodeRabbit inference engine (CLAUDE.md)

Never edit auto-generated IPC bindings file: tauri.ts

Do not edit auto-generated file: tauri.ts

Files:

  • apps/desktop/src/utils/tauri.ts
apps/desktop/**/*.{ts,tsx}

📄 CodeRabbit inference engine (CLAUDE.md)

apps/desktop/**/*.{ts,tsx}: In the desktop app, rely on unplugin-icons auto-imports; do not manually import icon modules
Use generated tauri_specta commands/events (commands, events) in the desktop frontend; listen to generated events directly
Use @tanstack/solid-query for server state in the desktop app

Files:

  • apps/desktop/src/utils/tauri.ts
  • apps/desktop/src/utils/createPresets.ts
  • apps/desktop/src/routes/editor/ConfigSidebar.tsx
  • apps/desktop/src/routes/editor/Timeline/ClipTrack.tsx
{apps/desktop,packages/ui-solid}/**/*.{ts,tsx}

📄 CodeRabbit inference engine (CLAUDE.md)

Component naming (Solid): components in PascalCase; hooks/utilities in camelCase starting with 'use' where applicable

Files:

  • apps/desktop/src/utils/tauri.ts
  • apps/desktop/src/utils/createPresets.ts
  • apps/desktop/src/routes/editor/ConfigSidebar.tsx
  • apps/desktop/src/routes/editor/Timeline/ClipTrack.tsx
**/*.{ts,tsx}

📄 CodeRabbit inference engine (CLAUDE.md)

Use strict TypeScript and avoid any; prefer shared types from packages

**/*.{ts,tsx}: Use Biome to format/lint TypeScript with a 2-space indent
TypeScript file names should be kebab-case (e.g., user-menu.tsx)

Files:

  • apps/desktop/src/utils/tauri.ts
  • apps/desktop/src/utils/createPresets.ts
  • apps/desktop/src/routes/editor/ConfigSidebar.tsx
  • apps/desktop/src/routes/editor/Timeline/ClipTrack.tsx
{crates,apps/cli,apps/desktop/src-tauri}/**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

{crates,apps/cli,apps/desktop/src-tauri}/**/*.rs: Rust code must be formatted with rustfmt and adhere to workspace clippy lints
Rust module files should use snake_case names

Files:

  • crates/editor/src/editor_instance.rs
  • crates/rendering/src/lib.rs
  • crates/editor/src/audio.rs
  • crates/project/src/configuration.rs
  • crates/audio/src/renderer.rs
  • crates/editor/src/playback.rs
{crates,apps/cli,apps/desktop/src-tauri}/**/{src,tests}/**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

Rust tests should live in src (inline/unit) or tests (integration) directories

Files:

  • crates/editor/src/editor_instance.rs
  • crates/rendering/src/lib.rs
  • crates/editor/src/audio.rs
  • crates/project/src/configuration.rs
  • crates/audio/src/renderer.rs
  • crates/editor/src/playback.rs
**/*.{tsx,jsx}

📄 CodeRabbit inference engine (AGENTS.md)

React/Solid components should be named using PascalCase

Files:

  • apps/desktop/src/routes/editor/ConfigSidebar.tsx
  • apps/desktop/src/routes/editor/Timeline/ClipTrack.tsx
🧠 Learnings (1)
📚 Learning: 2025-09-08T16:48:20.727Z
Learnt from: CR
PR: CapSoftware/Cap#0
File: AGENTS.md:0-0
Timestamp: 2025-09-08T16:48:20.727Z
Learning: Applies to **/tauri.ts : Do not edit auto-generated file: tauri.ts

Applied to files:

  • apps/desktop/src/utils/tauri.ts
🧬 Code graph analysis (6)
crates/rendering/src/lib.rs (2)
apps/desktop/src/utils/tauri.ts (1)
  • ClipOffsets (353-353)
crates/rendering/src/decoder/mod.rs (1)
  • get_frame (32-38)
crates/editor/src/audio.rs (2)
crates/audio/src/lib.rs (1)
  • cast_f32_slice_to_bytes (31-33)
crates/editor/src/playback.rs (1)
  • start (43-127)
crates/project/src/configuration.rs (2)
apps/desktop/src/utils/tauri.ts (5)
  • CaptionsData (349-349)
  • CaptionSegment (347-347)
  • CaptionSettings (348-348)
  • ClipOffsets (353-353)
  • ClipConfiguration (352-352)
apps/desktop/src/store/captions.ts (1)
  • CaptionSettings (12-25)
apps/desktop/src/routes/editor/ConfigSidebar.tsx (4)
apps/desktop/src/utils/tauri.ts (1)
  • ClipOffsets (353-353)
crates/editor/src/editor_instance.rs (1)
  • meta (98-100)
apps/desktop/src/routes/editor/context.ts (1)
  • meta (308-310)
apps/desktop/src/routes/editor/ui.tsx (1)
  • Field (25-47)
crates/audio/src/renderer.rs (2)
apps/desktop/src/utils/tauri.ts (1)
  • StereoMode (441-441)
crates/editor/src/audio.rs (3)
  • tracks (186-208)
  • data (54-56)
  • gain (58-60)
crates/editor/src/playback.rs (1)
crates/rendering/src/lib.rs (1)
  • get_frames (135-159)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: Clippy
  • GitHub Check: Build Desktop (x86_64-pc-windows-msvc, windows-latest)
  • GitHub Check: Build Desktop (aarch64-apple-darwin, macos-latest)
  • GitHub Check: Analyze (rust)
🔇 Additional comments (7)
apps/desktop/src/utils/tauri.ts (1)

416-416: Contract change acknowledged: ProjectConfiguration now includes clips.

Looks good as long as it’s generated. Please ensure UI and backend handle absence (undefined) and default to zero offsets.

crates/editor/src/editor_instance.rs (1)

219-220: Passing offsets through to decoding: LGTM.

Forwarding per-clip offsets to get_frames is correct.

crates/rendering/src/lib.rs (1)

223-236: Per-clip offset threading: LGTM.

Lookup + defaulting and passing into get_frames is correct.

crates/editor/src/playback.rs (1)

86-92: Clip lookup: verify index semantics.

Same as editor_instance.rs — ensure segment_i corresponds to the clip index intended for offsets, not just the current timeline segment index.

crates/editor/src/audio.rs (1)

189-206: ```shell
#!/bin/bash
set -euo pipefail
echo "PWD: $(pwd)"
echo "---- rg for AudioSegmentTrack ----"
rg -n -S --hidden --no-ignore -C3 'AudioSegmentTrack' || true
echo "---- show crates/editor/src/audio.rs (first 400 lines) ----"
if [ -f crates/editor/src/audio.rs ]; then
sed -n '1,400p' crates/editor/src/audio.rs
else
echo "file not found: crates/editor/src/audio.rs"
fi
echo "---- rg for ClipOffsets struct and system_audio ----"
rg -n -S --hidden --no-ignore -P -C3 'struct\s+ClipOffsets' || true
rg -n -S --hidden --no-ignore 'ClipOffsets' -C3 || true
rg -n -S --hidden --no-ignore 'system_audio' -C3 || true
rg -n -S --hidden --no-ignore 'mic' -C3 || true
echo "---- rg for AudioRendererTrack ----"
rg -n -S --hidden --no-ignore 'AudioRendererTrack' -C3 || true


</blockquote></details>
<details>
<summary>crates/project/src/configuration.rs (2)</summary><blockquote>

`583-589`: **LGTM: CaptionsData now implements Default**

This makes missing captions deserialize cleanly (empty segments + default settings).

---

`607-622`: **LGTM: clips added with serde(default) — backward compatible**

Empty Vec on missing field preserves old projects; deriving Default on ProjectConfiguration is consistent with load()/write() behavior.

</blockquote></details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->

Comment on lines +2326 to +2346
const clipConfig = () => project.clips?.[props.segmentIndex];
const offsets = () => clipConfig()?.offsets || {};

function setOffset(type: keyof ClipOffsets, offset: number) {
if (Number.isNaN(offset)) return;

setProject(
produce((proj) => {
const clips = (proj.clips ??= []);
let clip = clips.find(
(clip) => clip.index === (props.segment.recordingSegment ?? 0),
);
if (!clip) {
clip = { index: 0, offsets: {} };
clips.push(clip);
}

clip.offsets[type] = offset / 1000;
}),
);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

⚠️ Potential issue

Fix clip lookup and wrong index when creating a new clip config (offsets currently attach to clip 0).

  • offsets() reads project.clips?.[props.segmentIndex] (timeline index) but setOffset writes to the clip whose index equals props.segment.recordingSegment ?? 0. These indices are different concepts; using array indexing is incorrect.
  • Creating a missing clip with index: 0 for every segment is a bug; it misapplies offsets to clip 0.

Update both the reader and writer to consistently resolve by recordingSegment:

Apply this diff:

-  const clipConfig = () => project.clips?.[props.segmentIndex];
-  const offsets = () => clipConfig()?.offsets || {};
+  const clipIndex = props.segment.recordingSegment ?? 0;
+  const clipConfig = () => project.clips?.find(c => c.index === clipIndex);
+  const offsets = () => clipConfig()?.offsets ?? {};
@@
-    setProject(
-      produce((proj) => {
-        const clips = (proj.clips ??= []);
-        let clip = clips.find(
-          (clip) => clip.index === (props.segment.recordingSegment ?? 0),
-        );
-        if (!clip) {
-          clip = { index: 0, offsets: {} };
-          clips.push(clip);
-        }
-        clip.offsets[type] = offset / 1000;
-      }),
-    );
+    setProject(produce((proj) => {
+      const clips = (proj.clips ??= []);
+      let clip = clips.find((c) => c.index === clipIndex);
+      if (!clip) {
+        clip = { index: clipIndex, offsets: {} };
+        clips.push(clip);
+      }
+      clip.offsets[type] = offset / 1000;
+    }));
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const clipConfig = () => project.clips?.[props.segmentIndex];
const offsets = () => clipConfig()?.offsets || {};
function setOffset(type: keyof ClipOffsets, offset: number) {
if (Number.isNaN(offset)) return;
setProject(
produce((proj) => {
const clips = (proj.clips ??= []);
let clip = clips.find(
(clip) => clip.index === (props.segment.recordingSegment ?? 0),
);
if (!clip) {
clip = { index: 0, offsets: {} };
clips.push(clip);
}
clip.offsets[type] = offset / 1000;
}),
);
}
const clipIndex = props.segment.recordingSegment ?? 0;
const clipConfig = () => project.clips?.find(c => c.index === clipIndex);
const offsets = () => clipConfig()?.offsets ?? {};
function setOffset(type: keyof ClipOffsets, offset: number) {
if (Number.isNaN(offset)) return;
setProject(produce((proj) => {
const clips = (proj.clips ??= []);
let clip = clips.find((c) => c.index === clipIndex);
if (!clip) {
clip = { index: clipIndex, offsets: {} };
clips.push(clip);
}
clip.offsets[type] = offset / 1000;
}));
}
🤖 Prompt for AI Agents
In apps/desktop/src/routes/editor/ConfigSidebar.tsx around lines 2326 to 2346,
the reader uses project.clips?.[props.segmentIndex] (timeline index) while the
writer finds/creates clips by clip.index === (props.segment.recordingSegment ??
0), causing offsets to attach to clip 0; update both to consistently resolve
clips by recordingSegment: change clipConfig() and offsets() to use
props.segment.recordingSegment (with safe fallback) when indexing/reading, and
in setOffset when creating a missing clip set its index to the recordingSegment
value (not 0) so offsets are stored on the correct clip; keep Number.isNaN guard
and the same offset division by 1000.

Comment on lines +352 to 354
export type ClipConfiguration = { index: number; offsets: ClipOffsets }
export type ClipOffsets = { camera?: number; mic?: number; system_audio?: number }
export type CommercialLicense = { licenseKey: string; expiryDate: number | null; refresh: number; activatedOn: number }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Do not manually edit this generated file; ensure these types come from tauri-specta codegen.

If these additions weren’t produced by codegen, please revert and re-generate. Also verify Rust-side shapes: TS shows optional fields (ClipOffsets.camera/mic/system_audio and ProjectConfiguration.clips?), which must match Rust (Option vs plain). Mismatch will desync IPC contracts.

To confirm, please check the Rust definitions:


🏁 Script executed:

#!/bin/bash
# Verify Rust shapes for ProjectConfiguration.clips and ClipOffsets
fd -t f configuration.rs crates | xargs -I{} rg -n -C3 'struct\s+ProjectConfiguration|clips\s*:' {}
fd -t f configuration.rs crates | xargs -I{} rg -n -C3 'struct\s+ClipOffsets|impl\s+Default\s+for\s+ClipOffsets' {}
# Also locate get_segment_time to confirm index semantics
rg -n 'fn\s+get_segment_time' -C3

Length of output: 1593


Do not edit generated tauri.ts — regenerate from tauri-specta; Rust↔TS shapes mismatch

  • ClipOffsets mismatch: Rust (crates/project/src/configuration.rs, ~lines 590–596) defines camera/mic/system_audio as non-optional f32 with #[serde(default)]; TS currently has these as optional (camera?: number, etc.) — regenerate so TS fields are non-optional numbers.
  • ProjectConfiguration.clips mismatch: Rust (crates/project/src/configuration.rs, ~lines 606–622) has pub clips: Vec (#[serde(default)]); TS should use ClipConfiguration[] (not optional).
  • Index semantics: get_segment_time signature (crates/project/src/configuration.rs, lines 511 & 640) is fn get_segment_time(&self, frame_time: f64) -> Option<(f64, u32)> — segment index is a numeric u32; ensure ClipConfiguration.index is numeric and aligned.
  • Action: if apps/desktop/src/utils/tauri.ts was edited manually, revert and re-run tauri-specta codegen to resync IPC contracts.
🤖 Prompt for AI Agents
In apps/desktop/src/utils/tauri.ts around lines 352 to 354, the generated
TypeScript shapes no longer match the Rust definitions: make ClipOffsets fields
non-optional numbers (camera: number; mic: number; system_audio: number), ensure
ClipConfiguration.index is a numeric number (not a string/optional) and
ProjectConfiguration.clips is ClipConfiguration[] (not optional), then revert
any manual edits to this file and re-run the tauri-specta codegen to regenerate
the file so IPC/Rust↔TS contracts are in sync.

Comment on lines 37 to 43
// dbg!(i);
let i = i.wrapping_add_signed(dbg!(track.offset));
// dbg!(i);

let data = track.data;
let gain = gain_for_db(track.gain);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

⚠️ Potential issue

Remove dbg! and fix negative-offset indexing (current wrapping underflows and silences tracks).

wrapping_add_signed on usize makes negative offsets wrap to huge indices, so samples never play until wraparound (practically never). Also dbg! spams logs per sample.

Apply this diff:

-        for track in tracks {
-            // dbg!(i);
-            let i = i.wrapping_add_signed(dbg!(track.offset));
-            // dbg!(i);
+        for track in tracks {
+            // translate frame index by per-track offset in signed space
+            let ti = i as isize + track.offset;
+            if ti < 0 {
+                continue; // not yet reached this track’s first sample
+            }
+            let ti = ti as usize;
 
             let data = track.data;
             let gain = gain_for_db(track.gain);
@@
-            if data.channels() == 1 {
-                if let Some(sample) = data.samples().get(offset + i) {
+            if data.channels() == 1 {
+                if let Some(sample) = data.samples().get(offset + ti) {
                 ...
-            } else if data.channels() == 2 {
-                let base_idx = offset * 2 + i * 2;
+            } else if data.channels() == 2 {
+                let base_idx = (offset + ti) * 2;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// dbg!(i);
let i = i.wrapping_add_signed(dbg!(track.offset));
// dbg!(i);
let data = track.data;
let gain = gain_for_db(track.gain);
for track in tracks {
// translate frame index by per-track offset in signed space
let ti = i as isize + track.offset;
if ti < 0 {
continue; // not yet reached this track’s first sample
}
let ti = ti as usize;
let data = track.data;
let gain = gain_for_db(track.gain);
if data.channels() == 1 {
if let Some(sample) = data.samples().get(offset + ti) {
...
}
} else if data.channels() == 2 {
let base_idx = (offset + ti) * 2;
...
}
}
🤖 Prompt for AI Agents
In crates/audio/src/renderer.rs around lines 37 to 43, remove the dbg! calls and
stop using usize::wrapping_add_signed which wraps negative offsets into huge
indices; instead compute a signed sample position by adding the track offset to
the current sample index in a signed integer, check bounds (if pos < 0 or pos >=
data.len() treat as silence), and only index data when pos is in-range (pos as
usize). Keep gain_for_db(track.gain) as-is.

Comment on lines +214 to 216
let clip_config = project.clips.iter().find(|v| v.index == segment_i);
let clip_offsets = clip_config.map(|v| v.offsets).unwrap_or_default();

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Clip lookup: verify index space.

You’re matching on segment_i from get_segment_time; confirm this aligns with the “clip index” (recording segment index), not timeline segment index. If timeline segments can map to the same recording segment, consider using that mapping for a stable clip index.


🏁 Script executed:

#!/bin/bash
# Inspect get_segment_time return to confirm which index it yields
rg -n 'fn\s+get_segment_time\s*\(' -C4 crates
# Find where TimelineSegment.recordingSegment is used to map timeline->recording
rg -n 'recordingSegment' -C3

Length of output: 12663


🏁 Script executed:

#!/bin/bash
set -euo pipefail
file="crates/project/src/configuration.rs"
echo "=== Matches for pub fn get_segment_time in $file ==="
rg -n "pub fn get_segment_time" "$file" || true

echo
echo "=== Show context around each get_segment_time ==="
while IFS=: read -r f ln rest; do
  start=$((ln - 30)); [ $start -lt 1 ] && start=1
  end=$((ln + 140))
  echo
  echo "---- $f:$ln (showing lines $start-$end) ----"
  sed -n "${start},${end}p" "$f"
done < <(rg -n "pub fn get_segment_time" "$file" || true)

echo
echo "=== Search for TimelineSegment / recording_segment / recordingSegment across repo ==="
rg -nP 'struct\s+TimelineSegment' crates || true
rg -n 'recording_segment' crates || true
rg -n 'recordingSegment' -S || true

Length of output: 351


🏁 Script executed:

#!/bin/bash
set -euo pipefail
cfg="crates/project/src/configuration.rs"
ei="crates/editor/src/editor_instance.rs"

echo "=== file existence ==="
[ -f "$cfg" ] && echo "$cfg: exists" || { echo "$cfg: MISSING"; exit 0; }
[ -f "$ei" ] && echo "$ei: exists" || echo "$ei: MISSING"

echo
echo "=== rg matches for get_segment_time (full repo) ==="
rg -n "get_segment_time" -S || true

echo
echo "=== Matches in $cfg for pub fn get_segment_time ==="
rg -n "pub fn get_segment_time" "$cfg" || true

matches=$(rg -n "pub fn get_segment_time" "$cfg" || true)
if [ -n "$matches" ]; then
  echo
  echo "=== Show context around each get_segment_time ==="
  echo "$matches" | while IFS=: read -r file ln rest; do
    start=$((ln - 40)); [ $start -lt 1 ] && start=1
    end=$((ln + 180))
    echo
    echo "---- $file:$ln (showing lines $start-$end) ----"
    sed -n "${start},${end}p" "$file" || true
  done
else
  echo "No matches in $cfg"
fi

echo
echo "=== Show surrounding types/structs in $cfg (lines 1-220,220-520,520-880) ==="
sed -n '1,220p' "$cfg" || true
echo "----"
sed -n '220,520p' "$cfg" || true
echo "----"
sed -n '520,880p' "$cfg" || true

echo
echo "=== Search for TimelineConfiguration / TimelineSegment / SceneSegment in repo ==="
rg -n "TimelineConfiguration|TimelineSegment|SceneSegment|recordingSegment|recording_segment" -S || true

echo
echo "=== Show editor_instance.rs around lines 180-260 and full file if small ==="
if [ -f "$ei" ]; then
  sed -n '1,120p' "$ei" || true
  echo "----"
  sed -n '121,300p' "$ei" || true
  echo "---- full file size (lines):"
  wc -l "$ei"
fi

echo
echo "=== Search for usages of get_segment_time and where its result is consumed ==="
rg -n "get_segment_time\(" -S || true

echo
echo "=== Search for 'clip_config' and 'clips.iter().find' occurrences ==="
rg -n "clips.iter().find|clip_config" -S || true

echo
echo "=== Search for fields named 'index' inside clip type definitions (rust/ts) ==="
rg -n "struct .*Clip|type .*Clip|clips: Vec|clips:" -S || true

Length of output: 1367


Use the timeline→recording mapping when looking up clips (timeline index != recording index).

get_segment_time returns a timeline-segment index (index into timeline.segments); project.clips are keyed by recording-segment indexes, so using segment_i directly can pick the wrong clip when multiple timeline segments map to the same recording segment. Derive the recording index from the timeline segment's recording_segment/recordingSegment field and use that to find the clip (fallback to segment_i only if the mapping is absent).
Location: crates/editor/src/editor_instance.rs (around lines 214–216).

🤖 Prompt for AI Agents
In crates/editor/src/editor_instance.rs around lines 214 to 216, the code looks
up a clip by using the timeline segment index (segment_i) but project.clips are
keyed by recording-segment indexes; change the lookup to first obtain the
timeline segment at segment_i and read its recording_segment/recordingSegment
field to derive the recording index, then use that recording index to find the
clip in project.clips; if the timeline segment has no recording mapping, fall
back to using segment_i as before; keep the existing unwrap_or_default behavior
for offsets.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (1)
crates/audio/src/renderer.rs (1)

36-41: Fix negative-offset wrapping and deref bugs (silence due to wrap; compile risk on &f32 ops).

  • wrapping_add_signed on usize underflows for negative offsets, producing huge indices and silencing tracks.
  • samples().get(...) returns &f32; multiplying references without deref is invalid; use pattern Some(&x) or *ref.

Apply:

-        for track in tracks {
-            let i = i.wrapping_add_signed(track.offset);
-
-            let data = track.data;
-            let gain = gain_for_db(track.gain);
+        for track in tracks {
+            // translate frame index by per-track offset in signed space
+            let ti = i as isize + track.offset;
+            if ti < 0 {
+                continue; // not yet reached this track’s first sample
+            }
+            let ti = ti as usize;
+
+            let data = track.data;
+            let gain = gain_for_db(track.gain);
@@
-            if data.channels() == 1 {
-                if let Some(sample) = data.samples().get(offset + i) {
-                    left += sample * 0.707 * gain;
-                    right += sample * 0.707 * gain;
-                }
+            if data.channels() == 1 {
+                if let Some(&sample) = data.samples().get(offset + ti) {
+                    left += sample * 0.707 * gain;
+                    right += sample * 0.707 * gain;
+                }
             } else if data.channels() == 2 {
-                let base_idx = offset * 2 + i * 2;
-                let Some(l_sample) = data.samples().get(base_idx) else {
+                let base_idx = (offset + ti) * 2;
+                let Some(&l_sample) = data.samples().get(base_idx) else {
                     continue;
                 };
-                let Some(r_sample) = data.samples().get(base_idx + 1) else {
+                let Some(&r_sample) = data.samples().get(base_idx + 1) else {
                     continue;
                 };

Also applies to: 46-58, 60-73

🧹 Nitpick comments (3)
crates/audio/src/renderer.rs (3)

24-30: Clamp output length using per-track positive offsets.

Current cap uses only frames - offset, ignoring positive per-track offsets that shorten usable frames. Incorporate max(0, track.offset) (in frames) so we don’t overrun logical track ends and waste cycles mixing silence.

-    let samples = samples.min(
-        tracks
-            .iter()
-            .flat_map(|t| (t.data.samples().len() / t.data.channels() as usize).checked_sub(offset))
-            .max()
-            .unwrap_or(0),
-    );
+    let samples = samples.min(
+        tracks
+            .iter()
+            .flat_map(|t| {
+                let frames = t.data.samples().len() / t.data.channels() as usize;
+                let poff = if t.offset > 0 { t.offset as usize } else { 0 };
+                frames.checked_sub(offset + poff)
+            })
+            .max()
+            .unwrap_or(0),
+    );

9-14: Document offset units and sign semantics.

Add a short docstring to prevent unit drift (seconds vs samples) elsewhere.

 pub struct AudioRendererTrack<'a> {
     pub data: &'a AudioData,
     pub gain: f32,
     pub stereo_mode: StereoMode,
-    pub offset: isize,
+    /// Offset in sample frames. Positive = delay start; negative = start earlier (silence padded).
+    pub offset: isize,
 }

39-41: Micro-opt: avoid recomputing linear gain per sample.

gain_for_db(track.gain) is invariant across i; precompute once per track before the inner loop or store linear gain in AudioRendererTrack.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 516d0f1 and 7cc6c17.

📒 Files selected for processing (4)
  • apps/desktop/src/routes/editor/ConfigSidebar.tsx (5 hunks)
  • crates/audio/src/renderer.rs (3 hunks)
  • crates/editor/src/audio.rs (4 hunks)
  • crates/editor/src/segments.rs (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • apps/desktop/src/routes/editor/ConfigSidebar.tsx
🧰 Additional context used
📓 Path-based instructions (2)
{crates,apps/cli,apps/desktop/src-tauri}/**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

{crates,apps/cli,apps/desktop/src-tauri}/**/*.rs: Rust code must be formatted with rustfmt and adhere to workspace clippy lints
Rust module files should use snake_case names

Files:

  • crates/editor/src/segments.rs
  • crates/audio/src/renderer.rs
  • crates/editor/src/audio.rs
{crates,apps/cli,apps/desktop/src-tauri}/**/{src,tests}/**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

Rust tests should live in src (inline/unit) or tests (integration) directories

Files:

  • crates/editor/src/segments.rs
  • crates/audio/src/renderer.rs
  • crates/editor/src/audio.rs
🧬 Code graph analysis (3)
crates/editor/src/segments.rs (1)
crates/editor/src/audio.rs (4)
  • new (44-56)
  • new (84-93)
  • new (242-261)
  • new (314-334)
crates/audio/src/renderer.rs (2)
apps/desktop/src/utils/tauri.ts (1)
  • StereoMode (441-441)
crates/editor/src/audio.rs (4)
  • tracks (194-216)
  • offset (70-72)
  • data (58-60)
  • gain (62-64)
crates/editor/src/audio.rs (2)
apps/desktop/src/utils/tauri.ts (5)
  • StereoMode (441-441)
  • AudioConfiguration (324-324)
  • ClipOffsets (353-353)
  • ProjectConfiguration (416-416)
  • TimelineConfiguration (444-444)
crates/editor/src/playback.rs (1)
  • start (43-127)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Clippy
  • GitHub Check: Build Desktop (x86_64-pc-windows-msvc, windows-latest)
  • GitHub Check: Analyze (rust)
🔇 Additional comments (4)
crates/editor/src/audio.rs (3)

40-40: LGTM! New offset parameter properly integrated.

The get_offset function parameter is correctly added to the AudioSegmentTrack struct and constructor, enabling per-clip offset functionality.

Also applies to: 48-48, 54-54


70-72: LGTM! Clean offset delegation method.

The offset method properly delegates to the stored closure, maintaining consistency with the existing gain and stereo_mode methods.


197-215: LGTM! Proper offset computation and integration.

The offset lookup from clip configuration is handled correctly with a sensible default, and the conversion to samples using the sample rate is appropriate. The multiplication by SAMPLE_RATE and cast to isize properly converts from seconds to sample offsets.

crates/editor/src/segments.rs (1)

20-21: LGTM: per-clip offsets are correctly threaded into mic/system tracks.

Non-capturing closures coerce to fn, so this matches AudioSegmentTrack::new’s new signature. Please confirm ClipOffsets default to zero seconds to avoid accidental shifts when offsets are absent, and double-check unit consistency (UI ms vs stored seconds).

Also applies to: 28-29

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant