Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@olilarkin
Copy link
Member

@olilarkin olilarkin commented Dec 21, 2025

Finally add a cross-platform spectrogram control.

Analysis: With vs Without this Change

  Without the change (old behavior)

  When HasUI() returns false (headless plugins), the OnTimer() pump is completely skipped. This causes several problems:

  1. Parameter change queue never drains

  // IPlugAPIBase.cpp:169 (non-VST3 path)
  while(mParamChangeFromProcessor.ElementsAvailable())
  {
    ParamTuple p;
    mParamChangeFromProcessor.Pop(p);
    SendParameterValueFromDelegate(p.idx, p.value, false); // triggers OnParamChangeUI()
  }
  - When DSP code calls SendParameterValueFromAPI(), values are pushed to mParamChangeFromProcessor
  - Without pumping, this queue fills up indefinitely β†’ memory grows, parameter changes are lost

  2. MIDI messages from processor never forwarded

  while (mMidiMsgsFromProcessor.ElementsAvailable())
  {
    IMidiMsg msg;
    mMidiMsgsFromProcessor.Pop(msg);
    SendMidiMsgFromDelegate(msg);
  }
  - MIDI output from DSP (e.g., arpeggiators, sequencers) never reaches the host β†’ MIDI output broken

  3. SysEx messages from processor never forwarded

  while (mSysExDataFromProcessor.ElementsAvailable())
  {
    SysExData msg;
    mSysExDataFromProcessor.Pop(msg);
    SendSysexMsgFromDelegate({msg.mOffset, msg.mData, msg.mSize});
  }
  - SysEx output from DSP never reaches the host β†’ SysEx output broken

  4. AUv3: No parameter observer registered

  - Host automation changes don't trigger OnParamChangeUI() β†’ linked parameters, state updates broken

  With the change (new behavior)

  All pumps run unconditionally:
  - Queues are drained properly (no memory leak)
  - Parameter changes trigger OnParamChangeUI() even in headless plugins
  - MIDI/SysEx output works regardless of UI presence
  - AUv3 parameter observer always registered

  Conclusion

  1. The queues still need draining even without a UI
  2. OnParamChangeUI() may be overridden for non-visual purposes
  3. MIDI/SysEx forwarding to host is independent of UI
@claude
Copy link

claude bot commented Dec 21, 2025

PR Review: Add Spectrogram Control

Overall this is a solid addition to the iPlug2 controls library. The spectrogram control is feature-rich with smooth scrolling, multiple colormaps, and cross-platform support. Below are my findings:


Code Quality & Best Practices βœ…

Strengths:

  • Clean code organization with proper namespace usage
  • Good use of C++17 features (std::clamp, auto)
  • Member variable naming follows iPlug2 conventions (mCamelCase)
  • Proper doxygen documentation for public API
  • Template-based design consistent with other controls (e.g., IVBarGraphSpectrumAnalyzerControl)

Issues:

  1. Line 100: Enum mismatch in ColorMapName() - references EColorMap::Bone but enum defines EColorMap::GreyScale

    case EColorMap::Bone:   return "Greyscale";  // Should be EColorMap::GreyScale
  2. Line 527-529: Empty PostDraw() method should be removed (avoid dead code)

  3. Missing #include <chrono>: ITransport uses std::chrono::high_resolution_clock (line 442) but doesn't include the header

  4. Missing TimePoint typedef: Line 521-522 use TimePoint without defining it. Should add to ITransport class:

    using TimePoint = clock::time_point;

Potential Bugs πŸ›

  1. Critical - Line 491: Typo in documentation: "incomming" β†’ "incoming"

  2. Line 416-422: Redundant logic in SetScrollEnabled():

    if (mScrollEnabled && !enabled)
      mTransport.Reset();
    mScrollEnabled = enabled;
    if (!mScrollEnabled)  // This condition is redundant
      mTransport.Reset();

    Should be simplified to avoid double reset.

  3. Line 770: Loop starts at i = 1 instead of i = 0 in ScaleSpectrogramData(), potentially leaving powerSpectrum[0] unprocessed when using frequency scaling. This may be intentional for DC bin but should be commented.

  4. Lines 850-867: CheckSpectrogramDataSize() calls ResetSpectrogramData() which overwrites all magnitude data. On window resize, this clears the spectrogram history unnecessarily. Consider preserving existing data or only resetting when dimensions actually change.

  5. Line 924: Integer division could cause unexpected behavior:

    int scrollSpeedCoeff = static_cast<int>(8.0 / mScrollSpeed);

    If mScrollSpeed is 1, coefficient is 8 (correct). But this truncates - consider explicit rounding.


Performance Considerations ⚑

  1. Line 617-620 (Skia path): Creating a new SkImage from pixel buffer every frame is expensive:

    // Create SkImage from pixel buffer (recreated each frame for simplicity)

    Consider caching the SkImage and only recreating when mImageNeedsUpdate is true (similar to NanoVG path).

  2. Lines 826-831: RecolorAllPixels() processes entire buffer when changing colormaps. For large spectrograms (e.g., 2048 bins Γ— 256 rows), this could cause a frame hitch. Not critical but worth noting.

  3. Line 300: IsDirty() always returns true for continuous refresh. Acceptable for visualization controls but worth documenting energy impact for mobile/battery-powered devices.


Memory Safety πŸ”’

  1. Lines 805-806: Raw pointer arithmetic on vector data is safe here, but ensure mPixelBuffer and mMagnitudeBuffer are sized correctly. The CheckSpectrogramDataSize() handles this, but there's a window between construction and first CheckSpectrogramDataSize() call where buffers might be empty. Constructor calls CheckSpectrogramDataSize() at line 207, so this is actually safe βœ…

  2. Line 570-571: NanoVG image handle cleanup is correct - deletes old image before creating new one.


API Design 🎨

  1. Inconsistent setters: Some setters trigger ResetSpectrogramData() (lines 382, 398) while others don't. Consider documenting which operations clear history.

  2. Line 283-290: UI changes to FFT size/overlap send messages to delegate, but other settings (colormap, direction) don't. Intentional design, but may confuse users expecting consistent behavior.

  3. Missing validation: Constructor accepts fftSize without validating it's a power of 2 (typical FFT requirement). Constructor doesn't check fftSize <= MAX_FFT_SIZE until SetFFTSize() assertion.


Test Coverage πŸ§ͺ

The example usage in IPlugVisualizer.cpp is minimal - it instantiates the control but doesn't exercise:

  • Colormap changes
  • Frequency scale changes
  • Direction toggling
  • Dynamic FFT size changes
  • Multi-channel handling

Consider adding more comprehensive test/example code.


Minor Issues

  1. Line 100: ColorMapName() returns string literal - should be marked const for the entire function
  2. Line 172: Unused kMsgTagWindowType enum value
  3. Line 960: mCursorFreq is calculated but never displayed or used - consider removing or adding tooltip feature
  4. Style consistency: Mix of auto (line 762) and explicit types elsewhere - consider consistent approach per CLAUDE.md

Recommendations

Must fix before merge:

  • Fix EColorMap::Bone β†’ EColorMap::GreyScale (line 100)
  • Add missing #include <chrono> and TimePoint typedef
  • Fix typo "incomming" β†’ "incoming"

Should fix:

  • Remove empty PostDraw() method
  • Optimize Skia backend to cache SkImage like NanoVG does
  • Simplify SetScrollEnabled() redundant reset logic
  • Add comment explaining why ScaleSpectrogramData() starts at i=1

Nice to have:

  • Validate FFT size is power of 2 in constructor
  • Add tooltip showing frequency under cursor using mCursorFreq
  • More comprehensive example/test code

Great work overall! This is a well-implemented control that will be a valuable addition to the framework. The main concerns are the enum typo and missing header - everything else is polish.

@olilarkin olilarkin force-pushed the controls/spectrogram-new branch from fc814b7 to c7aef74 Compare December 22, 2025 08:16
@claude
Copy link

claude bot commented Dec 22, 2025

This is a comprehensive addition that brings a high-quality spectrogram visualization control to iPlug2. The implementation is solid with good attention to performance and cross-platform compatibility.

Critical Issues to Fix Before Merge

1. Clock Precision Bug (ISpectrogramControl.h:486, 497)
The .count() on duration returns nanoseconds, but ComputeTimeDifference treats it as milliseconds. Fix: use std::chrono::duration<double, std::milli>(now - mStartTime).count()

2. Division by Zero Risk (ISpectrogramControl.h:836)
If mColorMapContrast equals 1.0, causes division by zero in ContrastRange. Add clamping: contrast = std::clamp(contrast, 0.0f, 0.999f)

Minor Issues

3. Missing Runtime Validation
SetFFTSize uses assert which compiles out in release. Add explicit validation that fftSize is power of 2.

4. Unused Parameters
Parameter ch in ScaleSpectrogramData and UpdateSpectrogram is never used. Remove or add (void)ch to suppress warnings.

5. Message Tag Inconsistency
IPlugVisualizer.cpp uses IVSpectrumAnalyzerControl<>::kMsgTagFFTSize instead of ISpectrogramControl<>::kMsgTagFFTSize

Performance Notes

  • Skia recreates SkImage every frame (line 611-614) vs NanoVG using efficient nvgUpdateImage
  • IsDirty always returns true even when scrolling disabled - consider return mScrollEnabled || mImageNeedsUpdate
  • Commented code in IPlugVisualizer.cpp (lines 35-76) should be removed

Strengths

  • Excellent documentation and cross-platform support (NanoVG/Skia)
  • Feature-rich: multiple frequency scales, colormaps, smooth scrolling with drift correction
  • Follows iPlug2 conventions perfectly
  • Good memory management and input validation

Recommendation

Approve with minor fixes. Address issues #1 and #2 (critical bugs) before merging. Others are minor.

Great addition to iPlug2!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants