|
| 1 | +# ADR-035: Live Sensing UI Accuracy & Data Source Transparency |
| 2 | + |
| 3 | +## Status |
| 4 | +Accepted |
| 5 | + |
| 6 | +## Date |
| 7 | +2026-03-02 |
| 8 | + |
| 9 | +## Context |
| 10 | + |
| 11 | +Issue #86 reported that the live demo shows a static/barely-animated stick figure and the sensing page displays inaccurate data, despite a working ESP32 sending real CSI frames. Investigation revealed three root causes: |
| 12 | + |
| 13 | +1. **Docker defaults to `--source simulated`** — even with a real ESP32 connected, the server generates synthetic sine-wave data instead of reading UDP frames. |
| 14 | +2. **Live demo pose is analytically computed** — `derive_pose_from_sensing()` generates keypoints using `sin(tick)` math unrelated to actual signal content. No trained `.rvf` model is loaded by default. |
| 15 | +3. **Sensing feature extraction is oversimplified** — the server uses single-frame thresholds for motion detection and has no temporal analysis (breathing FFT, sliding window variance, frame history). |
| 16 | +4. **No data source indicator** — users cannot tell whether they are seeing real or simulated data. |
| 17 | + |
| 18 | +## Decision |
| 19 | + |
| 20 | +### 1. Docker: Auto-detect data source |
| 21 | +- Default `CSI_SOURCE` changed from `simulated` to `auto`. |
| 22 | +- `auto` probes UDP port 5005 for an ESP32; falls back to simulation if none found. |
| 23 | +- Users override via `CSI_SOURCE=esp32 docker-compose up`. |
| 24 | + |
| 25 | +### 2. Signal-responsive pose derivation |
| 26 | +- `derive_pose_from_sensing()` now reads actual sensing features: |
| 27 | + - `motion_band_power` drives limb splay and walking gait detection (> 0.55). |
| 28 | + - `breathing_band_power` drives torso expansion/contraction phased to breathing rate. |
| 29 | + - `variance` seeds per-joint noise so the skeleton moves independently. |
| 30 | + - `dominant_freq_hz` drives lateral torso lean. |
| 31 | + - `change_points` add burst jitter to extremity keypoints. |
| 32 | +- Tick rate reduced from 500ms to 100ms (2 fps → 10 fps). |
| 33 | +- `pose_source` field (`signal_derived` | `model_inference`) added to every WebSocket frame. |
| 34 | + |
| 35 | +### 3. Temporal feature extraction |
| 36 | +- 100-frame circular buffer (`VecDeque`) added to `AppStateInner`. |
| 37 | +- Per-subcarrier temporal variance via Welford-style accumulation. |
| 38 | +- Breathing rate estimation via 9-candidate Goertzel filter bank (0.1–0.5 Hz) with 3x SNR gate. |
| 39 | +- Frame-to-frame L2 motion score replaces single-frame amplitude thresholds. |
| 40 | +- Signal quality metric: SNR-based (RSSI − noise floor) blended with temporal stability. |
| 41 | +- Signal field driven by subcarrier variance spatial mapping instead of fixed animation. |
| 42 | + |
| 43 | +### 4. Data source transparency in UI |
| 44 | +- **Sensing tab**: Banner showing "LIVE - ESP32" (green), "RECONNECTING..." (yellow), or "SIMULATED DATA" (red). |
| 45 | +- **Live Demo tab**: "Estimation Mode" badge showing "Signal-Derived" (green) or "Model Inference" (blue). |
| 46 | +- **Setup Guide** panel explaining what each ESP32 count provides (1x: presence/breathing, 3x: localization, 4x+: full pose with trained model). |
| 47 | +- Simulation fallback delayed from immediate to 5 failed reconnect attempts (~30s). |
| 48 | + |
| 49 | +## Consequences |
| 50 | + |
| 51 | +### Positive |
| 52 | +- Users with real ESP32 hardware get real data by default (auto-detect). |
| 53 | +- Simulated data is clearly labeled — no more confusion about data authenticity. |
| 54 | +- Pose skeleton visually responds to actual signal changes (motion, breathing, variance). |
| 55 | +- Feature extraction produces physiologically meaningful metrics (breathing rate via Goertzel, temporal motion detection). |
| 56 | +- Setup guide manages expectations about what each hardware configuration provides. |
| 57 | + |
| 58 | +### Negative |
| 59 | +- Signal-derived pose is still an approximation, not neural network inference. Per-limb tracking requires a trained `.rvf` model + 4+ ESP32 nodes. |
| 60 | +- Goertzel filter bank adds ~O(9×N) computation per frame (negligible at 100 frames). |
| 61 | +- Users with only 1 ESP32 may still be disappointed that arm tracking doesn't work — but the UI now explains why. |
| 62 | + |
| 63 | +### 5. Dark mode consistency |
| 64 | +- Live Demo tab converted from light theme to dark mode matching the rest of the UI. |
| 65 | +- All sidebar panels, badges, buttons, dropdowns use dark backgrounds with muted text. |
| 66 | + |
| 67 | +### 6. Render mode implementations |
| 68 | +All four render modes in the pose visualization dropdown now produce distinct visual output: |
| 69 | + |
| 70 | +| Mode | Rendering | |
| 71 | +|------|-----------| |
| 72 | +| **Skeleton** | Green lines connecting joints + red keypoint dots | |
| 73 | +| **Keypoints** | Large colored dots with glow and labels, no connecting lines | |
| 74 | +| **Heatmap** | Gaussian radial blobs per keypoint (hue per person), faint skeleton overlay at 25% opacity | |
| 75 | +| **Dense** | Body region segmentation with colored filled polygons — head (red), torso (blue), left arm (green), right arm (orange), left leg (purple), right leg (yellow) | |
| 76 | + |
| 77 | +Previously heatmap and dense were stubs that fell back to skeleton mode. |
| 78 | + |
| 79 | +### 7. pose_source passthrough fix |
| 80 | +The `pose_source` field from the WebSocket message was being dropped in `convertZoneDataToRestFormat()` in `pose.service.js`. Now passed through so the Estimation Mode badge displays correctly. |
| 81 | + |
| 82 | +## Files Changed |
| 83 | +- `docker/Dockerfile.rust` — `CSI_SOURCE=auto` env, shell entrypoint for variable expansion |
| 84 | +- `docker/docker-compose.yml` — `CSI_SOURCE=${CSI_SOURCE:-auto}`, shell command string |
| 85 | +- `wifi-densepose-sensing-server/src/main.rs` — frame history buffer, Goertzel breathing estimation, temporal motion score, signal-driven pose derivation, pose_source field, 100ms tick default |
| 86 | +- `ui/services/sensing.service.js` — `dataSource` state, delayed simulation fallback, `_simulated` marker |
| 87 | +- `ui/services/pose.service.js` — `pose_source` passthrough in data conversion |
| 88 | +- `ui/components/SensingTab.js` — data source banner, "About This Data" card |
| 89 | +- `ui/components/LiveDemoTab.js` — estimation mode badge, setup guide panel, dark mode theme |
| 90 | +- `ui/utils/pose-renderer.js` — heatmap (Gaussian blobs) and dense (body region segmentation) render modes |
| 91 | +- `ui/style.css` — banner, badge, guide panel, and about-text styles |
| 92 | +- `README.md` — live pose detection screenshot |
| 93 | +- `assets/screen.png` — screenshot asset |
| 94 | + |
| 95 | +## References |
| 96 | +- Issue: https://github.com/ruvnet/wifi-densepose/issues/86 |
| 97 | +- ADR-029: RuvSense multistatic sensing mode (proposed — full pipeline integration) |
| 98 | +- ADR-014: SOTA signal processing |
0 commit comments