Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 7e8568a

Browse files
authored
Merge pull request ruvnet#91 from ruvnet/fix/issue-86-live-demo-real-data
fix: live demo accuracy, data source transparency, dark mode UI (issue ruvnet#86)
2 parents fdc7142 + 51140f5 commit 7e8568a

17 files changed

Lines changed: 2082 additions & 769 deletions

File tree

README.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ WiFi DensePose turns commodity WiFi signals into real-time human pose estimation
1212
[![ESP32 Ready](https://img.shields.io/badge/ESP32--S3-CSI%20streaming-purple.svg)](#esp32-s3-hardware-pipeline)
1313
[![crates.io](https://img.shields.io/crates/v/wifi-densepose-ruvector.svg)](https://crates.io/crates/wifi-densepose-ruvector)
1414

15+
1516
> | What | How | Speed |
1617
> |------|-----|-------|
1718
> | **Pose estimation** | CSI subcarrier amplitude/phase → DensePose UV maps | 54K fps (Rust) |
@@ -54,6 +55,12 @@ docker run -p 3000:3000 ruvnet/wifi-densepose:latest
5455

5556
---
5657

58+
59+
<img src="assets/screen.png" alt="WiFi DensePose — Live pose detection with setup guide" width="800">
60+
<br>
61+
<em>Real-time pose skeleton from WiFi CSI signals — no cameras, no wearables</em>
62+
63+
5764
## 🚀 Key Features
5865

5966
### Sensing

assets/screen.png

270 KB
Loading

docker/Dockerfile.rust

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,5 +42,15 @@ EXPOSE 5005/udp
4242

4343
ENV RUST_LOG=info
4444

45-
ENTRYPOINT ["/app/sensing-server"]
46-
CMD ["--source", "simulated", "--tick-ms", "100", "--ui-path", "/app/ui", "--http-port", "3000", "--ws-port", "3001"]
45+
# CSI_SOURCE controls which data source the sensing server uses at startup.
46+
# auto — probe UDP port 5005 for an ESP32 first; fall back to simulation (default)
47+
# esp32 — receive real CSI frames from an ESP32 device over UDP port 5005
48+
# wifi — use host Wi-Fi RSSI/scan data (Windows netsh; not available in containers)
49+
# simulated — generate synthetic CSI frames (no hardware required)
50+
# Override at runtime: docker run -e CSI_SOURCE=esp32 ...
51+
ENV CSI_SOURCE=auto
52+
53+
ENTRYPOINT ["/bin/sh", "-c"]
54+
# Shell-form CMD allows $CSI_SOURCE to be substituted at container start.
55+
# The ENV default above (CSI_SOURCE=auto) applies when the variable is unset.
56+
CMD ["/app/sensing-server --source ${CSI_SOURCE} --tick-ms 100 --ui-path /app/ui --http-port 3000 --ws-port 3001"]

docker/docker-compose.yml

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,14 @@ services:
1212
- "5005:5005/udp" # ESP32 UDP
1313
environment:
1414
- RUST_LOG=info
15-
command: ["--source", "simulated", "--tick-ms", "100", "--ui-path", "/app/ui", "--http-port", "3000", "--ws-port", "3001"]
15+
# CSI_SOURCE controls the data source for the sensing server.
16+
# Options: auto (default) — probe for ESP32 UDP then fall back to simulation
17+
# esp32 — receive real CSI frames from an ESP32 on UDP port 5005
18+
# wifi — use host Wi-Fi RSSI/scan data (Windows netsh)
19+
# simulated — generate synthetic CSI data (no hardware required)
20+
- CSI_SOURCE=${CSI_SOURCE:-auto}
21+
# command is passed as arguments to ENTRYPOINT (/bin/sh -c), so $CSI_SOURCE is expanded by the shell.
22+
command: ["/app/sensing-server --source ${CSI_SOURCE:-auto} --tick-ms 100 --ui-path /app/ui --http-port 3000 --ws-port 3001"]
1623

1724
python-sensing:
1825
build:
Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
# ADR-035: Live Sensing UI Accuracy & Data Source Transparency
2+
3+
## Status
4+
Accepted
5+
6+
## Date
7+
2026-03-02
8+
9+
## Context
10+
11+
Issue #86 reported that the live demo shows a static/barely-animated stick figure and the sensing page displays inaccurate data, despite a working ESP32 sending real CSI frames. Investigation revealed three root causes:
12+
13+
1. **Docker defaults to `--source simulated`** — even with a real ESP32 connected, the server generates synthetic sine-wave data instead of reading UDP frames.
14+
2. **Live demo pose is analytically computed**`derive_pose_from_sensing()` generates keypoints using `sin(tick)` math unrelated to actual signal content. No trained `.rvf` model is loaded by default.
15+
3. **Sensing feature extraction is oversimplified** — the server uses single-frame thresholds for motion detection and has no temporal analysis (breathing FFT, sliding window variance, frame history).
16+
4. **No data source indicator** — users cannot tell whether they are seeing real or simulated data.
17+
18+
## Decision
19+
20+
### 1. Docker: Auto-detect data source
21+
- Default `CSI_SOURCE` changed from `simulated` to `auto`.
22+
- `auto` probes UDP port 5005 for an ESP32; falls back to simulation if none found.
23+
- Users override via `CSI_SOURCE=esp32 docker-compose up`.
24+
25+
### 2. Signal-responsive pose derivation
26+
- `derive_pose_from_sensing()` now reads actual sensing features:
27+
- `motion_band_power` drives limb splay and walking gait detection (> 0.55).
28+
- `breathing_band_power` drives torso expansion/contraction phased to breathing rate.
29+
- `variance` seeds per-joint noise so the skeleton moves independently.
30+
- `dominant_freq_hz` drives lateral torso lean.
31+
- `change_points` add burst jitter to extremity keypoints.
32+
- Tick rate reduced from 500ms to 100ms (2 fps → 10 fps).
33+
- `pose_source` field (`signal_derived` | `model_inference`) added to every WebSocket frame.
34+
35+
### 3. Temporal feature extraction
36+
- 100-frame circular buffer (`VecDeque`) added to `AppStateInner`.
37+
- Per-subcarrier temporal variance via Welford-style accumulation.
38+
- Breathing rate estimation via 9-candidate Goertzel filter bank (0.1–0.5 Hz) with 3x SNR gate.
39+
- Frame-to-frame L2 motion score replaces single-frame amplitude thresholds.
40+
- Signal quality metric: SNR-based (RSSI − noise floor) blended with temporal stability.
41+
- Signal field driven by subcarrier variance spatial mapping instead of fixed animation.
42+
43+
### 4. Data source transparency in UI
44+
- **Sensing tab**: Banner showing "LIVE - ESP32" (green), "RECONNECTING..." (yellow), or "SIMULATED DATA" (red).
45+
- **Live Demo tab**: "Estimation Mode" badge showing "Signal-Derived" (green) or "Model Inference" (blue).
46+
- **Setup Guide** panel explaining what each ESP32 count provides (1x: presence/breathing, 3x: localization, 4x+: full pose with trained model).
47+
- Simulation fallback delayed from immediate to 5 failed reconnect attempts (~30s).
48+
49+
## Consequences
50+
51+
### Positive
52+
- Users with real ESP32 hardware get real data by default (auto-detect).
53+
- Simulated data is clearly labeled — no more confusion about data authenticity.
54+
- Pose skeleton visually responds to actual signal changes (motion, breathing, variance).
55+
- Feature extraction produces physiologically meaningful metrics (breathing rate via Goertzel, temporal motion detection).
56+
- Setup guide manages expectations about what each hardware configuration provides.
57+
58+
### Negative
59+
- Signal-derived pose is still an approximation, not neural network inference. Per-limb tracking requires a trained `.rvf` model + 4+ ESP32 nodes.
60+
- Goertzel filter bank adds ~O(9×N) computation per frame (negligible at 100 frames).
61+
- Users with only 1 ESP32 may still be disappointed that arm tracking doesn't work — but the UI now explains why.
62+
63+
### 5. Dark mode consistency
64+
- Live Demo tab converted from light theme to dark mode matching the rest of the UI.
65+
- All sidebar panels, badges, buttons, dropdowns use dark backgrounds with muted text.
66+
67+
### 6. Render mode implementations
68+
All four render modes in the pose visualization dropdown now produce distinct visual output:
69+
70+
| Mode | Rendering |
71+
|------|-----------|
72+
| **Skeleton** | Green lines connecting joints + red keypoint dots |
73+
| **Keypoints** | Large colored dots with glow and labels, no connecting lines |
74+
| **Heatmap** | Gaussian radial blobs per keypoint (hue per person), faint skeleton overlay at 25% opacity |
75+
| **Dense** | Body region segmentation with colored filled polygons — head (red), torso (blue), left arm (green), right arm (orange), left leg (purple), right leg (yellow) |
76+
77+
Previously heatmap and dense were stubs that fell back to skeleton mode.
78+
79+
### 7. pose_source passthrough fix
80+
The `pose_source` field from the WebSocket message was being dropped in `convertZoneDataToRestFormat()` in `pose.service.js`. Now passed through so the Estimation Mode badge displays correctly.
81+
82+
## Files Changed
83+
- `docker/Dockerfile.rust``CSI_SOURCE=auto` env, shell entrypoint for variable expansion
84+
- `docker/docker-compose.yml``CSI_SOURCE=${CSI_SOURCE:-auto}`, shell command string
85+
- `wifi-densepose-sensing-server/src/main.rs` — frame history buffer, Goertzel breathing estimation, temporal motion score, signal-driven pose derivation, pose_source field, 100ms tick default
86+
- `ui/services/sensing.service.js``dataSource` state, delayed simulation fallback, `_simulated` marker
87+
- `ui/services/pose.service.js``pose_source` passthrough in data conversion
88+
- `ui/components/SensingTab.js` — data source banner, "About This Data" card
89+
- `ui/components/LiveDemoTab.js` — estimation mode badge, setup guide panel, dark mode theme
90+
- `ui/utils/pose-renderer.js` — heatmap (Gaussian blobs) and dense (body region segmentation) render modes
91+
- `ui/style.css` — banner, badge, guide panel, and about-text styles
92+
- `README.md` — live pose detection screenshot
93+
- `assets/screen.png` — screenshot asset
94+
95+
## References
96+
- Issue: https://github.com/ruvnet/wifi-densepose/issues/86
97+
- ADR-029: RuvSense multistatic sensing mode (proposed — full pipeline integration)
98+
- ADR-014: SOTA signal processing

0 commit comments

Comments
 (0)