Add mirror_horizontal for holistic poses#222
Merged
Merged
Conversation
Horizontally mirrors a holistic pose to match what mediapipe produces on a left-right flipped image: - image-space points reflected as width - x - body landmarks relabelled via FLIPPED_BODY_POINTS (LEFT<->RIGHT) - LEFT_HAND_LANDMARKS / RIGHT_HAND_LANDMARKS swapped - face mesh reindexed via FLIPPED_FACE_POINTS, derived from mediapipe's symmetric canonical face model (reflect x, nearest vertex) plus iris swaps - POSE_WORLD_LANDMARKS left unchanged: mediapipe reconstructs world landmarks in a canonical 3D frame and does not mirror them under image flip FLIPPED_BODY_POINTS is now computed from BODY_POINTS instead of hardcoded. Tested against holistic run on the actually-flipped image: every point of each flipping component lands within 10px of the flipped-image run (with and without refine_face_landmarks). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
- precompute name->index map per component (removes O(n^2) list.index scans)
- replace if/elif map selection with a {component: flip_map} table
- build result body via type(pose.body)(...) instead of copy-then-overwrite
- test: slice world block explicitly; add _component lookup helper
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
mirror_horizontal(pose)topose_format/utils/holistic.py— horizontally mirrors a holistic pose so it matches what MediaPipe would produce on a left-right flipped image.Flipping the x coordinate alone is not enough, because mirroring swaps the subject's left and right. The function:
width - xFLIPPED_BODY_POINTS(LEFT↔RIGHT)LEFT_HAND_LANDMARKS↔RIGHT_HAND_LANDMARKSFLIPPED_FACE_POINTSpermutationPOSE_WORLD_LANDMARKSunchangedFace permutation
FLIPPED_FACE_POINTS(478 entries) is derived from MediaPipe's symmetric canonical face model — reflect each vertex's x and take the nearest vertex (exact, clean involution) — plus the 10 iris swaps. The provenance is documented in a comment. The first 468 entries are self-contained, so it works withrefine_face_landmarkson (478) or off (468).Why world landmarks are left unchanged
Single-image 2D→3D lifting has an inherent left/right ambiguity, and MediaPipe's world-landmark head resolves it to a fixed canonical frame. Re-running holistic on a flipped image returns essentially the same world landmarks (verified: identity matches at ~8px mean vs. ~18px for a geometric mirror), so a faithful mirror leaves them as-is.
FLIPPED_BODY_POINTSis now computed fromBODY_POINTSvia a LEFT↔RIGHT swap instead of being hardcoded.Testing
tests/holistic_mirror_test.pyruns holistic on a committed test image and on the actually-flipped image, then asserts that for every point of each flipping component,mirror_horizontal(original)lands within 10px of the flipped-image run — with and withoutrefine_face_landmarks. Also covers: face-permutation involution, header unchanged, double-mirror is identity, confidence is a permutation, world landmarks unchanged, and rejection of non-holistic poses.🤖 Generated with Claude Code