Use saved optical flow to use only on adjacent frames #870
Conversation
There was a problem hiding this comment.
Hi @getzze,
Thanks for the contribution! This looks like some interesting functionality and I'd be curious to see how it works in practice.
Since this is making a change to a core component within SLEAP, do you mind adding a short test to tests/nn/test_tracker_components.py?
We just want to make sure nothing strange happens with the other components and that this runs as expected.
Thanks!!
Talmo
|
Hi, I looked at making a test but it's not so simple as testing this requires having some images to run OpenCV optical flow function on them. I cannot promise I will have time to look at it during the next month. |
Codecov Report
@@ Coverage Diff @@
## develop #870 +/- ##
===========================================
+ Coverage 67.37% 67.40% +0.03%
===========================================
Files 130 130
Lines 22091 22110 +19
===========================================
+ Hits 14883 14903 +20
+ Misses 7208 7207 -1
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
- Unable to push to sleap/gh-pages from fork - Likely do not want forks changing the website regardless
Description
Using tracking with optical flow, similarity between frames is computed with the optical flow between the current frame and the reference frame. This is fine if the 2 frames are close in time, but it works poorly with say 20 time-steps difference, as the optical flow is poorly computed.
Instead, save the instances shifted by optical flow step by step, so the optical flow is always computed between two adjacent frames.
The
save_shifted_instancesargument ofFlowCandidateMakercould be made accessible from the GUI, so we could switch between the 2 options.Types of changes
Does this address any currently open issues?
N/A
Outside contributors checklist
Thank you for contributing to SLEAP!
❤️