Refactor the max_tracking code #1896
Conversation
WalkthroughThe recent updates enhance the tracking capabilities and usability of the system. Key modifications include streamlined command-line usage, refined configuration parameters, improved inference logic, and better handling of tracking instances. These changes collectively aim to simplify user interactions, optimize performance, and clarify documentation, ensuring a more robust and efficient tracking experience. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant CLI
participant Tracker
participant Inference
User->>CLI: Execute tracking command
CLI->>Tracker: Initialize tracker
Tracker->>Inference: Process data
Inference-->>Tracker: Return results
Tracker-->>CLI: Output results
CLI-->>User: Display tracking results
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 6
Outside diff range, codebase verification and nitpick comments (2)
sleap/nn/tracker/components.py (1)
367-369: Clarify the merging penalty description.The description of the
merging_penaltyparameter should mention that it is a value between 0 and 1.- merging_penalty: a float between 0 and 1. All scores of the merged + merging_penalty: A float between 0 and 1. All scores of the mergedsleap/config/pipeline_form.yaml (1)
458-463: Clarify the help text fortracking.save_shifted_instances.The help text should be more explicit about the impact on memory usage.
- help: 'Save the flow-shifted instances between elapsed frames. It improves - instance matching at the cost of using a bit more of memory.' + help: 'Save the flow-shifted instances between elapsed frames. This improves + instance matching but increases memory usage.'
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (13)
- docs/guides/cli.md (2 hunks)
- sleap/config/pipeline_form.yaml (2 hunks)
- sleap/gui/learning/dialog.py (1 hunks)
- sleap/gui/learning/runners.py (6 hunks)
- sleap/gui/widgets/video.py (2 hunks)
- sleap/instance.py (2 hunks)
- sleap/nn/inference.py (13 hunks)
- sleap/nn/tracker/components.py (8 hunks)
- sleap/nn/tracking.py (35 hunks)
- sleap/util.py (2 hunks)
- tests/nn/test_inference.py (8 hunks)
- tests/nn/test_tracker_components.py (8 hunks)
- tests/nn/test_tracking_integration.py (6 hunks)
Files skipped from review due to trivial changes (1)
- docs/guides/cli.md
Additional context used
Ruff
sleap/util.py
38-38: Undefined name
Task(F821)
sleap/nn/tracking.py
7-7:
collections.defaultdictimported but unusedRemove unused import:
collections.defaultdict(F401)
532-532: Use
enumerate()for index variablen_processedinforloop(SIM113)
tests/nn/test_inference.py
1820-1820: Use
key in dictinstead ofkey in dict.keys()Remove
.keys()(SIM118)
sleap/instance.py
21-21:
operatorimported but unusedRemove unused import:
operator(F401)
Additional comments not posted (27)
tests/nn/test_tracking_integration.py (3)
22-22: Verify the change in expected number of tracks.The expected number of tracks has changed from 27 to 8. Ensure this change is intentional and aligns with the updated tracking logic or input data.
Line range hint
25-34:
LGTM! Verify the function's behavior.The function has been renamed and updated to test the max tracking feature. Ensure the function behaves as expected with the new parameters.
44-44: LGTM! Verify the function's behavior.The functions have been updated to use the
tracker.run_trackermethod with verbosity set to 'none'. Ensure the functions behave as expected with the new method.Also applies to: 51-51
tests/nn/test_tracker_components.py (3)
18-29: LGTM! Verify the function's behavior.The function has been renamed and updated to support dynamic adjustment of image scale. Ensure the function behaves as expected with the new parameter.
36-58: LGTM! Verify the test cases.The test function has been updated to include the new
img_scaleparameter. Ensure the test cases validate the behavior of the tracker with varying image scales.
Line range hint
286-298:
LGTM! Verify the test cases.The test function has been updated to include the
max_trackingparameter. Ensure the test cases validate the behavior of the tracker with the new parameter.sleap/nn/tracker/components.py (1)
355-356: Add type hints for new parameters.The new parameters
merge_instancesandmerging_penaltyshould have type hints for better readability and type checking.- merge_instances: bool = False, - merging_penalty: float = 0.2, + merge_instances: bool = False, + merging_penalty: float = 0.2,Likely invalid or redundant comment.
sleap/gui/learning/runners.py (1)
241-247: Ensure compatibility with all tracker names.The compatibility logic should ensure that all tracker names are correctly handled.
Verification successful
Ensure compatibility with all tracker names.
The compatibility logic correctly handles tracker names with the "maxtracks" suffix by stripping the suffix and setting
max_trackingtoTrue. The search results confirm that the tracker names are handled consistently across the codebase.
sleap/gui/learning/runners.py: Lines 241-247Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify all tracker names are correctly handled. # Test: Search for all occurrences of tracker names in the codebase. Expect: Correct handling of tracker names. rg --type python -A 5 $'tracking.tracker'Length of output: 71
Script:
#!/bin/bash # Description: Verify all tracker names are correctly handled. # Test: Search for all occurrences of tracker names in the codebase. Expect: Correct handling of tracker names. rg --type py -A 5 $'tracking.tracker'Length of output: 7299
sleap/gui/learning/dialog.py (1)
739-739: LGTM! But verify the function usage in the codebase.The code changes are approved.
However, ensure that all function calls to
runare tested to confirm the new logic works as expected.sleap/nn/tracking.py (6)
118-131: Abstract base classCandidateMakerlooks good.The abstract method
get_candidatesenforces a consistent interface for candidate makers.
Line range hint
390-405:
ClassSimpleCandidateMakerimplementation looks good.The method for generating candidates from prior frames is well-defined.
Tools
Ruff
408-408: Local variable
ref_tis assigned to but never usedRemove assignment to unused variable
ref_t(F841)
Line range hint
621-892:
ClassTrackerimplementation looks good.The new attributes and methods enhance the control over the tracking process.
However, ensure that all methods are thoroughly tested to confirm the new logic works as expected.
Verification successful
The previous shell script failed because
rgdid not recognize thepythonfile type. I will modify the script to use the correct file type for Python files and rerun the search forTrackerclass usage in the codebase.
Ensure all
Trackermethods are thoroughly tested.The
Trackerclass and its methods are used extensively across multiple test files, includingtests/nn/test_tracking_integration.py,tests/nn/test_tracker_components.py, andtests/nn/test_inference.py. These tests cover various aspects of theTrackerfunctionality, ensuring that the new logic is well-tested.
- Test files:
tests/nn/test_tracking_integration.pytests/nn/test_tracker_components.pytests/nn/test_inference.pyPlease verify that these tests are comprehensive and cover all new functionality introduced in the
Trackerclass.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify all method calls to `Tracker` in the codebase. # Test: Search for the class usage. Expect: Only occurrences of the new logic. rg --type python -A 5 $'Tracker'Length of output: 62
Script:
#!/bin/bash # Description: Verify all method calls to `Tracker` in the codebase. # Test: Search for the class usage. Expect: Only occurrences of the new logic. rg --type py -A 5 TrackerLength of output: 30182
Line range hint
133-384:
ClassFlowCandidateMakerimplementation looks good.The methods for generating and managing shifted instances are well-defined.
However, ensure that all methods are thoroughly tested to confirm the new logic works as expected.
Verification successful
Class
FlowCandidateMakerimplementation looks good.The methods for generating and managing shifted instances are well-defined and the class is being tested in
tests/nn/test_inference.py.
sleap/nn/tracking.pytests/nn/test_inference.pyScripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify all method calls to `FlowCandidateMaker` in the codebase. # Test: Search for the class usage. Expect: Only occurrences of the new logic. rg --type python -A 5 $'FlowCandidateMaker'Length of output: 73
Script:
#!/bin/bash # Description: Verify all method calls to `FlowCandidateMaker` in the codebase. # Test: Search for the class usage. Expect: Only occurrences of the new logic. rg --type py -A 5 $'FlowCandidateMaker'Length of output: 2364
438-572: ClassBaseTrackerimplementation looks good.The new attributes and methods enhance the functionality and flexibility of the tracking process.
However, ensure that all methods are thoroughly tested to confirm the new logic works as expected.
Tools
Ruff
532-532: Use
enumerate()for index variablen_processedinforloop(SIM113)
1425-1430: ClassKalmanTrackerimplementation looks good.The new attributes and methods enhance the robustness and flexibility of the tracking process.
However, ensure that all methods are thoroughly tested to confirm the new logic works as expected.
tests/nn/test_inference.py (4)
Line range hint
1273-1315:
LGTM!The function
test_topdown_predictor_saveis well-structured and covers various aspects of the model saving process. The use of@pytest.mark.slow()is appropriate for this test.
Line range hint
1316-1360:
LGTM!The function
test_topdown_id_predictor_saveis well-structured and comprehensive in testing various aspects of the model saving process. The use of@pytest.mark.slow()is appropriate for this test.
Line range hint
1784-1829:
LGTM!The function
test_max_tracks_matching_queueis well-structured and covers the necessary checks for maximum tracks instance generation. The use of parameterization is appropriate for this test.Tools
Ruff
1820-1820: Use
key in dictinstead ofkey in dict.keys()Remove
.keys()(SIM118)
Line range hint
1851-1873:
LGTM!The function
test_movenet_predictoris well-structured and covers various aspects of theMoveNetPredictormodel. The use of@pytest.mark.slow()is appropriate for this test.sleap/instance.py (1)
1183-1184: LGTM!The
all_disjointfunction is well-implemented and correctly checks for disjoint sequences.sleap/gui/widgets/video.py (2)
807-808: Initialization of_down_posThe addition of
_down_posinitialized toNoneis a good practice to ensure it is defined before being used in other methods.
1032-1032: Safeguard for_down_posThe added check to ensure
_down_posis notNonebefore comparing it with the current mouse position is a robust safeguard against potential errors.sleap/nn/inference.py (5)
48-51: Ensure backward compatibility with Python versions prior to 3.8.The code correctly defines
cached_propertyas a standard property when the version check fails, ensuring backward compatibility.
163-165: Correct use ofcached_propertyforreport_period.The
report_periodmethod is correctly optimized withcached_propertyto cache its value after the first computation.
414-416: Performance improvement by pre-compiling loop examples.Storing the dataset in
examplesbefore entering the loop prevents repeated calls tomake_dataset(), enhancing performance.
4917-4919: Conditional tracker initialization logic.The code correctly checks if
tracker_max_instancesis greater than zero before setting the tracker, ensuring that the tracker is only created when necessary.
5339-5341: Correct handling of labeled frames.The function correctly creates a
LabelsReaderfor labeled frames, ensuring only labeled frames are processed.
There was a problem hiding this comment.
Actionable comments posted: 3
Outside diff range, codebase verification and nitpick comments (15)
tests/nn/test_tracking_integration.py (3)
25-28: Approve changes and suggest documentation updates.The function has been renamed for clarity, and the CLI parameters have been updated to reflect the new tracking configuration.
Consider updating the related documentation to reflect these changes:
- The renaming of the test function from
test_simplemax_trackertotest_simple_max_tracks.- The change in the tracker parameter from "simplemaxtracks" to "simple".
- The addition of the new "--tracking.max_tracks 2" parameter.
This will ensure that the documentation remains in sync with the code changes.
110-116: Approve simplification ofmake_trackerfunction.The
make_trackerfunction has been simplified by removing themax_trackingparameter and related conditional logic. Themax_tracksparameter is now directly passed to the tracker constructor, which aligns with the changes observed in the test functions.Consider updating any related documentation or comments that might reference the old
max_trackingparameter or the removed conditional logic. This will ensure that the documentation accurately reflects the current implementation.
44-44: Complete the transition totracker.run_trackeracross the codebase.The codebase still contains instances of the
run_trackerfunction definition and calls, indicating that the transition to usingtracker.run_trackeris incomplete. To maintain consistency and adhere to object-oriented principles, please update all occurrences ofrun_trackertotracker.run_tracker.
run_trackerfunction is defined insleap/nn/tracking.py.- Calls to
run_trackerare found in:
tests/nn/test_tracking_integration.pytests/nn/test_tracker_components.pysleap/nn/inference.pysleap/nn/tracking.pyAnalysis chain
Approve the change to use
tracker.run_tracker.The modification to use
tracker.run_trackerinstead of a separaterun_trackerfunction improves encapsulation and follows object-oriented principles.To ensure consistency across the codebase, please run the following script to check for any remaining instances of the old
run_trackerfunction:Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Check for remaining instances of the old run_tracker function. # Test: Search for remaining instances of run_tracker function. Expect: No results. rg --type python 'def run_tracker\(' # Test: Search for calls to run_tracker function. Expect: No results. rg --type python 'run_tracker\(' # Test: Search for new tracker.run_tracker calls. Expect: Multiple results. rg --type python 'tracker\.run_tracker\('Length of output: 210
Script:
#!/bin/bash # Description: Check for remaining instances of the old run_tracker function. # Test: Search for remaining instances of run_tracker function. Expect: No results. rg 'def run_tracker\(' --glob '*.py' # Test: Search for calls to run_tracker function. Expect: No results. rg 'run_tracker\(' --glob '*.py' # Test: Search for new tracker.run_tracker calls. Expect: Multiple results. rg 'tracker\.run_tracker\(' --glob '*.py'Length of output: 980
tests/nn/test_tracker_components.py (4)
18-32: Approve changes torun_tracker_by_namefunction with a minor suggestion.The renaming and addition of the
img_scaleparameter improve the function's flexibility. The new logic for handlingimg_scaleis well-implemented.Consider raising a more specific exception instead of using
pytest.skipwhenimg_scaleis not defined for the tracker. This would provide clearer error information outside of the testing context. For example:if img_scale: if hasattr(t, "candidate_maker") and hasattr(t.candidate_maker, "img_scale"): t.candidate_maker.img_scale = img_scale else: raise ValueError("img_scale is not defined for this tracker")
36-40: Approve changes to test parameters with suggestions for additional test cases.The updates to the
trackeroptions and the addition of theimg_scaleparameter align well with the changes made to therun_tracker_by_namefunction.Consider adding more diverse test cases for
img_scaleto ensure robust testing of the scaling feature. For example:@pytest.mark.parametrize("img_scale", [0, 1, 0.25, 0.5, 2])This would test the behavior with no scaling (0), normal scale (1), downscaling (0.25, 0.5), and upscaling (2).
Line range hint
248-285: Approve changes totest_max_tracks_large_gap_single_trackwith a suggestion.The function renaming and the modification of the
max_tracksparameter improve the test coverage by checking behavior both with and without track limits.Consider adding a comment to explain the significance of setting
max_tracks=-1. For example:# Test with unlimited tracks tracker = Tracker.make_tracker_by_name( tracker="simple", match="hungarian", track_window=2, max_tracks=-1, # Allows unlimited tracks ) # ... (existing code) ... # Test with limited tracks tracker = Tracker.make_tracker_by_name( tracker="simple", match="hungarian", track_window=2, max_tracks=2, # Limits the number of tracks to 2 )This would enhance the readability and clarify the purpose of each test case.
Also applies to: 297-300
Line range hint
312-345: Approve changes totest_max_tracks_small_gap_on_both_tracksandtest_max_tracks_extra_detectionswith a suggestion.The function renamings and modifications to the
max_tracksparameter are consistent with the changes made totest_max_tracks_large_gap_single_track. These changes improve test coverage by checking behavior both with and without track limits.For consistency and clarity, consider adding comments similar to those suggested for
test_max_tracks_large_gap_single_trackto explain the significance ofmax_tracks=-1andmax_tracks=2in both of these functions as well.Also applies to: 357-360, 372-410, 422-425
docs/guides/cli.md (2)
127-127: LGTM! Consider clarifying the description of--max_tracks.The changes align with the PR objectives to simplify the
max_trackingfeature. The removal of--max_trackingand the addition of--max_tracksstreamline the command usage.Consider updating the description of
--max_tracksto provide more context:- Maximum number of tracks to be tracked by the tracker. No limit if None or -1. (default: None) + Maximum number of tracks to be maintained by the tracker. Unlimited if set to None or -1. (default: None)This change clarifies that the parameter sets a limit on the number of maintained tracks, which may help users better understand its purpose.
Also applies to: 188-188
262-262: LGTM! Consider adding a brief explanation to the examples.The examples have been correctly updated to use
--tracking.max_tracksinstead of--max_tracking, maintaining consistency with the changes in the command-line arguments.Consider adding a brief explanation to examples 5 and 6 to clarify the purpose of the
--tracking.max_tracksargument:- sleap-track -m "models/my_model" --tracking.tracker simple --tracking.max_tracks 4 -o "output_predictions.slp" "input_video.mp4" + sleap-track -m "models/my_model" --tracking.tracker simple --tracking.max_tracks 4 -o "output_predictions.slp" "input_video.mp4" + # This example limits the tracker to maintain a maximum of 4 tracks - sleap-track --tracking.tracker simple --tracking.max_tracks 4 -o "retracked.slp" "input_predictions.slp" + sleap-track --tracking.tracker simple --tracking.max_tracks 4 -o "retracked.slp" "input_predictions.slp" + # This example re-tracks existing predictions, limiting the tracker to 4 tracksThese brief explanations will help users understand the purpose and effect of the
--tracking.max_tracksargument in different scenarios.Also applies to: 268-268
sleap/nn/tracker/components.py (2)
355-356: Updated function signature with new parameters.The
cull_frame_instancesfunction now includes two new parameters:merge_instancesandmerging_penalty. This change aligns with the PR objectives to introduce an option for merging instances.However, consider adding type hints for these new parameters to maintain consistency with the existing type hints:
def cull_frame_instances( instances_list: List[InstanceType], instance_count: int, iou_threshold: Optional[float] = None, - merge_instances: bool = False, - merging_penalty: float = 0.2, + merge_instances: bool = False, + merging_penalty: float = 0.2, ) -> List["LabeledFrame"]:Also, update the function's docstring to include descriptions for these new parameters.
Also applies to: 367-369
377-384: New merging functionality added.The new code block implements the merging of instances when
merge_instancesis True. This aligns with the PR objectives and enhances the functionality of thecull_frame_instancesfunction.Consider adding error handling for potential exceptions that might be raised by
create_merged_instances. Also, you might want to add a debug log to show how many instances were merged:if merge_instances: logger.info("Merging instances with penalty: %f", merging_penalty) + original_count = len(instances_list) merged_instances = create_merged_instances( instances_list, penalty=merging_penalty ) instances_list.extend(merged_instances) + logger.debug("Merged instances: original=%d, after_merge=%d", original_count, len(instances_list))sleap/gui/learning/runners.py (1)
256-261: New tracking options added to boolean-to-integer conversion listThe addition of new tracking options to the
bool_items_as_intslist is approved. These changes align with the PR objectives of enhancing tracking capabilities.Consider grouping related items together for improved readability. For example:
bool_items_as_ints = ( "tracking.pre_cull_to_target", "tracking.pre_cull_merge_instances", "tracking.post_connect_single_breaks", "tracking.save_shifted_instances", "tracking.oks_score_weighting", "tracking.prefer_reassigning_track", "tracking.allow_reassigning_track", )sleap/nn/tracking.py (1)
Line range hint
4-130: LGTM: New imports and CandidateMaker abstract base class.The new imports and the introduction of the
CandidateMakerabstract base class improve the overall structure and extensibility of the code. This change allows for different implementations of candidate makers to inherit from a common interface.Consider removing the unused import of
defaultdictfromcollectionsas flagged by the static analysis tool:-from collections import deque, defaultdict +from collections import dequetests/nn/test_inference.py (1)
Line range hint
1783-1825: LGTM: Good extension of test coverage.The changes to test_max_tracks_matching_queue improve the test by covering both 'flow' and 'simple' trackers. The conditional checks and generalized assertions make the test more robust and comprehensive.
Consider renaming the 'trackername' variable to 'tracker_name' for better readability and consistency with Python naming conventions.
sleap/instance.py (1)
1182-1183: Approve implementation with a minor suggestion for readability.The
all_disjointfunction is implemented correctly and efficiently. It uses a list comprehension withall()andset().isdisjoint()to check if all pairs of sequences are disjoint.Consider adding a type hint for the return value and using tuple unpacking for slightly improved readability:
-def all_disjoint(x: Sequence[Sequence]) -> bool: - return all((set(p0).isdisjoint(set(p1))) for p0, p1 in combinations(x, 2)) +def all_disjoint(x: Sequence[Sequence]) -> bool: + return all(set(p0).isdisjoint(set(p1)) for p0, p1 in combinations(x, 2))
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (11)
- docs/guides/cli.md (4 hunks)
- sleap/config/pipeline_form.yaml (3 hunks)
- sleap/gui/learning/runners.py (6 hunks)
- sleap/instance.py (2 hunks)
- sleap/nn/inference.py (13 hunks)
- sleap/nn/tracker/components.py (8 hunks)
- sleap/nn/tracking.py (35 hunks)
- sleap/util.py (3 hunks)
- tests/nn/test_inference.py (8 hunks)
- tests/nn/test_tracker_components.py (11 hunks)
- tests/nn/test_tracking_integration.py (6 hunks)
Additional context used
Ruff
sleap/nn/tracking.py
7-7:
collections.defaultdictimported but unusedRemove unused import:
collections.defaultdict(F401)
532-532: Use
enumerate()for index variablen_processedinforloop(SIM113)
Additional comments not posted (43)
tests/nn/test_tracking_integration.py (2)
51-51: Approve consistent use oftracker.run_tracker.The change to use
tracker.run_trackeris consistent with the modification in themake_ground_truthfunction, reinforcing the object-oriented approach.This change maintains consistency across the codebase and improves the overall design.
22-22: Verify the change in the number of expected tracks.The assertion for the number of tracks has been changed from 27 to 8. This significant reduction in the number of tracks suggests a major change in the tracking algorithm's behavior or output.
Please confirm if this change is intentional and aligns with the expected behavior of the refactored tracking algorithm. Consider running the following script to verify the change:
tests/nn/test_tracker_components.py (1)
47-58: Approve changes totest_tracker_by_namefunction.The addition of the
img_scaleparameter to the function signature and its inclusion in therun_tracker_by_namefunction call correctly implement the new scaling feature in the test.sleap/util.py (2)
17-17: LGTM: New import for rich.progressThe addition of
import rich.progressis appropriate for the newRateColumnclass.
Line range hint
1-44: Overall assessment: Good addition to the utility moduleThe new
RateColumnclass enhances the progress reporting capabilities of the module. The implementation is clean and follows best practices.To ensure that the
RateColumnclass is used correctly throughout the project, please run the following script:This will help verify the integration of the new class and identify any potential areas where it could be utilized.
Verification successful
RateColumn Integration Verified
The
RateColumnclass is successfully integrated into the project, with multiple usages found. There are no remaining instances ofProgressColumnthat require replacement withRateColumn. The implementation is complete and correct.
- Instances of
RateColumnare found in the project, confirming its usage.- No additional
ProgressColumninstances need replacement.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Check for usage of RateColumn class in the project # Search for RateColumn usage echo "Searching for RateColumn usage:" rg --type python "RateColumn" -C 3 # Check if there are any remaining uses of ProgressColumn that could be replaced with RateColumn echo "\nChecking for potential RateColumn replacement opportunities:" rg --type python "ProgressColumn" -C 3Length of output: 343
Script:
#!/bin/bash # Description: Check for usage of RateColumn class in the project # Find all Python files and search for RateColumn usage echo "Searching for RateColumn usage:" fd -e py --exec rg "RateColumn" -C 3 # Find all Python files and check if there are any remaining uses of ProgressColumn that could be replaced with RateColumn echo "\nChecking for potential RateColumn replacement opportunities:" fd -e py --exec rg "ProgressColumn" -C 3Length of output: 1778
sleap/config/pipeline_form.yaml (3)
458-463: New parameter added for saving shifted instancesA new configuration option
tracking.save_shifted_instanceshas been introduced. This parameter allows saving flow-shifted instances between frames, which can improve instance matching at the cost of increased memory usage.The addition of this parameter provides users with more control over the tracking process and memory usage. It's a good trade-off between performance and resource utilization.
Line range hint
1-463: Cleanup of commented-out sectionsSeveral commented-out sections related to tracking parameters have been removed from the file.
This cleanup improves the readability of the configuration file by removing unused or outdated options. It's a good practice to keep configuration files clean and up-to-date.
Line range hint
1-463: Summary of changesThe modifications to this file enhance the tracking capabilities of the system by introducing a new option to save flow-shifted instances. This addition allows for improved instance matching at the cost of slightly increased memory usage. Additionally, the removal of commented-out sections contributes to a cleaner, more maintainable configuration file.
These changes appear to be well-considered and align with the PR objectives of refactoring and improving the max_tracking code. The new option provides users with more control over the tracking process, potentially addressing some of the issues mentioned in the PR summary.
sleap/nn/tracker/components.py (2)
27-27: New import for merging instances functionality.The import of
create_merged_instancesfromsleap.instanceis correctly added to support the new merging functionality incull_frame_instances.
404-407: Minor formatting changes in sorting extra instances.The formatting changes improve code readability by breaking the
sortedfunction call into multiple lines. This is a good practice for maintaining clean and readable code.sleap/gui/learning/runners.py (3)
241-246: Improved compatibility for tracker names with "maxtracks" suffixThis change enhances the flexibility of the system by allowing compatibility with tracker names that include the "maxtracks" suffix. It aligns well with the PR objective of simplifying the
max_trackingfeature implementation.
Line range hint
1-1000: Overall assessment of changes in runners.pyThe modifications in this file align well with the PR objectives of simplifying the
max_trackingfeature and improving tracking functionality. Key improvements include:
- Enhanced compatibility for tracker names with "maxtracks" suffix.
- Addition of new tracking options for more flexible control.
- Simplification of the
max_trackingfeature by removing it from the boolean-to-integer conversion list.These changes should contribute to a more robust and flexible tracking system. The code quality is maintained, and the modifications are implemented cleanly.
To ensure the changes don't introduce any regressions, consider running the existing test suite, particularly focusing on tests related to tracking functionality.
256-261: Removal of "tracking.max_tracking" from boolean-to-integer conversion listThe removal of "tracking.max_tracking" from the
bool_items_as_intslist is approved. This change is consistent with the PR's objective of simplifying themax_trackingfeature implementation.To ensure this change doesn't introduce any unintended side effects, please run the following script to check for any remaining references to "tracking.max_tracking" in the codebase:
If any results are found, please review them to ensure that the removal of "tracking.max_tracking" from
bool_items_as_intsdoesn't cause any issues in other parts of the codebase.Verification successful
No remaining references to "tracking.max_tracking" found
The removal of "tracking.max_tracking" from the
bool_items_as_intslist is confirmed to be safe. There are no remaining references to "tracking.max_tracking" in the codebase, ensuring that this change does not impact other parts of the code.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Check for remaining references to "tracking.max_tracking" # Search for "tracking.max_tracking" in Python files echo "Searching for 'tracking.max_tracking' references:" rg --type python "tracking\.max_tracking" # Check if "tracking.max_tracking" is still used in function calls or assignments echo "Checking for 'tracking.max_tracking' in function calls or assignments:" ast-grep --lang python --pattern '$_($$$, tracking.max_tracking: $_, $$$)' ast-grep --lang python --pattern '$_ = $$$tracking.max_tracking$$$'Length of output: 493
Script:
#!/bin/bash # Description: Check for remaining references to "tracking.max_tracking" # Corrected search for "tracking.max_tracking" in all files echo "Searching for 'tracking.max_tracking' references:" rg "tracking\.max_tracking" # Check if "tracking.max_tracking" is still used in function calls or assignments echo "Checking for 'tracking.max_tracking' in function calls or assignments:" ast-grep --lang python --pattern '$_($$$, tracking.max_tracking: $_, $$$)' ast-grep --lang python --pattern '$_ = $$$tracking.max_tracking$$$'Length of output: 448
sleap/nn/tracking.py (11)
Line range hint
133-384: LGTM: FlowCandidateMaker updates.The
FlowCandidateMakerclass now properly inherits from theCandidateMakerabstract base class, improving the overall structure of the code. The addition of thesave_shifted_instancesattribute provides more control over memory usage.The updated
get_candidatesmethod now handles the logic for retrieving shifted instances from earlier times and pruning old shifted instances, which should improve performance and memory management.
Line range hint
390-413: LGTM: SimpleCandidateMaker updates.The
SimpleCandidateMakerclass now correctly inherits from theCandidateMakerabstract base class, maintaining consistency with the new structure. The updatedget_candidatesmethod is more streamlined and focused on its core functionality.Tools
Ruff
408-408: Local variable
ref_tis assigned to but never usedRemove assignment to unused variable
ref_t(F841)
Line range hint
621-650: LGTM: Enhanced Tracker class attributes.The addition of new attributes to the
Trackerclass improves its configurability and reporting capabilities:
verbosityandreport_rateallow for more flexible progress reporting.found_trackshelps in tracking the last time an instance was found for each track.max_tracksprovides the ability to limit the number of tracks, which can be useful for performance or application-specific requirements.prefer_reassigning_trackandallow_reassigning_trackoffer more control over track assignment strategies.These changes should make the Tracker more versatile and easier to use in various scenarios.
Line range hint
692-892: LGTM: Improved Tracker class methods.The changes to the Tracker class methods significantly enhance its functionality and code organization:
- The new
infer_next_timestepmethod centralizes the logic for determining the current timestep, reducing code duplication.- The refactored
trackmethod is more organized and easier to follow, with clear separation of concerns.- New methods
spawn_new_trackandassign_to_trackprovide more granular control over track management.- The updated
spawn_for_untracked_instancesmethod incorporates the new track assignment strategies, making it more flexible.These improvements should make the Tracker class more maintainable and easier to extend in the future.
Line range hint
958-1028: LGTM: Enhanced Tracker.make_tracker_by_name method.The updates to the
Tracker.make_tracker_by_namemethod improve its configurability and align with the new functionality added to the Tracker class:
- New parameters
max_tracks,prefer_reassigning_track, andallow_reassigning_trackprovide more control over the tracking process.- The addition of
pre_cull_merge_instancesandpre_cull_merging_penaltyoptions allows for more sophisticated instance culling strategies.- Updated progress reporting options (
verbosityandreport_rate) improve the usability of the tracker in different environments.These changes make the tracker more versatile and easier to customize for specific use cases.
Line range hint
1059-1163: LGTM: Updated Tracker.get_by_name_factory_options method.The changes to the
Tracker.get_by_name_factory_optionsmethod improve the documentation and configurability of the tracker:
- New options for
max_tracks,pre_cull_merge_instances,pre_cull_merging_penalty,allow_reassigning_track, andprefer_reassigning_trackhave been added, reflecting the new functionality in the Tracker class.- The help text for existing options has been updated to provide more detailed information.
These updates make it easier for users to understand and configure the tracker according to their specific needs.
1265-1279: LGTM: Improved CLI argument handling in Tracker.add_cli_parser_args.The changes to the
Tracker.add_cli_parser_argsmethod enhance the CLI argument handling:
- The 'tracker' argument is now handled separately from other arguments.
- No default value is set for the 'tracker' argument, which allows for better detection of malformed command lines.
This improvement should make it easier to identify and report issues with user-provided CLI arguments, leading to a better user experience.
1421-1426: LGTM: Consistent progress reporting in KalmanTracker.The addition of
verbosityandreport_rateattributes to theKalmanTrackerclass aligns it with the progress reporting capabilities of theTrackerclass. This consistency improves the overall user experience and makes the codebase more maintainable.
1643-1643: LGTM: Simplified retrack function with improved progress reporting.The
retrackfunction now uses therun_trackermethod of the tracker object for progress reporting. This change:
- Simplifies the
retrackfunction by removing redundant progress reporting code.- Ensures consistency in progress reporting across different parts of the codebase.
- Leverages the new progress reporting capabilities built into the Tracker class.
This improvement demonstrates good code reuse and maintains consistency throughout the tracking system.
532-532: Static analysis suggestion not applicable in this context.The static analysis tool suggests using
enumerate()for the index variablen_processedin the for loop. However, in this specific case, usingenumerate()may not be beneficial:
- The
n_processedvariable is used for more than just counting iterations. It's part of the progress tracking and reporting logic.- The loop doesn't directly use the items from the
framesiterable, makingenumerate()less relevant.- The current implementation allows for more flexibility in how
n_processedis incremented and used within the loop.Given these factors, the current implementation is appropriate for its purpose, and changing it to use
enumerate()might make the code less clear or flexible.Tools
Ruff
532-532: Use
enumerate()for index variablen_processedinforloop(SIM113)
Line range hint
1-1664: Overall improvements to the tracking system.The changes made to
sleap/nn/tracking.pysignificantly enhance the tracking system:
- Introduction of the
CandidateMakerabstract base class improves code organization and extensibility.- Updates to
FlowCandidateMakerandSimpleCandidateMakerclasses align with the new abstract base class structure.- Enhanced
Trackerclass with new attributes and methods provides more flexibility and control over the tracking process.- Improved progress reporting capabilities across different parts of the system.
- Better CLI argument handling for improved user experience.
- Consistent updates to related classes like
KalmanTrackermaintain overall system coherence.These changes should make the tracking system more robust, flexible, and easier to maintain and extend in the future.
Tools
Ruff
532-532: Use
enumerate()for index variablen_processedinforloop(SIM113)
tests/nn/test_inference.py (3)
1273-1273: LGTM: Good use of @pytest.mark.slow() decorator.The addition of the @pytest.mark.slow() decorator is a good practice for identifying tests that may take longer to execute. This allows for better management of test suites and the ability to selectively run faster tests when needed.
1316-1316: LGTM: Consistent use of @pytest.mark.slow() decorator.The addition of the @pytest.mark.slow() decorator to test_topdown_id_predictor_save is consistent with the previous change and follows good testing practices for managing test execution times.
1848-1848: LGTM: Consistent use of @pytest.mark.slow() decorator.The addition of the @pytest.mark.slow() decorator to test_movenet_predictor is consistent with the previous changes and continues to follow good testing practices for managing test execution times.
sleap/instance.py (6)
1186-1189: Function signature and docstring look good.The function signature and docstring are well-defined. The use of type hints is appreciated.
1202-1212: Ensure same skeleton and track for all instances.Good practice to check for consistency in skeleton and track across all instances before proceeding with the merge.
1214-1223: Efficient generation of instance subsets.The use of
combinationsandchain.from_iterablefor generating instance subsets is efficient. Theall_disjointfunction is used correctly to ensure non-intersecting visible nodes.
1228-1235: Score calculation and penalty application.The score calculation and penalty application look correct. Good use of the
reducefunction for calculating the instance score.
1237-1245: Creation and appending of merged instances.The creation of
PredictedInstanceobjects and appending them to themerged_instanceslist is implemented correctly.
1182-1246: Well-integrated new functionality.The new functions
all_disjointandcreate_merged_instancesare well-integrated into the existing codebase. They provide new functionality without disrupting the existing code structure.To ensure that these new functions don't introduce any unintended side effects, it would be beneficial to add unit tests. Here's a script to check for existing tests:
sleap/nn/inference.py (10)
48-52: Conditional import of cached_property based on Python versionThis change adds compatibility for Python versions prior to 3.8 by conditionally importing
cached_property. If the Python version is 3.8 or higher, it's imported fromfunctools; otherwise, it falls back to using the standardpropertydecorator.This is a good practice for maintaining backwards compatibility while still leveraging newer features when available.
163-166: Replace property decorator with cached_property for report_periodThe
report_periodmethod in thePredictorclass now uses thecached_propertydecorator instead of the standardpropertydecorator.This change can improve performance by caching the result of
report_periodafter the first access, which is beneficial if this method is called frequently and the computation is expensive.
414-416: Pre-compile loop examples before starting time measurementThe
examplesvariable is now created before the time measurement and progress reporting logic. This change helps to provide a more accurate ETA by excluding the time taken to compile the dataset from the progress calculation.This modification improves the accuracy of the progress reporting by separating the dataset compilation time from the actual processing time.
432-433: Simplify loop iteration over examplesThe loop now directly iterates over
examplesinstead of usingself.pipeline.make_dataset().This change is consistent with the earlier modification where
exampleswas pre-compiled. It simplifies the loop and avoids repeated calls tomake_dataset().
452-453: Simplify loop iteration over examples in JSON verbosity modeSimilar to the previous change, the loop in the JSON verbosity mode now directly iterates over
examples.This change is consistent with the earlier modifications and maintains consistency across different verbosity modes.
489-490: Simplify loop iteration over examples in non-verbose modeThe loop in the non-verbose mode also now directly iterates over
examples.This change maintains consistency with the earlier modifications across all verbosity modes.
5335-5337: Update LabelsReader initialization in _make_provider_from_cli functionThe
LabelsReaderis now initialized with anexample_indicesparameter, which is set to the result offrame_list(args.frames).This change allows for more flexible frame selection when creating the
LabelsReader. It uses theframe_listfunction to parse theframesargument, which can improve usability by allowing users to specify frames in various formats (e.g., comma-separated list, range).
5431-5434: Update Tracker initialization in _make_tracker_from_cli functionThe
Tracker.make_tracker_by_namemethod is now called with an additionalprogress_reportingparameter set toargs.verbosity.This change allows the tracker to use the same verbosity setting as the rest of the application, which can lead to more consistent progress reporting across different components.
5593-5596: Optimize frame filtering in main functionWhen
provider.example_indicesis not None, the code now converts the indices to a set for O(1) search time, and uses a list comprehension to filter the frames.This optimization can significantly improve performance when dealing with large numbers of frames, as it reduces the time complexity of the filtering operation from O(n^2) to O(n).
5599-5599: Update tracker execution in main functionThe
tracker.run_trackermethod is now called with aframesparameter instead of separatevideoandframesparameters.This change simplifies the tracker execution by passing all necessary information in the
framesparameter. It's likely that therun_trackermethod has been updated to work with this new signature.
change default save_shifted_instances
c656bb8 to
bb67467
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (13)
- docs/guides/cli.md (4 hunks)
- sleap/config/pipeline_form.yaml (3 hunks)
- sleap/gui/learning/dialog.py (1 hunks)
- sleap/gui/learning/runners.py (6 hunks)
- sleap/gui/widgets/video.py (2 hunks)
- sleap/instance.py (2 hunks)
- sleap/nn/inference.py (12 hunks)
- sleap/nn/tracker/components.py (8 hunks)
- sleap/nn/tracking.py (33 hunks)
- sleap/util.py (2 hunks)
- tests/nn/test_inference.py (8 hunks)
- tests/nn/test_tracker_components.py (11 hunks)
- tests/nn/test_tracking_integration.py (6 hunks)
Files skipped from review as they are similar to previous changes (7)
- sleap/config/pipeline_form.yaml
- sleap/gui/learning/runners.py
- sleap/nn/inference.py
- sleap/nn/tracker/components.py
- sleap/util.py
- tests/nn/test_inference.py
- tests/nn/test_tracker_components.py
Additional context used
Ruff
sleap/nn/tracking.py
486-486: Use
enumerate()for index variablen_processedinforloop(SIM113)
sleap/instance.py
28-28:
typing.ForwardRefimported but unusedRemove unused import:
typing.ForwardRef(F401)
Additional comments not posted (18)
tests/nn/test_tracking_integration.py (4)
22-22: Clarify the reason for the change in the expected number of tracks.The assertion has been updated to check if the length of
labels.tracksis equal to 8 instead of 27. What is the reason for this change? Is it due to modifications in the tracking algorithm or configuration parameters?
25-28: Approve the function name change for improved clarity.The function name change from
test_simplemax_trackertotest_simple_max_tracksimproves clarity by explicitly mentioningmax_tracks.Clarify the reason for the changes in the CLI command.
The CLI command within the test has been modified:
- The tracker has been changed from
simplemaxtrackstosimple.- The
max_tracksparameter has been added and set to 2.What is the reason for these changes? Do they reflect modifications in the tracking algorithm being tested?
45-45: LGTM: Approve callingrun_trackerdirectly on thetrackerobject.The change to call
run_trackerdirectly on thetrackerobject improves code organization and maintainability by encapsulating the tracking logic within thetrackerinstance.
110-116: LGTM: Approve simplifying themake_trackerfunction.The changes to the
make_trackerfunction, including:
- Removing the
max_trackingparameter from the function signature.- Directly passing the
max_tracksparameter to the tracker constructor.These changes improve code readability and maintainability by simplifying the function and removing conditional logic based on
max_tracking.docs/guides/cli.md (2)
127-127: LGTM!The removal of the
--max_trackingargument simplifies the tracking options and the documentation has been updated accordingly.
191-191: LGTM!The documentation now emphasizes the
--max_tracksargument and the examples have been updated accordingly. The changes are consistent with the removal of the--max_trackingargument.Also applies to: 265-271
sleap/gui/learning/dialog.py (1)
736-736: LGTM!The code change ensures that
no_result_countis always non-negative by usingmax(). This improves the robustness of the code and aligns with the AI-generated summary.sleap/nn/tracking.py (7)
118-131: LGTM!The changes to make
CandidateMakeran abstract base class are good architectural improvements. It enforces a common interface for all candidate makers.
Line range hint
133-388: LGTM!The changes to
FlowCandidateMakerlook good. Inheriting fromCandidateMakerensures it implements the required interface, and the updatedget_candidatesmethod is more streamlined.
Line range hint
390-414: LGTM!The changes to
SimpleCandidateMakerlook good. Inheriting fromCandidateMakerensures it implements the required interface.Tools
Ruff
408-408: Local variable
ref_tis assigned to but never usedRemove assignment to unused variable
ref_t(F841)
438-580: LGTM!The changes to
BaseTrackerlook good. The newverbosityandreport_rateattributes allow configuring the progress reporting, and the updatedrun_trackermethod handles the newverbosityoption correctly.Tools
Ruff
486-486: Use
enumerate()for index variablen_processedinforloop(SIM113)
Line range hint
629-1293: LGTM!The changes to
Trackerlook good overall:
- The new attributes allow for more flexible configuration of the tracking behavior.
- The
infer_next_timestepmethod reduces code duplication by centralizing the timestep logic.- The refactored
trackmethod is cleaner and easier to follow.- The new
spawn_new_trackandassign_to_trackmethods provide clearer logic for managing tracks.- The updated
spawn_for_untracked_instancesmethod incorporates the new track assignment strategies based on user preferences.
1435-1440: LGTM!The addition of the
verbosityandreport_rateattributes toKalmanTrackeris consistent with the changes made toBaseTrackerandTracker. It allows configuring the progress reporting for the Kalman filter-based tracking.
486-486: Ignore the Ruff suggestion.The variable
n_processedis not used as an index within the loop. It is a counter that is incremented on each iteration. Usingenumerate()would not provide any benefit here and would make the code less readable.Tools
Ruff
486-486: Use
enumerate()for index variablen_processedinforloop(SIM113)
sleap/instance.py (2)
1182-1183: LGTM!The code changes are approved.
1186-1248: LGTM!The code changes are approved.
sleap/gui/widgets/video.py (2)
819-820: LGTM!The code changes are approved. Initializing
_down_postoNoneand checking it before use improves the robustness of the mouse event handling.
1044-1044: LGTM!The code changes are approved. Checking
_down_posis notNonebefore comparing it withevent.pos()prevents potential errors when_down_posis uninitialized.
bb67467 to
06ef136
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
Outside diff range, codebase verification and nitpick comments (2)
tests/nn/test_tracker_components.py (2)
18-33: Add a docstring to document the function.The function can be improved by adding a docstring to document the parameters and return value.
def run_tracker_by_name(frames=None, img_scale: float = 0, **kwargs): """ Run a tracker by name with the specified parameters. Args: frames: The frames to track. img_scale: The image scale to use for tracking. **kwargs: Additional keyword arguments to pass to the tracker. Returns: The tracked frames. """ ...
Line range hint
61-81: Add theimg_scaleparameter to the test function.For consistency with the other test functions, add the
img_scaleparameter to therun_tracker_by_namefunction call.def test_oks_tracker_by_name( centered_pair_predictions_sorted, tracker, oks_score_weighting, oks_normalization, ): # This is slow, so limit to 5 time points frames = centered_pair_predictions_sorted[:5] run_tracker_by_name( frames=frames, tracker=tracker, similarity="object_keypoint", matching="greedy", oks_score_weighting=oks_score_weighting, oks_normalization=oks_normalization, max_tracks=2, + img_scale=0, )
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (10)
- docs/guides/cli.md (4 hunks)
- sleap/config/pipeline_form.yaml (3 hunks)
- sleap/gui/learning/runners.py (6 hunks)
- sleap/instance.py (2 hunks)
- sleap/nn/inference.py (12 hunks)
- sleap/nn/tracker/components.py (8 hunks)
- sleap/nn/tracking.py (33 hunks)
- tests/nn/test_inference.py (8 hunks)
- tests/nn/test_tracker_components.py (11 hunks)
- tests/nn/test_tracking_integration.py (6 hunks)
Files skipped from review as they are similar to previous changes (5)
- sleap/config/pipeline_form.yaml
- sleap/gui/learning/runners.py
- sleap/nn/inference.py
- sleap/nn/tracker/components.py
- tests/nn/test_inference.py
Additional context used
Ruff
sleap/nn/tracking.py
486-486: Use
enumerate()for index variablen_processedinforloop(SIM113)
Additional comments not posted (22)
tests/nn/test_tracking_integration.py (4)
22-22: Please provide more context for the change in expected tracks.The assertion has been updated to expect 8 tracks instead of 27. What is the reason behind this change? Have there been updates to the tracking algorithm or test data that would explain the reduced number of expected tracks?
25-28: LGTM!The renaming of the function and the update to the CLI command improve clarity and consistency. The test still verifies the expected behavior of limiting the maximum number of tracks.
45-45: LGTM!The integration of the
run_trackerfunctionality directly into thetrackerobject simplifies the code structure and improves readability.
109-114: Please provide more context for the removal ofmax_trackingchecks.The conditional checks for
max_trackinghave been removed from themake_trackerfunction. Can you explain the reason behind this change? Are there any potential side effects or unintended consequences of always passing themax_tracksparameter to the tracker constructor?tests/nn/test_tracker_components.py (2)
35-58: LGTM!The code changes are approved.
Line range hint
244-306: LGTM!The code changes are approved.
Also applies to: 308-366, 368-430
docs/guides/cli.md (4)
127-127: LGTM!The removal of
--max_trackingand the updated description of--max_trackssimplify the tracking options available to users.
191-191: LGTM!The updated description of
--tracking.max_tracksclarifies that there is no limit if set toNoneor-1.
265-265: LGTM!The removal of
--max_trackingfrom the example usage is consistent with the changes made to thesleap-trackusage.
271-271: LGTM!The removal of
--max_trackingfrom the example usage is consistent with the changes made to thesleap-trackusage.sleap/nn/tracking.py (10)
118-130: LGTM!The changes to the
CandidateMakerclass are approved. Making it an abstract base class with theget_candidatesabstract method improves extensibility.
Line range hint
133-384: LGTM!The changes to the
FlowCandidateMakerclass are approved. Inheriting fromCandidateMaker, updating theget_candidatesmethod, and streamlining the candidate generation logic improves the code.
Line range hint
390-413: LGTM!The changes to the
SimpleCandidateMakerclass are approved. Inheriting fromCandidateMakerand updating theget_candidatesmethod is consistent with the new design.Tools
Ruff
408-408: Local variable
ref_tis assigned to but never usedRemove assignment to unused variable
ref_t(F841)
438-579: LGTM!The changes to the
BaseTrackerclass are approved. Adding theverbosityandreport_rateattributes and updating therun_trackermethod with verbosity handling logic improves configurability and progress reporting.Tools
Ruff
486-486: Use
enumerate()for index variablen_processedinforloop(SIM113)
Line range hint
629-908: LGTM!The changes to the
Trackerclass are approved. The new attributes improve configurability, theinfer_next_timestepmethod centralizes timestep inference logic, and the newspawn_new_track,assign_to_track, andupdate_matched_instance_tracksmethods provide clearer track management. The updatedspawn_for_untracked_instancesmethod incorporates the new track assignment strategies effectively.
Line range hint
962-1057: LGTM!The changes to the
Tracker.make_tracker_by_namemethod are approved. Adding the new parametersmax_tracks,prefer_reassigning_track,allow_reassigning_track,pre_cull_merge_instances, andpre_cull_merging_penaltyimproves tracking options and pre-culling functionality. The parsing ofmax_tracksand the incorporation of the new parameters inpre_cull_functionare implemented correctly.
Line range hint
1073-1224: LGTM!The changes to the
Tracker.get_by_name_factory_optionsmethod are approved. Adding the new options formax_tracks,allow_reassigning_track,prefer_reassigning_track,pre_cull_merge_instances, andpre_cull_merging_penaltyimproves tracking configurability. The updated help text for thesave_shifted_instancesoption provides clearer information.
1279-1293: LGTM!The change to the
Tracker.add_cli_parser_argsmethod for thetrackerargument is approved. Avoiding the definition of a default value helps detect malformed command-line arguments, which is a good practice.
1435-1439: LGTM!The changes to the
KalmanTrackerclass are approved. Adding theverbosityandreport_rateattributes is consistent with the changes made to theBaseTrackerclass and improves configurability and progress reporting.
486-486: Skipping Ruff suggestion.The suggestion to use
enumerate()for then_processedvariable is a false positive.n_processedis used as a counter and not as an index, so usingenumerate()would not provide any benefit and would reduce code readability.Tools
Ruff
486-486: Use
enumerate()for index variablen_processedinforloop(SIM113)
sleap/instance.py (2)
1182-1183: LGTM!The code changes are approved.
1186-1248: LGTM!The code changes are approved. The function follows the specified merging criteria and correctly applies the penalty factor to the scores of the merged instances.
Codecov Report❌ Patch coverage is
❌ Your patch status has failed because the patch coverage (65.87%) is below the target coverage (100.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## develop #1896 +/- ##
===========================================
+ Coverage 73.30% 75.21% +1.90%
===========================================
Files 134 133 -1
Lines 24087 24558 +471
===========================================
+ Hits 17658 18471 +813
+ Misses 6429 6087 -342 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Hi @getzze, Thanks for this! We notice that you tried to simplify our tracking method by removing The PR refactors the code to implement a Let us know if you have any questions! Thanks! Divya |
|
Hi @gitttt-1234, Thanks for reviewing the PR! I chose to remove the Instead I added an (optional) step to assign an instance to a track that has disappeared for more than Something I didn't do in this PR and that should probably be done, is to put back the |
|
Hi @getzze, Thanks for the contributions! Apologies it's taken us so long to work through it all, but we did have some questions and concerns about the changes. @gitttt-1234 described those inline, but I think maybe there's also some misunderstandings that I want to clarify to make sure we're on the same page -- see below.
As @gitttt-1234 mentioned, the point of the local queues is indeed to limit the maximum number of tracks that can be possibly spawned in an entire project, but it does so using a specific algorithmic approach. By specifying the maximum number of tracks, what we can do is create buffers (up to that limit) that hold the past The main thing this solves is that it prevents the explosion of new tracklets being created in scenarios where there is a lot of track fragmentation due to frequent occlusions (or lots of animals in the FOV).
Again, to be clear, with local track queues, the instances can be used no matter how many instances are hidden and no matter the amount of time.
Using instances from a long time in the past is not great since the detections will be pretty stale, but it can work well when animals are sleeping or hiding under a hut or something else out of view. It does not, however, mean that the instance needs to reappear in the exact same place. It just means that the last detection(s) from that track will still be in consideration during the candidate generation and matching steps. If we lose a track for a long amount of time, the truly appropriate thing to do would be to explicitly use ReID features, but we don't have that implemented in here yet, so in the meantime, this is a "best effort" approach that doesn't require any new information to be integrated in the algorithm.
Could you elaborate on what you mean by that? How does a track "become empty"? Again, by design, if an instance is lost for more than
The track local queues should handle this explicitly. We don't make a distinction between tracks that are "stale" vs "active", but treat all instances from all queues as valid candidates. I could see an advantage to separating these two (e.g., for preferring active tracks that have recent detections before considering the stale ones), but this is fundamentally a different tracking algorithm and not a replacement for what we have now.
We removed pre-culling as this was superseded by a feature set for limiting the number of instances right at the model inference level. This is exposed via the We implemented these in:
The advantage is that it allows the users to cull to a specific number of instances regardless of whether tracking will be applied next -- so it's usable during the human-in-the-loop annotation stage as well to filter out some false positives when models aren't trained well enough yet. We also had a question about the instance merging functionality you added. It makes total sense, but you might imagine how this will have lots of edge cases. For example: if the two node sets are exactly disjoint, but correctly belong to two different animals, we'd be incorrectly merging them into what we call a "chimera" (top of one mouse connected to the bottom of another mouse). Another scenario we see often that this wouldn't quite resolve is when we have two almost disjoint sets, except for one node that is maybe off by a couple of pixels. Adding a bit of fuzziness to the keypoint matching to test for "disjoint-ness" before triggering this function could help cover those cases as well. The thing I'm a bit more confused by is that you're mentioning this as useful when there are two instances on the same frame that also belong to the same track. How is this even possible given any of the current tracking approaches? We always do matching that ensures that no more than one instance is assigned to a given track within a frame. Establishing that two (partial) instances indeed belong to the same track is the hard problem that's at the crux of this instance merging problem -- and why we haven't implemented this before. Some clarity on what you had in mind for dealing with this would help us understand the solution. In general, we're super appreciative of the contributions, but we want to make sure that:
If you prefer to have a chat in realtime, just drop me an email --> Thanks again for all the work! Looking forward to getting this merged in :) Talmo |
|
Now I understand the use of Local queues. Maybe the best approach is to make 2 PR, one for bringing the I will try to send a screenshot to explain what I mean by a track becoming empty. But basically, when a track is not detected for I didn't realize that you could limit the number of instances from inference, thanks. Although I like to try different tracking method with new videos, so I run inference and tracking separately and I would still keep all the instances in the inference stage and pre-cull when tracking. Maybe other people would do that also, I don't know. |
|
Regarding the merging function:
Yes but this chimera will be less likely to be the best match for the track (but it's not sure).
I also see that. I only implemented merging totally disjoint nodes, but we could think of merging group of nodes with one overlapping node (taking the average position of the duplicate node). It would not be too hard to add.
The problem I tried to solve with this merging function is the following:
So the idea of merging instances was to avoid the step 3, stopping the identity swap. |
ONLY AFTER #1894 is merged.
I refactored the
max_trackingfeature introduced in #1447 and simplified the implementation. It may give different results as the goal is similar but implemented differently.One problem I had with the current implementation is that instances cannot be assigned to created but unused tracks. Say we have
max_tracks=2, and the 2 tracks are created. If a track is not assigned instances fortrack_windowtimesteps, it does not contribute to instance candidates anymore, hence instances cannot be assigned to this track anymore, forever. What was happening is that instances stopped being tracked in the middle of the video. Therefore it was unusable for me and the "pre_cull" options are not available from the GUI anymore, so I need to use the CLI.I added an option to reassign unused tracks to unmatched instances if the maximum number of tracks was reached.
I also added an option to merge instances in the "pre_cull" stage. That is because I often have two instances from inference: front of mouse and back of mouse, that could perfectly be merged. The idea is that only "good" merges are kept by the tracking step, but it is not perfect.
Summary by CodeRabbit
New Features
Bug Fixes
Documentation
Tests