The current pytest configuration is suppressing all deprecation warnings, which includes warnings from dependency libraries. This makes it much more likely for code to fall out-of-date, as can be seen in the recent failing CI jobs due to changes in the lastest version of numpy which was just releases (1.24).

This PR addresses the issue in two ways:

1. Updates code to take into account changes in the underlying libraries, and
2. Removes the global warnings filter in favor of fine-grained warnings filtering.

The second bullet involves adding explicit checks for warnings raised by deepcell-toolbox itself, to ensure that warnings are being raised correctly in the expected cases.

Adds support for Python 3.10. No changes to the underlying library but some dependency updates were required, the main ones being:

• removing the upper-bound on pytest, and
• Dropping the pytest-pep8 extension, which hasn't been actively maintained in over 8 years

We can of course re-add code linting using different tooling - I'm happy to do so but would prefer to leave it for a follow-up PR.

This PR supports vanvalenlab/deepcell-tf#595 which upgrades tensorflow to 2.8 and allows new versions of numpy.

Addresses the bug identified in #121.

Also bump up version for the next release

• Use segmentation.watershed instead of morphology.watershed.
• Remove correct_drift (Fixes #6) and register_translation is removed in 0.19.x.
• Set array slices with tuples to prevent warnings.
• Update skimage version requirements to avoid bug in peak_local_max in 0.16.x and 0.17.x.

numpy restriction for the toolbox requires Python < 3.10

• Use segmentation.watershed instead of morphology.watershed.
• Remove correct_drift (Fixes #6) and register_translation is removed in 0.19.x.
• Set array slices with tuples to prevent warnings.
• Update skimage version requirements to avoid bug in peak_local_max in 0.16.x and 0.17.x.

Precision and SEG were being multiplied by 100 while Recall was not. This removes the 100x casting to make all metrics be in the same range [0, 1].

Additionally, fixes the norecursedirs value in pytest.ini to be space-delimited instead of comma-delimited.

Finally, pins coveralls<3.3.0 to prevent a breaking change in the latest release.

Fixes #117

No need to have each function in their own files.

Resolves a breaking change in the new release causing import issues.

This PR closes #113. It replaces the previous hole-filling algorithm with an optimized, regionprops-based version. The remaining bottleneck in the post-processing is now peak finding, so the warning message for large images is now raised when h_maxima is selected for large images instead of peak_local_max

What

• Cache the entire Python environment in the testing GitHub Action workflow.

Why

• Drastically speed up build times.

This PR combines deep_watershed and deep_watershed_mibi into a single function, deep_watershed.

The following arguments have been deprecated in favor of their deep_watershed_mibi alternatives (due to readability/understandability):

• min_distance deprecated in favor of radius
• distance_threshold deprecated in favor of interior_threshold
• detection_threshold deprecated in favor of maxima_threshold

Additionally, new arguments have been moved over from deep_watershed_mibi:

• maxima_smooth and interior_smooth to smooth the inputs with a gaussian filter.
• fill_holes_threshold to fill holes smaller than this many pixels
• pixel_expansion to expand the interior array
• maxima_algorithm to switch from h_maxima peak finder to peak_local_max. By default, deep_watershed now uses h_maxima to find the peaks in an image. However, to use the previous algorithm, the user can pass peak_local_max instead. This is found to be less accurate but faster, and may be suitable for unambiguous peaks.

The input to the function is still a list of two numpy arrays, though more can be passed as long as the indices of the maxima and interior arrays are passed with maxima_index and interior_index.

Finally, label_erosion was added as a parameter to enable eroding labels (as performed by deep_watershed by default).

This PR leaves deep_watershed_3D and deep_watershed_mibi as importable functions, but they will just use deep_watershed internally. These will be removed in a future version, along with the deprecated arguments.

Fixes #51

This PR significantly refactors deepcell_toolbox.metrics.Metrics. Now, a Detection object is created for each combination of overlapping objects in each y_pred and y_true image. The Detections allow for easier calculation of all metrics and error types on the fly. These metrics/error types are now tracked as properties on ObjectMetrics rather than each having an individual attribute that is mutated with each function call. Metrics also now has a summarize_object_metrics_df method to help convert the data frame of every metric into summary statistics. This has greatly cleaned up the existing attributes and their instantiation; the pattern of adding attributes when helper functions are called has been removed entirely.
Extra attention was also given to cases where there may be division by 0 or other NaN issues, which should prevent the spammy warnings that we had seen in previous versions.

The performance has also been greatly improved - from 60 batches/s to 300 batches/s (or from 45s to 9s for my 2880 test batches). This was primarily accomplished through:

• Creating a single pd.DataFrame using from_records, rather than iteratively creating and appending data frames for each ObjectMetrics calculated.
• Using count_nonzero instead of sum where possible, which prevents coercing the datatype from boolean to float.

⚠️ Breaking Changes ⚠️

This PR has several breaking changes to deepcell_toolbox.metrics:

• seg has been deprecated, but not removed (yet). The SEG score is always calculated.
• stats and output are no longer attributes
• ObjectAccuracy is now ObjectMetrics
• stats_pixelbased has been replaced with PixelMetrics
• all_pixel_stats has been replaced with calc_pixel_stats to be consistent with calc_object_stats.
• calc_pixel_confusion_matrix has been replaced with PixelMetrics.get_confusion_matrix
• pixel_df_to_dict has been replaced with df_to_dict
• save_error_ids, assign_plot_values, and plot_errors have all been removed in favor of ObjectMetrics.plot_errors.
• to_precision has been removed as it is seemingly redundant to the round built-in.

I hope these breaking changes do not disrupt many downstream processes - I only found the use of Metrics.calc_object_stats, which should be unaffected.

Fixes #26 (Object stats and force_event_links should likely resolve this. If not we can re-open it or make a new issue)
Fixes #68 (Performance has improved by a factor of ~5)

The only difference in these two functions was the dimensionality of the coords found in peak_local_max. Switching to a comprehension allows us to easily use 2D or 3D data with the same function.

deep_watershed_3D is now deprecated (and throws a Deprecation Warning), but is kept for now for backward compatibility.

skimage.morphology.watershed is deprecated and removed in skimage v0.19. This PR uses skimage.segmentation.watershed instead, which is supported by at least 0.14+.

This resolves an annoying deprecation warning we get every time we use the post-processing function.

Adding a template and GitHub workflow for the release-drafter. This should automatically create and update a draft for the next release of the project.

We currently don't check that there are valid predictions when computing precision, which can lead to divide by zero errors.