Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@Eldies
Copy link

@Eldies Eldies commented May 26, 2025

Summary

To access items from a subset, StreamDataset needs to iterate all the items of the dataset.
It adds a significant overhead if there are a lot of subsets.

Making it more efficient for some cases when the extractor provides access to subsets.

  1. if the dataset is not transformed -> no extra iterations
  2. if dataset is transformed with local transformations (ItemTransform) and they do not change subsets of items -> one extra iteration to determine whether the condition applies

How to test

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added the description of my changes into CHANGELOG.​
  • I have updated the documentation accordingly

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2025 CVAT.ai Corporation
#
# SPDX-License-Identifier: MIT

@Eldies Eldies mentioned this pull request May 26, 2025
6 tasks
@sonarqubecloud
Copy link

@zhiltsov-max zhiltsov-max merged commit f32ca9c into develop May 27, 2025
6 checks passed
@zhiltsov-max zhiltsov-max deleted the dl/better-streaming branch May 27, 2025 11:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants