Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@naglepuff
Copy link
Collaborator

@naglepuff naglepuff commented Oct 6, 2025

Fix #1512

Changes (Desktop only)

Following the precedent of the Multi-Training tab, a new tab has been added to the Dive Desktop navigation bar, called "Pipeline."
image

This tab contains an interface similar to that of the multi-training interface, but with the ability to run a specified pipeline over multiple datasets.

Intended Workflow

Step 1. Choose a pipeline type

This is handled by a simple dropdown menu. This must be done first to enable the pipeline select to become enabled.
image

Limitation: The pipeline types "2-cam" and "3-cam" are filtered out for now to reduce the complexity of selecting datasets.

Step 2. Choose a pipeline

Once a pipeline type is chosen, the "Pipeline" dropdown will allow users to select a specific pipeline of that type. Once this is done, the available datasets appear for selection.

image

Because the pipeline/type pipeline are chosen first, we have additional control over what datasets are presented for staging. For example, measurement pipelines only allow users to select from stereo datasets.

image
Step 3. Stage datasets

Much like how datasets are staged/unstaged for multi-training, users can add from the available datasets table, and remove from the staged datasets table as needed.

image
Step 4. Run pipelines

Clicking the button will create a request for each selected dataset. If all requests come back to the frontend with no issues, the user will be moved to the Jobs tab to monitor the jobs that they just started. If any requests come back with issues, the user will be shown a prompt informing them that some of the pipelines did not start.

@naglepuff naglepuff requested a review from BryonLewis October 6, 2025 22:11
@naglepuff naglepuff marked this pull request as ready for review October 6, 2025 22:11
Copy link
Collaborator

@BryonLewis BryonLewis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great!

I thought we did this but didn't realize until testing that we don't have any sort of Job Queueing system on desktop. I.E. if I configure 20 pipelines to run they all start at the same time. I've created another issue #1532 for a future PR. This would implement a queuing system for pipeline and training jobs (Any job that could use limited resources like the GPU).

@naglepuff naglepuff merged commit 6cde9f0 into main Oct 7, 2025
4 checks passed
@naglepuff naglepuff deleted the issue-1512-desktop-multi-dataset-pipeline branch October 7, 2025 14:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Desktop - Multi Dataset Pipeline Running

2 participants