-
Notifications
You must be signed in to change notification settings - Fork 23
AssetStore Import Updates #1507
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…on destination imports
…recursive dangling annotation processing
naglepuff
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! From what I can tell this all works as intended. I found the test script to be very helpful and easy to use.
When I was looking at some of the scripts in VSCode the editor was complaining about a blank line missing.
| minio_ip = get_container_ip(minio_container, "dive_default") | ||
| # Helper function: run mc inside persistent container | ||
| def mc_cmd(*args, capture_output=False): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not a huge deal, but I think there should be a blank line somewhere in here. My editor was complaining
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup my linter was yelling about this as well
resolves #1505
Importing Logic:
Below is the logic as each file is imported (GirderItem) in this case:
Once all Items are imported a secondary process begins to kick off transcoding jobs. Injected into that is a new process to manage all dangling AnnotationFileFutureProcess files to place them in the proper folder.
Finally it kicks off process to import all video files as datasets by running the quick check to see if files need to be transcoded. During this process it calles the /postprocess endpoint that will check and look for CSV/JSON files in the same folder as the DIVEDataset and will import it as annotations if it exists.
Instructions for Testing are found in the ./samples/scripts/assetStoreImport/README.md file
Discussion of Technical Implementation:
DIVE-AssetStoreImport-TechnicalDetails_small.mp4
Testing Process:
DIVE-AssetStoreImport-Demo_small.mp4