Decouple image import and feature extraction #3462
Conversation
| if (exists_mask) { | ||
| if (!image_data.mask.Read(mask_path, false)) { | ||
| LOG(ERROR) << image.Name() << " Failed to read the mask file"; | ||
| image_data.mask.Deallocate(); // Skip the mask but not the image! |
There was a problem hiding this comment.
Previously, no features would be extracted for an image when its mask cannot be found or read. This seems wrong to me.
| enum class FeatureExtractionStatus { | ||
| FAILURE, | ||
| SUCCESS, | ||
| IMAGE_EXISTS, | ||
| BITMAP_ERROR, | ||
| }; | ||
|
|
||
| std::string FeatureExtractionStatusToString(FeatureExtractionStatus status) { | ||
| switch (status) { | ||
| case FeatureExtractionStatus::SUCCESS: | ||
| return "SUCCESS"; | ||
| case FeatureExtractionStatus::FAILURE: | ||
| return "FAILURE: Failed to process the image."; | ||
| case FeatureExtractionStatus::IMAGE_EXISTS: | ||
| return "IMAGE_EXISTS: Features for image were already extracted."; | ||
| case FeatureExtractionStatus::BITMAP_ERROR: | ||
| return "BITMAP_ERROR: Failed to read the image file format."; | ||
| default: | ||
| return "Unknown"; | ||
| } | ||
| } |
There was a problem hiding this comment.
All of this is unnecessary if we decide to not propagate the image data through the queues when it is unsuccessful. The only downside of this is that error messages won't appear in the order of the FeatureWriterThread.
|
Thanks, this is a good initiative and I had the same in mind in the past. The main motivation for keeping them coupled was the one you already mentioned in that I wanted to avoid reading images twice. Another one is that an image in the database has valid features. I don't think we can start assuming images to be ordered by name, in particular, as we want to support the use case, where one can run multiple rounds of image import / feature extraction. |
|
Another note is that this PR will have a lot of conflicts with the changes here: #3068 I should probably extract the non-ALIKED/LightGlue related refactoring into a separate PR until we land the torch changes (which still need some more testing due to cross-platform issues). |
B1ueber2y
left a comment
There was a problem hiding this comment.
Thank you. I like the overall idea. Reading images twice will not be a problem after switching to OpenImageIO as it supports reading EXIF header info without reading pixels. I also think that adding an extra step in the CLI is fine and makes sense to make the feature extractor more flexible in python side.
Many lines in the PR are commented out for now. I assume that it is still in progress?
This PR moves the image import step (creating images, cameras, rigs, pose priors in the database) outside of the feature extraction. Benefits:
One downside of this is that we now need to read image pixels twice (to read the EXIF for import and later for extraction) but this is likely negligible, and we can later make it possible to read EXIF without the pixels.
Some points should be discussed:
feature_extractorcallimage_importerif the database is empty? We will still break the argument names of themask_pathandcamera_mask_pathoptions, but these are not widely used.mask_pathandcamera_mask_pathpart of theSiftExtractionOptions?TODO:
ImageReaderandImageImporter-- in a follow-up PR.FeatureExtractorController::ReadImageData-- in a follow-up PR.