❓Question

I'm trying to use Aim to track a lot of audio files (basically, whatever has been misclassified and how). Apparently, these files are copied to RocksDB as BLOBs and then the corresponding .sst files are kept open. I assume this is multiple files per audio object. Eventually, the system runs out of file descriptors.

Inspecting the RocksDB file .aim/seqs/chunks/<some chunk id>/LOG reveals that

Options.max_open_files: -1

This apparently means exactly, what I mentioned above: alls .sst files are kept open indefinitely, which in my case means roughly 190K file handles before the process errors with "too many open files".

So my question is, how can I change the RocksDB config so that this problem can be avoided? (I'd much rather correctly configure RocksDB/Aim than changing the system's file descriptor limits)

Thanks for your help!

PS: When logging an audio object using its path, I would love to see an option that lets me choose to just log the path and not the contents of the audio file in order to avoid copying my whole dataset to the .aim folder... In the UI, one could still display the audio object, just load it from the file system and not as BLOB from the RocksDB.