A Web UI where you can upload CSV/JSON files, create vector embeddings, and query them. Soon, you'll be able to convert unstructured data to JSON/CSV using an integrated LLM.
Project under heavy/active development [may be] unstable. Embeddings and Query pages WIP ⚠️
| Requirement | Description |
|---|---|
| The application requires Python 3.7 or higher to leverage modern libraries and syntax. | |
| Essential for running embedding models. Utilized by transformers. | |
| Enables CORS for frontend-backend communication. | |
| Used for creating vector embeddings with pre-trained models. | |
| Essential for running embedding models. Utilized by transformers. | |
| Handles arrays and mathematical operations. Used throughout the application. | |
| Processes CSV and JSON files. Utilized throughout the application. |
| Step | Instructions |
|---|---|
| Clone the repository | |
| Install the required packages | |
| Step | Instructions |
|---|---|
| Start the Flask server | |
| Automatically launch React frontend | The Python endpoint will launch the React frontend in a separate subprocess. |
| Open your web browser | Navigate to http://127.0.0.1:4000 |
| Step | Instructions |
|---|---|
| Drag and Drop a File | Drag and drop a CSV or JSON file into the upload area or click to select a file from your computer. |
| View Uploaded File Information | Once uploaded, the file information such as name and size will be displayed. |
To upload a file using curl:
curl -X POST -F 'file=@/path/to/your/file.csv' http://127.0.0.1:4000/upload_file| Step | Instructions |
|---|---|
| Select Embedding Keys | After uploading a file, select the keys (columns) you want to include in the embeddings. |
| Preview Document | View a preview of the document created from the selected keys. |
| Preview Embeddings | View the generated embeddings and token count for the selected document. |
To preview a file's keys using curl:
curl -X POST -H "Content-Type: application/json" -d '{"file_path":"uploads/your-file.csv"}' http://127.0.0.1:4000/preview_fileTo preview a document's embeddings using curl:
curl -X POST -H "Content-Type: application/json" -d '{"file_path":"uploads/your-file.csv", "selected_keys":["key1", "key2"]}' http://127.0.0.1:4000/preview_document| Step | Instructions |
|---|---|
| Start Embedding Creation | Click the "Create Vector DB" button to start the embedding creation process. |
| View Progress | Monitor the progress of the embedding creation with a circular progress indicator. 📈 |
To create a vector database using curl:
curl -X POST -H "Content-Type: application/json" -d '{"file_path":"uploads/your-file.csv", "selected_keys":["key1", "key2"]}' http://127.0.0.1:4000/create_vector_database| Step | Instructions |
|---|---|
| Enter Query | Type your query into the input field. |
| Select Similarity Metric | Choose between cosine similarity or Euclidean distance. |
| Submit Query | Click the "Submit" button to query the vector database. |
| View Results | Inspect the results, which display the document, score, and a button to view detailed data. |
To query the vector database using curl:
curl -X POST -H "Content-Type: application/json" -d '{"query_text":"Your query text here", "similarity_metric":"cosine"}' http://127.0.0.1:4000/query| Step | Instructions |
|---|---|
| Backup Database | Click the "Backup Database" button to create a backup of the current vector database. |
| Delete Database | Click the "Delete Database" button to delete the current vector database. |
| View Database Statistics | View statistics such as total documents and average vector length. |
To check if the vector database exists using curl:
curl -X GET http://127.0.0.1:4000/check_vector_dbTo view database statistics using curl:
curl -X GET http://127.0.0.1:4000/db_statsTo backup the database using curl:
curl -X POST http://127.0.0.1:4000/backup_dbTo delete the database using curl:
curl -X POST http://127.0.0.1:4000/delete_db| Feature | Description |
|---|---|
| Uploading Files |
|
| Previewing Data |
|
| Creating Vector Embeddings |
|
| Querying the Vector Database |
|
| Managing the Vector Database |
|