Complete Architecture:

(ipynb files)

Final : https://www.kaggle.com/code/rohansai2208/final

Packed unpacked detection : https://www.kaggle.com/code/rohansai2208/packed-unpacked/notebook

QWEN OCR : https://www.kaggle.com/code/vamshikuruva/qwen-ocr-01/notebook

Ultralytics SAM : https://www.kaggle.com/code/rohansai2208/ultralytics-sam-1/

Fruits and Vegetables Detection and Freshess Identification : https://www.kaggle.com/code/rohansai2208/fruits-and-vegetables

Dataset Created :

Dataset used for Vegetables and Fruits Detection : https://www.kaggle.com/datasets/rohansai2208/woking-dataset

This project is an advanced image analysis system designed to process and classify both single and multiple object images. It utilizes state-of-the-art machine learning models to segment, classify, and extract detailed information from images of packed and unpacked items, with a focus on groceries and household products.

1. Dual Image Processing: Analyzes both front and back images of objects.
2. Single and Multiple Object Handling: Capable of processing images containing single or multiple objects.
3. Object Segmentation: Uses SAM (Segment Anything Model) for precise object detection and segmentation in multi-object images.
4. Classification:
   • Packed vs Unpacked items
   • Fruit and Vegetable classification (54 categories)
   • Fresh vs Rotten for produce items
5. Detailed Product Information Extraction: For packed items, extracts product name, expiry date, description, and category.
6. Output Generation: Produces CSV files for combined items, packed items, and unpacked items.

• Python 3.x
• PyTorch and torchvision
• Ultralytics SAM
• OpenCV
• Pandas
• Pillow (PIL)
• Scikit-learn
• Transformers (Hugging Face)
• Matplotlib

1. SAM (Segment Anything Model): For object segmentation
2. EfficientNet-B0:
   • Packed/Unpacked classification
   • Fruit/Vegetable classification (54 categories)
   • Fresh/Rotten classification
3. Qwen2VL: For extracting detailed product information

1. Clone the repository:

   git clone [repository-url]
   cd [repository-name]
   

2. Install required packages:

   pip install -r requirements.txt
   

3. Ensure you have the following pre-trained model files in the appropriate directory:

   • sam2_b.pt
   • efficientnet_b0_packed_unpacked.pth
   • efficientnet_b0_fruit_veg_1.pth
   • efficientnet_b0_fruit_veg.pth

The main execution code is contained in final.ipynb. To use the system:

1. Open final.ipynb in a Jupyter Notebook environment.
2. Run all cells in the notebook.
3. When prompted, specify whether you're processing a single object or multiple objects.
4. Provide the paths to the front and back images when asked.
5. The system will process the images and generate CSV files with the results.

The script generates three CSV files:

1. combined_items.csv: All processed items
2. packed_items.csv: Information on packed items
3. unpacked_items.csv: Information on unpacked items (fruits, vegetables)

• final.ipynb: Main Jupyter notebook containing all functions and the execution flow
• requirements.txt: List of required Python packages
• README.md: This file
• Other supporting Python scripts and model files

• Implement a graphical user interface for easier interaction
• Enhance object matching between front and back images
• Integrate with a database for persistent storage
• Add support for video processing
• Implement more robust error handling and logging

Aniketh : https://github.com/Aniketh007

Vamshi: https://github.com/Vamshikuruva

Rajesh: https://github.com/venkatknsr

[MIT License]