PaliGemma

This folder is organized into several categories, each focusing on a speicific aspect of working with PaliGemma models:

• Inference : How to load and run PaliGemma models for inference
• Finetuning : Dive into finetuning PaliGemma models for specific tasks and domains

Inference

Notebook Name	Description
[PaliGemma_1]Common_use_cases.ipynb	Illustrate some common use cases for PaliGemma.
[PaliGemma_1]Image_captioning.ipynb	Use PaliGemma to generate image captions using Keras.
[PaliGemma_1]Referring_expression_segmentation_in_images.ipynb	Referring Expression Segmentation in images using PaliGemma.
[PaliGemma_1]Referring_expression_segmentation_in_videos.ipynb	Referring Expression Segmentation in videos using PaliGemma.
[PaliGemma_1]Using_with_Mesop.ipynb	Integrate PaliGemma with Google Mesop.
[PaliGemma_1]Zero_shot_object_detection_in_images.ipynb	Zero-shot Object Detection in images using PaliGemma.
[PaliGemma_1]Zero_shot_object_detection_in_videos.ipynb	Zero-shot Object Detection in videos using PaliGemma.
[PaliGemma_2]Convert_PaliGemma2_to_ONNX.ipynb	Convert and quantize PaliGemma 2 to ONNX format, making it compatible for inferencing with Transformers.js.
[PaliGemma_2]Inference_PaliGemma2_with_Transformers_js.ipynb	Inference PaliGemma 2 with Transformers.js for tasks like image captioning, zero-shot object detection, OCR, and visual Q&A.
[PaliGemma_2]Keras_Quickstart.ipynb	PaliGemma 2 3B DOCCI model quickstart tutorial with Keras
[PaliGemma_2]Using_with_Transformersjs.ipynb	Run PaliGemma 2 with Transformers.js.

Finetuning

Notebook Name	Description
[PaliGemma_1]Finetune_with_Keras.ipynb	Finetune PaliGemma with Keras.
[PaliGemma_1]Finetune_with_image_captioning.ipynb	Compare the image captioning results using different PaliGemma versions with Hugging Face.
[PaliGemma_1]Finetune_with_image_description.ipynb	Finetune PaliGemma for image description using JAX.
[PaliGemma_1]Finetune_with_object_detection.ipynb	Fine-tune PaliGemma for object detection on a fashion dataset using JAX.
[PaliGemma_2]Finetune_with_JAX.ipynb	Finetune PaliGemma 2 with JAX.
[PaliGemma_2]Finetune_with_Keras.ipynb	Finetune PaliGemma 2 with Keras.