Examples

title	summary	input_data_type	uniflow_type	url
OpenAI Evaluate Answer Completeness Accuracy for Given Questions	This notebook uses uniflow to evaluate the completeness and accuracy of answers for given questions using OpenAI model. It provides insights into the performance of the model in generating accurate and complete answers.	Jupyter Notebook	TransformOpenAIFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/rater/openai_evaluate_answer_completeness_accuracy_for_given_questions.ipynb
OpenAI Compare Generated Answers to Grounding Answer	This notebook compares the answers generated by OpenAI language model to a grounding answer for evaluation. It provides a method to assess the quality of the generated answers.	Jupyter Notebook	TransformOpenAIFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/rater/openai_compare_generated_answers_to_grounding_answer.ipynb
Bedrock Evaluate Answer Completeness Accuracy for Given Questions	This notebook uses uniflow to evaluate the completeness and accuracy of answers for given questions using the Bedrock model. It provides insights into the quality of answers and helps in identifying areas for improvement.	Jupyter Notebook	BedrockFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/rater/bedrock_evaluate_answer_completeness_accuracy_for_given_questions.ipynb
Huggingface Evaluate Answer Completeness Accuracy for Given Questions	This notebook uses Huggingface model to evaluate the completeness and accuracy of answers for given questions. It provides a comprehensive analysis of the model's performance in understanding and answering questions.	Jupyter Notebook	TransformHuggingFaceFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/rater/huggingface_evaluate_answer_completeness_accuracy_for_given_questions.ipynb
PDF Extraction and Text Cleaning with Uniflow	This notebook demonstrates how to use uniflow for PDF extraction and text cleaning, including data clustering for further analysis. It provides a step-by-step guide for processing PDF data and preparing it for clustering.	PDF	TransformLMQGFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/pipeline/pipeline_s3_txt.ipynb
PDF Extraction and Text Cleaning with Data Clustering	This notebook demonstrates the process of extracting text from PDF documents, cleaning the text data, and clustering the cleaned text data for further analysis. It provides a comprehensive pipeline for preprocessing PDF data and preparing it for downstream tasks.	PDF	TransformLMQGFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/pipeline/pipeline_pdf_extract_transform.ipynb
Pipeline Web Summary	This notebook demonstrates the use of uniflow for PDF extraction, text cleaning, and data clustering to generate a summary of web content. It showcases the end-to-end pipeline for web content analysis using uniflow.	PDF, HTML	TransformLMQGFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/pipeline/pipeline_web_summary.ipynb
PDF Extraction and Text Cleaning with Data Clustering	This notebook demonstrates the process of extracting text from PDF documents, cleaning the text data, and clustering the cleaned text data for further analysis.	PDF	TransformLMQGFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/pipeline/pipeline_pdf.ipynb
LLM Based PDF Extraction, Text Cleaning, Data Clustering	This notebook demonstrates the use of LLM for PDF extraction, text cleaning, and data clustering. It showcases the end-to-end workflow of processing PDF documents, cleaning the text data, and clustering similar documents based on their content.	PDF	TransformLLMFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/vector_database/setup_resources.ipynb
Extract PDF with Recursive Splitter	This notebook demonstrates how to use uniflow to extract text from PDF files using a recursive splitter, and then clean the extracted text data. It also showcases data clustering techniques to organize the extracted text data.	PDF	TransformLMQGFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/extract/extract_pdf_with_recursive_splitter.ipynb
Extract HTML	This notebook demonstrates how to use uniflow to extract text from HTML documents and clean the extracted text for data clustering. It also provides examples of using LLM-based PDF extraction for text cleaning.	HTML	TransformLLMFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/extract/extract_html.ipynb
Extracting Text from PDF and Cleaning Data for Clustering	This notebook demonstrates how to use uniflow to extract text from PDF and clean the data for clustering. It includes preprocessing steps such as text extraction, data cleaning, and feature engineering.	PDF	TransformLMQGFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/extract/extract_md.ipynb
Extract PDF Nougat QA	This notebook demonstrates the process of extracting text from PDF documents using uniflow's LLM-based PDF extraction and performing data cleaning and clustering for QA purposes.	PDF	LLM-based PDF Extraction	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/extract/extract_pdf_nougat_qa.ipynb
Extract Text from PDF	This notebook demonstrates how to use uniflow to extract text from PDF documents, clean the extracted text, and perform data clustering for further analysis.	PDF	TransformLMQGFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/extract/extract_txt.ipynb
Extract Text from PDF and Clean	This notebook demonstrates how to extract text from PDF files and clean the extracted text for further processing. It includes techniques for handling special characters, removing noise, and normalizing the text data.	PDF	TransformLMQGFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/extract/extract_txt_from_s3.ipynb
PDF Extraction and Text Cleaning with Data Clustering	This notebook demonstrates the process of extracting text from PDF documents, cleaning the text data, and clustering the cleaned text data for further analysis.	PDF	TransformLMQGFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/transform/openai_html_QA.ipynb
Question Answering with OpenAI GPT-3 on HTML Data	Using OpenAI GPT-3 model, this notebook performs question answering on HTML data, demonstrating the capability of the model to understand and respond to questions based on the provided HTML content.	HTML	TransformOpenAIFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/transform/openai_html_QA.ipynb
OpenAI PDF Source 10k Summary	This notebook demonstrates the use of uniflow for extracting text from PDF documents, cleaning the text data, and clustering the cleaned text data. It provides a summary of 10k PDF documents using OpenAI model.	PDF	TransformOpenAIFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/transform/openai_pdf_source_10k_summary.ipynb
OpenAI Jupyter Notebook QA	This notebook demonstrates how to use uniflow to perform question answering on Jupyter notebooks using OpenAI's language model. It includes examples of extracting text from Jupyter notebooks, cleaning the text, and performing data clustering.	Jupyter Notebook	TransformOpenAIFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/transform/openai_jupyter_notebook_QA.ipynb
Huggingface Model Benchmark Neuron	This notebook benchmarks the performance of a Huggingface model for text extraction and data clustering using uniflow. It compares the model's speed and accuracy with different input data.	Jupyter Notebook	TransformHuggingFaceFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/transform/huggingface_model_benchmark_neuron.ipynb
PDF Extraction and Text Cleaning with Uniflow	This notebook demonstrates how to use uniflow for extracting text from PDF documents and cleaning the text data for further analysis. It includes preprocessing steps such as text extraction, text cleaning, and data clustering.	PDF	TransformGoogleMultiModalModelFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/transform/google_multimodal_model.ipynb
OpenAI PDF Source 10k QA	This notebook demonstrates the use of uniflow for extracting text from PDF documents, cleaning the text data, and clustering the data using OpenAI's language model. It also includes a question-answering task on the extracted text data.	PDF	TransformOpenAIFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/transform/openai_pdf_source_10k_QA.ipynb
Huggingface Model Benchmark G5	This notebook benchmarks the performance of a Huggingface model G5 for text extraction and data clustering using uniflow. It compares the model's accuracy and efficiency in processing large datasets.	Jupyter Notebook	TransformHuggingFaceFlow	https://github.com/CambioML/uniflow-llm-based-pdf-extraction-text-cleaning-data-clustering/tree/main/example/transform/huggingface_model_benchmark_g5.ipynb

---

Uniflow Configs

Base Config

The base Config is the base configuration that all other configurations inherit from. Here are the default parameters:

Parameter	Type	Default	Description
flow_name	str	[ModelFlow]	The name of the flow to run.
prompt_template	PromptTemplate	Default	The template to use for the guided prompt.
num_threads	int	1	The number of threads to use.
model_config	ModelConfig	ModelConfig	The model configuration to use.

Here are the default parameters for the ModelConfig:

Parameter	Type	Default	Description
model_name	str	gpt-3.5-turbo-1106	The name of the model to use.

The model.ipynb notebook shows a basic example of how to use the base Config, where it also passes the OpenAIModelConfig as a model_config argument.

OpenAIConfig

The OpenAIConfig configuration runs the following default parameters:

Parameter	Type	Default	Description
flow_name	str	OpenAIModelFlow	The name of the flow to run.
prompt_template	PromptTemplate	Default	The template to use for the guided prompt.
num_threads	int	1	The number of threads to use.
model_config	ModelConfig	OpenAIModelConfig	The model configuration to use.

Here are the default parameters for the OpenAIModelConfig:

Parameter	Type	Default	Description
model_name	str	gpt-3.5-turbo-1106	The name of the model to use.
num_call	int	1	The number of calls to make to the OpenAI model
temperature	float	1.5	The temperature to use for the OpenAI model.
response_format	Dict[str, str]	{"type": "text"}	The response format to use for the OpenAI model.

See the openai_json_model.ipynb notebook for a working example.

HuggingfaceConfig

The HuggingfaceConfig configuration has the following default parameters:

Parameter	Type	Default	Description
flow_name	str	HuggingfaceModelFlow	The name of the flow to run.
prompt_template	PromptTemplate	Default	The template to use for the guided prompt.
num_threads	int	1	The number of threads to use.
model_config	ModelConfig	HuggingfaceModelConfig	The model configuration to use.

Here are the default parameters for the HuggingfaceModelConfig:

Parameter	Type	Default	Description
model_name	str	mistralai/Mistral-7B-Instruct-v0.1	The name of the model to use.
batch_size	int	1	The batch size to use for the Huggingface model.

See the huggingface_model.ipynb notebook for a working example.

LMQGModelConfig

The LMQGModelConfig configuration runs with the following default parameters:

Parameter	Type	Default	Description
flow_name	str	LMQGModelFlow	The name of the flow to run.
prompt_template	PromptTemplate	Default	The template to use for the guided prompt.
num_threads	int	1	The number of threads to use.
model_config	ModelConfig	LMQGModelConfig	The model configuration to use.

Here are the default parameters for the LMQGModelConfig:

Parameter	Type	Default	Description
model_name	str	lmqg/t5-base-squad-qg-ae	The name of the model to use.
batch_size	int	1	The batch size to use for the LMQG model.

See the lmqg_model.ipynb notebook for a working example.