New RAG example #427
New RAG example #427
Conversation
We could rename the old example to "tfidf_rag" or something like that. And then, we can use the name "rag" for your new example, to indicate that it is now the main official RAG example. |
There was a problem hiding this comment.
Nice example!
When trying to execute the program, I had a few dependency missing. I had to do:
pip install pymilvus pypdf
You could also say in the readme which models to install with ollama.
And here are some suggestions on how to rewrite pdf_index.pdl:
# Load PDF document into vector database
description: Load document into vector database
lastOf:
- include: rag_library1.pdl
- defs:
input_data:
call: ${ pdf_parse }
args:
filename: "docs/assets/pdl_quick_reference.pdf"
chunk_size: 400
chunk_overlap: 100
call: ${ rag_index }
args:
inp: ${ input_data }
encoder_model: "ollama/mxbai-embed-large"
embed_dimension: 1024
database_name: "./pdl-rag-demo.db"
collection_name: "pdl_rag_collection"
- "Success!"
and pdf_query.pdl:
# Query vector database for relevant passages; use passages to query LLM.
defs:
QUESTIONS:
data: [
"Does PDL have a contribute keyword?",
"Is Brooklyn the capital of New York?"
]
lastOf:
- include: rag_library1.pdl
- defs:
CONCLUSIONS:
for:
question: ${ QUESTIONS }
repeat:
# Define MATCHING_PASSAGES as the text retrieved from the vector DB
defs:
MATCHING_PASSAGES:
call: ${ rag_retrieve }
args:
# I am passing the client in implicitly. NOT WHAT I WANT
inp: ${ question }
encoder_model: "ollama/mxbai-embed-large"
limit: 3
collection_name: "pdl_rag_collection"
database_name: "./pdl-rag-demo.db"
# debug:
# lang: python
# code: |
# print(f"MATCHING_PASSAGES='{MATCHING_PASSAGES}'")
# result = None
CONCLUSION:
model: ollama/granite-code:8b
input: >
Here is some information:
${ MATCHING_PASSAGES }
Question: ${ question }
Answer:
parameters:
# I couldn't get this working
stop_sequences: ['Yes', 'No']
temperature: 0
ANSWER:
lang: python
code: |
# split()[0] needed because of stop_sequences not working
# print(f"CONCLUSION={CONCLUSION}")
result = CONCLUSION.split()[0]
data:
${question}: ${ANSWER}
join:
as: array
text: "${ CONCLUSIONS | tojson }\n"
There was a problem hiding this comment.
Example looks great! See comments.
Langchain provides different types of retrievers with a unified interface. Do the abstractions here reflect that unified interface?
https://python.langchain.com/docs/concepts/retrievers/
There was a problem hiding this comment.
To make the model stop, use stop instead of stop_sequences
There was a problem hiding this comment.
Could we keep both RAG examples? It would be good for the README to reflect that.
Signed-off-by: Ed Snible <[email protected]>
Signed-off-by: Ed Snible <[email protected]>
Signed-off-by: Ed Snible <[email protected]>
Signed-off-by: Ed Snible <[email protected]>
* New RAG example
This is a RAG example that uses the correct lifecycle and the Milvus vector database.
Questions. I need feedback before merging.
logging_objnot found - unable to track `llm_api_duration_ms". I am not sure if these are my fault.