Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[E-3] 16-Evaluations / 12-LangSmith-Pairwise-Evaluation #399

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 8 commits into from
Closed

[E-3] 16-Evaluations / 12-LangSmith-Pairwise-Evaluation #399

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
ac0148f
feat: First version of Pairwise-Evaluation tutorial
BokyungisaGod Jan 15, 2025
9dd0f58
0117 - 16-Evaluations / 12-LangSmith-Pairwise-Evaluation 2차
BokyungisaGod Jan 16, 2025
bd72c08
0118 - 16-Evaluations / 12-LangSmith-Pairwise-Evaluation 이미지 대소문자 변경
BokyungisaGod Jan 17, 2025
56ac683
12-RAG / 1-RAG-Basic-PDF 번역검수
BokyungisaGod Jan 17, 2025
def8697
12-RAG / 3-RAG-Advanced 번역검수
BokyungisaGod Jan 17, 2025
c5b20bf
12-RAG / 04-FlashRank-Reranker 번역/검수
BokyungisaGod Jan 17, 2025
e6829f3
Merge branch 'LangChain-OpenTutorial:main' into BokyungisaGodTutorial2
BokyungisaGod Jan 17, 2025
0a25375
Delete 16-Evaluations/assets/12-LangSmith-Pairwise-Evaluation-01.png
BokyungisaGod Jan 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions 11-Reranker/04-FlashRank-Reranker.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@
"\n",
"## Overview\n",
"\n",
"> [FlashRank](https://github.com/PrithivirajDamodaran/FlashRank) is an ultra-lightweight and ultra-fast Python library designed to add reranking to existing search and `retrieval` pipelines. It is based on state-of-the-art (`SoTA`) `cross-encoders`.\n",
"> [FlashRank](https://github.com/PrithivirajDamodaran/FlashRank) is an ultra-lightweight and ultra-fast Python library designed to add reranking to existing search and **retrieval** pipelines. It is based on state-of-the-art (**SoTA**) **cross-encoders**.\n",
"\n",
"This notebook introduces the use of `FlashRank-Reranker` within the LangChain framework, showcasing how to apply reranking techniques to improve the quality of search or `retrieval` results. It provides practical code examples and explanations for integrating `FlashRank` into a LangChain pipeline, highlighting its efficiency and effectiveness. The focus is on leveraging `FlashRank`'s capabilities to enhance the ranking of outputs in a streamlined and scalable way.\n",
"This notebook introduces the use of `FlashRank-Reranker` within the LangChain framework, showcasing how to apply reranking techniques to improve the quality of search or **retrieval** results. It provides practical code examples and explanations for integrating `FlashRank` into a LangChain pipeline, highlighting its efficiency and effectiveness. The focus is on leveraging `FlashRank`'s capabilities to enhance the ranking of outputs in a streamlined and scalable way.\n",
"\n",
"### Table of Contents\n",
"\n",
Expand Down Expand Up @@ -63,7 +63,7 @@
"id": "7d83ee066d91fb4f",
"metadata": {},
"source": [
"You can alternatively set OPENAI_API_KEY in .env file and load it.\n",
"You can alternatively set OPENAI_API_KEY in `.env` file and load it.\n",
"\n",
"[Note] This is not necessary if you've already set OPENAI_API_KEY in previous steps."
]
Expand Down Expand Up @@ -137,7 +137,7 @@
"source": [
"## FlashrankRerank\n",
"\n",
"Load data for a simple example and create a retriever."
"Load data for a simple example and create a **retriever** ."
]
},
{
Expand Down Expand Up @@ -185,7 +185,7 @@
"id": "ea07e244c9171d26",
"metadata": {},
"source": [
"Now, let's wrap the base `retriever` with a `ContextualCompressionRetriever` and use `FlashrankRerank` as the compressor."
"Now, let's wrap the base **retriever** with a `ContextualCompressionRetriever` and use `FlashrankRerank` as the compressor."
]
},
{
Expand Down Expand Up @@ -224,7 +224,7 @@
"id": "4a147fc787860bac",
"metadata": {},
"source": [
"Compare the results after reanker is applied."
"Compare the results after reranker is applied."
]
},
{
Expand Down
38 changes: 17 additions & 21 deletions 12-RAG/01-RAG-Basic-PDF.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
"\n",
"### 1. Pre-processing - Steps 1 to 4\n",
"![rag-1.png](./assets/12-rag-rag-basic-pdf-rag-process-01.png)\n",
"![rag-1-graphic](./assets/12-rag-rag-basic-pdf-rag-graphic-1.png)\n",
"![](./assets/12-rag-rag-basic-pdf-rag-graphic-1.png)\n",
"\n",
"\n",
"The pre-processing stage involves four steps to load, split, embed, and store documents into a Vector DB (database).\n",
Expand All @@ -31,7 +31,7 @@
"\n",
"### 2. RAG Execution (RunTime) - Steps 5 to 8\n",
"![rag-2.png](./assets/12-rag-rag-basic-pdf-rag-process-02.png)\n",
"![rag-2-graphic](./assets/12-rag-rag-basic-pdf-rag-graphic-2.png)\n",
"![](./assets/12-rag-rag-basic-pdf-rag-graphic-2.png)\n",
"\n",
"\n",
"- **Step 5: Retriever** : Define a retriever to fetch results from the database based on the input query. Retrievers use search algorithms and are categorized as Dense or Sparse:\n",
Expand All @@ -40,7 +40,7 @@
"\n",
"- **Step 6: Prompt** : Create a prompt for executing RAG. The `context` in the prompt includes content retrieved from the document. Through prompt engineering, you can specify the format of the answer. \n",
"\n",
"- **Step 7: LLM** : Define the language model (e.g., GPT-3.5, GPT-4, Claude, etc.). \n",
"- **Step 7: LLM** : Define the language model (e.g., GPT-3.5, GPT-4, Claude). \n",
"\n",
"- **Step 8: Chain** : Create a chain that connects the prompt, LLM, and output. \n",
"\n",
Expand Down Expand Up @@ -116,12 +116,14 @@
"from langchain_opentutorial import package\n",
"\n",
"package.install(\n",
" [\"langchain_community\",\n",
" \"langsmith\"\n",
" \"langchain\"\n",
" \"langchain_text_splitters\"\n",
" \"langchain_core\"\n",
" \"langchain_openai\"],\n",
" [\n",
" \"langchain_community\",\n",
" \"langsmith\",\n",
" \"langchain\",\n",
" \"langchain_text_splitters\",\n",
" \"langchain_core\",\n",
" \"langchain_openai\"\n",
" ],\n",
" verbose=False,\n",
" upgrade=False,\n",
")"
Expand Down Expand Up @@ -185,7 +187,8 @@
"id": "9b0d050a",
"metadata": {},
"source": [
"## RAG Basic Pipeline"
"## RAG Basic Pipeline\n",
"(콘텐츠 채워주세요)"
]
},
{
Expand Down Expand Up @@ -471,7 +474,7 @@
"source": [
"# Step 5: Create Retriever\n",
"# Search and retrieve information contained in the documents.\n",
"retriever = vectorstore.as_retriever()\n"
"retriever = vectorstore.as_retriever()"
]
},
{
Expand Down Expand Up @@ -582,7 +585,7 @@
"id": "e79f4aeb",
"metadata": {},
"source": [
"Input a query (question) into the created chain and execute it."
"Input a query (`question`) into the created `chain` and execute it."
]
},
{
Expand Down Expand Up @@ -612,7 +615,8 @@
"id": "f8444e43",
"metadata": {},
"source": [
"## Complete code"
"## Complete code\n",
"(콘텐츠 채워주세요)"
]
},
{
Expand Down Expand Up @@ -699,14 +703,6 @@
"response = chain.invoke(question)\n",
"print(response)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "436b9bb7",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand Down
50 changes: 19 additions & 31 deletions 12-RAG/03_RAG_Advanced.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -289,7 +289,8 @@
"id": "9b0d050a"
},
"source": [
"## Explore Each Module"
"## Explore Each Module\n",
"(콘텐츠를 채워주세요)"
]
},
{
Expand Down Expand Up @@ -318,7 +319,7 @@
"id": "48e783c4"
},
"source": [
"Below is an example of using a basic RAG model for handling web pages `(WebBaseLoader)` .\n",
"Below is an example of using a basic RAG model for handling web pages (`WebBaseLoader`) .\n",
"\n",
"In each step, you can configure various options or apply new techniques."
]
Expand All @@ -330,7 +331,7 @@
"id": "2cd56394"
},
"source": [
"If a warning is displayed due to the `USER_AGENT` not being set when using the WebBaseLoader,\n",
"If a warning is displayed due to the `USER_AGENT` not being set when using the `WebBaseLoader`,\n",
"\n",
"please add `USER_AGENT = myagent` to the `.env` file."
]
Expand Down Expand Up @@ -558,7 +559,8 @@
"id": "f43be1cf"
},
"source": [
"### PDF\n"
"### PDF\n",
"(콘텐츠를 채워주세요)"
]
},
{
Expand Down Expand Up @@ -682,7 +684,8 @@
"id": "4866c8ea"
},
"source": [
"### TXT file\n"
"### TXT file\n",
"(콘텐츠를 채워주세요)"
]
},
{
Expand Down Expand Up @@ -953,7 +956,8 @@
"id": "b0321a46"
},
"source": [
"## Step 2: Split Documents"
"## Step 2: Split Documents\n",
"(콘텐츠를 채워주세요)"
]
},
{
Expand Down Expand Up @@ -1546,7 +1550,8 @@
"id": "9affbaf8"
},
"source": [
"### Paid Embeddings (OpenAI)"
"### Paid Embeddings (OpenAI)\n",
"(콘텐츠를 채워주세요)"
]
},
{
Expand Down Expand Up @@ -1688,7 +1693,8 @@
"id": "cd42083c"
},
"source": [
"## Step 4: Create Vectorstore"
"## Step 4: Create Vectorstore\n",
"(콘텐츠를 채워주세요)"
]
},
{
Expand Down Expand Up @@ -1878,7 +1884,8 @@
"id": "e9a96b86"
},
"source": [
"### Create a variety of queries"
"### Create a variety of queries\n",
"(콘텐츠를 채워주세요)"
]
},
{
Expand Down Expand Up @@ -2416,7 +2423,7 @@
"id": "00f6cd40"
},
"source": [
"### Use Higgingface\n",
"### Use HuggingFace\n",
"\n",
"You need a Hugging Face token to access LLMs on HuggingFace.\n",
"\n",
Expand Down Expand Up @@ -2505,7 +2512,8 @@
"id": "0d74046f"
},
"source": [
"## RAG Template Experiment\n"
"## RAG Template Experiment\n",
"(콘텐츠를 채워주세요)"
]
},
{
Expand Down Expand Up @@ -2725,26 +2733,6 @@
"response = rag_chain.invoke(question)\n",
"print(response)"
]
},
{
"cell_type": "code",
"execution_count": 60,
"id": "1e153841",
"metadata": {
"id": "1e153841"
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": 60,
"id": "3ec2358d",
"metadata": {
"id": "3ec2358d"
},
"outputs": [],
"source": []
}
],
"metadata": {
Expand Down
Loading
Loading