[E-2] 16-Evaluations / 07-LangSmith-Custom-LLM-Evaluation #359

kofsitho87 · 2025-01-14T10:02:21Z

{PR Message - Write freely, remove this bracket if unnecessary}

Author Checklist

PR Title Format: I have confirmed that the PR title follows the correct format. (e.g., [N-2] 07-Text Splitter / 07-RecursiveCharacterTextSplitter)
Committed Files: I have ensured that no unnecessary files (e.g., .bin, .gitignore, poetry.lock, pyproject.toml) are included. These files are not allowed.
(Optional) Related Issue: If this PR is linked to an issue, I have referenced the issue number in the PR message. (e.g., Fixes Update 01-PromptTemplate.ipynb #123)
❌ Do not include unnecessary files (e.g., .bin, .gitignore, poetry.lock, pyproject.toml) or other people's code. If included, close the PR and create a new PR.

Review Template (Intial PR)

🖥️ OS: Win/Mac/Linux   
✅ Checklist      
 - [ ] **Template**: Tutorials follows the required template. 
 - [ ] **Table of Contents(TOC) Links**: All Table of Contents links work. (Yes/No)
 - [ ] **Image**: Image filenames follow guidelines.
 - [ ] **Imports**: All import statements use the latest versions. Ensure "langchain-teddynote" is not used. 
 - [ ] **Code Execution**: Code runs without errors.
 - Comments: {Write freely, 한국어 기술 가능}

If no one reviews your PR within a few days, please @-mention one of teddylee777, musangk, BAEM1N

sungchul2

🖥️ OS: Linux
✅ Checklist

Template: Tutorials follows the required template.
Table of Contents(TOC) Links: All Table of Contents links work. (Yes/No)
Image: Image filenames follow guidelines.
Imports: All import statements use the latest versions. Ensure "langchain-teddynote" is not used.
Code Execution: Code runs without errors.
Comments: {Write freely, 한국어 기술 가능}
- 전반적으로 깔끔하게 작성해주셔서 좋았습니다. 감사합니다.

16-Evaluations/07-LangSmith-Custom-LLM-Evaluation.ipynb

- add `pymupdf`, `faiss-cpu` - update clean text format

- Change package installation method

architectyou

🖥️ OS: Win
✅ Checklist

Template: Tutorials follows the required template.
Table of Contents(TOC) Links: All Table of Contents links work. (Yes/No)
Image: Image filenames follow guidelines.
Imports: All import statements use the latest versions. Ensure "langchain-teddynote" is not used.
Code Execution: Code runs without errors.
Comments: 문서 작성해주시느라 너무 고생 많으셨습니다. 백틱 수정내용 말고 문제없이 모두 동작합니다!

architectyou · 2025-01-16T04:34:32Z

16-Evaluations/07-LangSmith-Custom-LLM-Evaluation.ipynb

+    "\n",
+    "## Overview\n",
+    "\n",
+    "`LangSmith Custom LLM Evaluation` is a customizable evaluation framework in LangChain that enables users to assess LLM application outputs based on their specific requirements.\n",


backtick은 변수, 함수, 모듈명에 사용하는 것으로 알고 있습니다! 볼드체로 변경해주심이 어떨까 싶습니다!

architectyou · 2025-01-16T04:50:34Z

16-Evaluations/07-LangSmith-Custom-LLM-Evaluation.ipynb

+    "\n",
+    "We'll build a basic **RAG** (Retrieval-Augmented Generation) system to test Custom Evaluators. This implementation creates a question-answering system based on PDF documents, which will serve as our foundation for evaluation purposes.\n",
+    "\n",
+    "This **RAG** system will be used to evaluate answer quality and accuracy through `Custom Evaluators` in later sections.\n",


이 부분 backtick도 함께 수정되면 좋을 것 같습니다! 'Custom Evaluators' > 볼드체

architectyou · 2025-01-16T04:52:00Z

16-Evaluations/07-LangSmith-Custom-LLM-Evaluation.ipynb

+    "\n",
+    "**Random Score Evaluator Implementation**\n",
+    "- Takes `Run` and `Example` objects as input parameters\n",
+    "- Returns a dictionary in the format: `{\"key\": \"score_name\", \"score\": score}`\n",


{\"key\": \"score_name\", \"score\": score} > 이부분도 수정해주시면 좋을 것 같습니다! (볼드체)

- update style to standardize markdown formatting

sungchul2

🖥️ OS: Win/Mac/Linux
✅ Checklist

Template: Tutorials follows the required template.
Table of Contents(TOC) Links: All Table of Contents links work. (Yes/No)
Image: Image filenames follow guidelines.
Imports: All import statements use the latest versions. Ensure "langchain-teddynote" is not used.
Code Execution: Code runs without errors.
Comments: {Write freely, 한국어 기술 가능}

16-Evaluations/07-LangSmith-Custom-LLM-Evaluation.ipynb

sungchul2 · 2025-01-16T15:35:21Z

16-Evaluations/07-LangSmith-Custom-LLM-Evaluation.ipynb

+    "from langchain_core.runnables import RunnablePassthrough, Runnable\n",
+    "\n",
+    "\n",
+    "class PDFRAG:\n",


최신 main 브랜치에 myrag.py가 추가되었습니다.
개인적으로 LLM 평가 주제에 PDFRAG를 설명하는 부분이 너무 많게 느껴져서 새로 추가된 모듈로 대체하는 것이 좋을 것 같습니다.
어떻게 생각하시나요?

해당 부분에 대해서 myrag.py 로 대체하겠습니다.
감사합니다.

16-Evaluations/07-LangSmith-Custom-LLM-Evaluation.ipynb

sungchul2 · 2025-01-16T15:38:54Z

16-Evaluations/07-LangSmith-Custom-LLM-Evaluation.ipynb

+    "   - `split_documents()`: Splits documents into appropriate sizes using RecursiveCharacterTextSplitter\n",
+    "\n",
+    "2. **Vector Store Creation**\n",
+    "   - `create_vectorstore()`: Creates vector DB using OpenAI Embeddings and FAISS\n",


Suggested change

" - `create_vectorstore()`: Creates vector DB using OpenAI Embeddings and FAISS\n",

" - `create_vectorstore()`: Creates vector DB using `OpenAIEmbeddings` and `FAISS`\n",

@kofsitho87 OpenAIEmbeddings로 빈칸 없이 수정해주세요.

16-Evaluations/07-LangSmith-Custom-LLM-Evaluation.ipynb

…OpenTutorial#359)

kofsitho87 · 2025-01-17T01:21:08Z

@rlatjcj 님 리뷰내용 감사드립니다. 요청하신 내용 반영하여 다시 커밋하였습니다.

architectyou

확인했습니다! 이상 없는 것 같습니다. 고생 많으셨습니다🤗

16-Evaluations/07-LangSmith-Custom-LLM-Evaluation.ipynb

sungchul2 · 2025-01-17T04:42:49Z

16-Evaluations/07-LangSmith-Custom-LLM-Evaluation.ipynb

+    "   - `split_documents()`: Splits documents into appropriate sizes using RecursiveCharacterTextSplitter\n",
+    "\n",
+    "2. **Vector Store Creation**\n",
+    "   - `create_vectorstore()`: Creates vector DB using OpenAI Embeddings and FAISS\n",


@kofsitho87 OpenAIEmbeddings로 빈칸 없이 수정해주세요.

sungchul2

🖥️ OS: Win/Mac/Linux
✅ Checklist

Template: Tutorials follows the required template.
Table of Contents(TOC) Links: All Table of Contents links work. (Yes/No)
Image: Image filenames follow guidelines.
Imports: All import statements use the latest versions. Ensure "langchain-teddynote" is not used.
Code Execution: Code runs without errors.
Comments: {Write freely, 한국어 기술 가능}

16-Evaluations/07-LangSmith-Custom-LLM-Evaluation.ipynb

…OpenTutorial#359)

architectyou

수정 반영 감사합니다! 고생많으셨습니다!

sungchul2

🖥️ OS: Linux
✅ Checklist

Template: Tutorials follows the required template.
Table of Contents(TOC) Links: All Table of Contents links work. (Yes/No)
Image: Image filenames follow guidelines.
Imports: All import statements use the latest versions. Ensure "langchain-teddynote" is not used.
Code Execution: Code runs without errors.
Comments: {Write freely, 한국어 기술 가능}
- 고생많으셨습니다!

[E-2] 16-Evaluations / 07-LangSmith-Custom-LLM-Evaluation

556f186

kofsitho87 requested review from sungchul2 and architectyou January 14, 2025 10:02

sungchul2 requested changes Jan 15, 2025

View reviewed changes

kofsitho87 added 3 commits January 16, 2025 08:26

Merge branch 'LangChain-OpenTutorial:main' into main

61ab8e2

[E-2] 16-Evaluations / 07-LangSmith-Custom-LLM-Evaluation

2edb5f0

- add `pymupdf`, `faiss-cpu` - update clean text format

[E-2] 16-Evaluations / 07-LangSmith-Custom-LLM-Evaluation

777af35

- Change package installation method

kofsitho87 requested a review from sungchul2 January 16, 2025 04:12

architectyou requested changes Jan 16, 2025

View reviewed changes

[E-2] 16-Evaluations / 07-LangSmith-Custom-LLM-Evaluation

c90d8a1

- update style to standardize markdown formatting

sungchul2 requested changes Jan 16, 2025

View reviewed changes

kofsitho87 added 2 commits January 17, 2025 10:16

Merge remote-tracking branch 'upstream/main'

a2a4605

[E-2] 16-Evaluations / 07-LangSmith-Custom-LLM-Evaluation (LangChain-…

5c477e8

…OpenTutorial#359)

architectyou previously approved these changes Jan 17, 2025

View reviewed changes

sungchul2 reviewed Jan 17, 2025

View reviewed changes

sungchul2 requested changes Jan 17, 2025

View reviewed changes

16-Evaluations/07-LangSmith-Custom-LLM-Evaluation.ipynb Outdated Show resolved Hide resolved

sungchul2 reviewed Jan 17, 2025

View reviewed changes

16-Evaluations/07-LangSmith-Custom-LLM-Evaluation.ipynb Outdated Show resolved Hide resolved

[E-2] 16-Evaluations / 07-LangSmith-Custom-LLM-Evaluation (LangChain-…

9db9f34

…OpenTutorial#359)

kofsitho87 dismissed architectyou’s stale review via 9db9f34 January 17, 2025 08:45

architectyou approved these changes Jan 17, 2025

View reviewed changes

sungchul2 approved these changes Jan 17, 2025

View reviewed changes

Merge branch 'LangChain-OpenTutorial:main' into main

725dbaa

teddylee777 merged commit dba139d into LangChain-OpenTutorial:main Jan 18, 2025
1 of 2 checks passed

sohyunwriter added the docs tutorial label Jan 18, 2025

	" - `create_vectorstore()`: Creates vector DB using OpenAI Embeddings and FAISS\n",
	" - `create_vectorstore()`: Creates vector DB using `OpenAIEmbeddings` and `FAISS`\n",

[E-2] 16-Evaluations / 07-LangSmith-Custom-LLM-Evaluation #359

[E-2] 16-Evaluations / 07-LangSmith-Custom-LLM-Evaluation #359

Uh oh!

Conversation

kofsitho87 commented Jan 14, 2025

Author Checklist

Review Template (Intial PR)

Uh oh!

sungchul2 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

architectyou left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sungchul2 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kofsitho87 commented Jan 17, 2025

Uh oh!

architectyou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sungchul2 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

architectyou left a comment

Choose a reason for hiding this comment

Uh oh!

sungchul2 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

architectyou left a comment •

edited

Loading