Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[N-3] 13-LangChain-Expression-Language / 12-RunnableRetry #256

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 11, 2025
Merged

[N-3] 13-LangChain-Expression-Language / 12-RunnableRetry #256

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
373 changes: 373 additions & 0 deletions 13-LangChain-Expression-Language/12-RunnableRetry.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,373 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# RunnableRetry\n",
"\n",
"- Author: [PangPangGod](https://github.com/pangpanggod)\n",
"- Design: []()\n",
"- Peer Review : []()\n",
"- This is a part of [LangChain Open Tutorial](https://github.com/LangChain-OpenTutorial/LangChain-OpenTutorial)\n",
"\n",
"[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/LangChain-OpenTutorial/LangChain-OpenTutorial/blob/main/13-LangChain-Expression-Language/12-RunnableRetry.ipynb) [![Open in GitHub](https://img.shields.io/badge/Open%20in%20GitHub-181717?style=flat-square&logo=github&logoColor=white)](https://github.com/LangChain-OpenTutorial/LangChain-OpenTutorial/blob/main/13-LangChain-Expression-Language/12-RunnableRetry.ipynb)\n",
"\n",
"## Overview\n",
"\n",
"This tutorial covers how to use `RunnableRetry` to handle retry logic effectively in LangChain workflows. \n",
"We'll demonstrate how to configure and use `RunnableRetry` with examples that showcase custom retry policies to make your workflow resilient to failures.\n",
"\n",
"### Table of Contents\n",
"\n",
"- [Overview](#overview)\n",
"- [Environement Setup](#environment-setup)\n",
"- [What is RunnableRetry](#what-is-runnableretry)\n",
"- [Why Use RunnableRetry](#why-use-runnableretry)\n",
"- [Base RunnableRetry Example](#base-runnableretry-example)\n",
"- [RunnableRetry Bind with Chains](#runnableretry-bind-with-chains)\n",
"\n",
"### References\n",
"\n",
"- [LangChain API Reference: RunnableRetry](https://python.langchain.com/api_reference/core/runnables/langchain_core.runnables.retry.RunnableRetry.html)\n",
"- [LangChain OpenTutorial: PydanticOutputParser](https://github.com/LangChain-OpenTutorial/LangChain-OpenTutorial/blob/main/03-OutputParser/01-PydanticOuputParser.ipynb)\n",
"----\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Environment Setup\n",
"\n",
"Set up the environment. You may refer to [Environment Setup](https://wikidocs.net/257836) for more details.\n",
"\n",
"**[Note]**\n",
"- `langchain-opentutorial` is a package that provides a set of easy-to-use environment setup, useful functions and utilities for tutorials. \n",
"- You can checkout the [`langchain-opentutorial`](https://github.com/LangChain-OpenTutorial/langchain-opentutorial-pypi) for more details."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"[notice] A new release of pip is available: 23.2.1 -> 24.3.1\n",
"[notice] To update, run: python.exe -m pip install --upgrade pip\n"
]
}
],
"source": [
"%%capture --no-stderr\n",
"%pip install langchain-opentutorial"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"# Install required packages\n",
"from langchain_opentutorial import package\n",
"\n",
"package.install(\n",
" [\n",
" \"langchain_core\",\n",
" \"langchain_openai\",\n",
" ],\n",
" verbose=False,\n",
" upgrade=False,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Environment variables have been set successfully.\n"
]
}
],
"source": [
"# Set environment variables\n",
"from langchain_opentutorial import set_env\n",
"\n",
"set_env(\n",
" {\n",
" \"OPENAI_API_KEY\": \"\",\n",
" \"LANGCHAIN_API_KEY\": \"\",\n",
" \"LANGCHAIN_TRACING_V2\": \"true\",\n",
" \"LANGCHAIN_ENDPOINT\": \"https://api.smith.langchain.com\",\n",
" \"LANGCHAIN_PROJECT\": \"RunnableRetry\",\n",
" }\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can alternatively set API keys such as `OPENAI_API_KEY` in a `.env` file and load them.\n",
"\n",
"[Note] This is not necessary if you've already set the required API keys in previous steps."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Load API keys from .env file\n",
"from dotenv import load_dotenv\n",
"\n",
"load_dotenv(override=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## What is RunnableRetry\n",
"\n",
"`RunnableRetry` is a utility provided by LangChain that allows you to add retry mechanisms for individual `Runnable` objects. \n",
"Instead of wrapping your entire workflow in retry logic, you can apply retry policies at the level of specific tasks. \n",
"This helps you handle transient issues, such as network errors or intermittent failures, without restarting the entire workflow.\n",
"\n",
"## Why Use RunnableRetry\n",
"\n",
"By using `RunnableRetry`, you can:\n",
"\n",
"- **Avoid wrapping the entire workflow with retry logic**: Instead of restarting the entire process during frequent network calls or API failures, you can retry individual `Runnable` units.\n",
"- **Implement retries per task**: This enables more efficient task recovery and makes workflows more robust.\n",
"- **Flexible implementation**: You can implement retries using `.with_retry()` or define a custom retry strategy by creating a `RunnableRetry` with specific events, such as exception types and exponential backoff."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Base RunnableRetry Example\n",
"\n",
"Below is a simple example to demonstrate the effectiveness of `RunnableRetry`. \n",
"In this example, we simulate a task with a chance of failure and use `RunnableRetry` to automatically retry it up to a maximum attempts."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Failed! The number is 3.\n",
"Success! The number is 1.\n"
]
}
],
"source": [
"import random\n",
"from langchain_core.runnables import RunnableLambda\n",
"from langchain_core.runnables.retry import RunnableRetry\n",
"\n",
"# Define a simple function with a chance of failure.\n",
"def random_number(input):\n",
" number = random.randint(1, 3)\n",
" if number == 1:\n",
" print(\"Success! The number is 1.\")\n",
" else:\n",
" print(f\"Failed! The number is {number}.\")\n",
" raise ValueError\n",
"\n",
"# Bind the function to RunnableLambda\n",
"runnable = RunnableLambda(random_number)\n",
"\n",
"# Then bind it to RunnableRetry\n",
"runnable_with_retries = RunnableRetry(\n",
" bound=runnable,\n",
" retry_exception_types=(ValueError,),\n",
" max_attempt_number=5,\n",
" wait_exponential_jitter=True,\n",
")\n",
"\n",
"# In this example, there is no need for input, but LangChain's Runnable requires an input argument.\n",
"# TypeError: RunnableRetry.invoke() missing 1 required positional argument: 'input'\n",
"input=None\n",
"runnable_with_retries.invoke(input)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"or you can simply implemented with `.with_retry()` method."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Failed! The number is 3.\n",
"Failed! The number is 2.\n",
"Success! The number is 1.\n"
]
}
],
"source": [
"# Bind the function to RunnableLambda\n",
"runnable = RunnableLambda(random_number)\n",
"\n",
"# with .with_retry(), no need to bind with Runnableretry(bind= ...)\n",
"runnable_with_retries = runnable.with_retry(\n",
" retry_if_exception_type=(ValueError,),\n",
" stop_after_attempt=3,\n",
" wait_exponential_jitter=True,\n",
")\n",
"\n",
"input=None\n",
"runnable_with_retries.invoke(None)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## RunnableRetry Bind with Chains\n",
"\n",
"In this example, we’ll take it a step further and demonstrate how to construct a Chain using `ChatOpenAI`. The example will show not just the basic chain setup but also how to enhance it by incorporating `RunnableRetry` for robust error handling and `PydanticOutputParser` for structured output validation.\n",
"\n",
"### Components Used:\n",
"- `RunnableRetry`: Automatically retries failed tasks to handle transient issues, such as API call failures or timeouts.\n",
"- `PydanticOutputParser`: Ensures the output is parsed and validated against a defined schema, making the workflow more reliable and predictable.\n",
"\n",
"with `PydanticOutputParser`, check our another tutorial [here](https://github.com/LangChain-OpenTutorial/LangChain-OpenTutorial/blob/main/03-OutputParser/01-PydanticOuputParser.ipynb)."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"from langchain_openai import ChatOpenAI\n",
"from langchain_core.runnables.retry import RunnableRetry\n",
"from langchain_core.prompts import PromptTemplate\n",
"\n",
"# Note: Each model provider may have different error classes for API-related failures.\n",
"# For example, OpenAI uses \"InternalServerError\", while other providers may define different exceptions.\n",
"from openai import InternalServerError\n",
"\n",
"# first, define model and prompt\n",
"model = ChatOpenAI(model=\"gpt-4o-mini\")\n",
"prompt = PromptTemplate.from_template(\"tell me a joke about {topic}.\")\n",
"\n",
"# bind with RunnableRetry\n",
"model_bind_with_retry = RunnableRetry(\n",
" bound=model,\n",
" retry_exception_types=(InternalServerError,),\n",
" max_attempt_number=3,\n",
" wait_exponential_jitter=True,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"AIMessage(content='Why do programmers prefer dark mode?\\n\\nBecause light attracts bugs!', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 13, 'prompt_tokens': 14, 'total_tokens': 27, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_d02d531b47', 'finish_reason': 'stop', 'logprobs': None}, id='run-56bd0842-7465-496b-a4a9-b4e1545dd1da-0', usage_metadata={'input_tokens': 14, 'output_tokens': 13, 'total_tokens': 27, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}})"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"chain = prompt | model_bind_with_retry\n",
"chain.invoke(\"programming\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"or you can use `.with_retry()` with Runnables."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"AIMessage(content='Why do bears have hairy coats?\\n\\nBecause they look silly in sweaters!', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 15, 'prompt_tokens': 14, 'total_tokens': 29, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_0aa8d3e20b', 'finish_reason': 'stop', 'logprobs': None}, id='run-a73176c2-8550-40bf-8284-acb56a302c07-0', usage_metadata={'input_tokens': 14, 'output_tokens': 15, 'total_tokens': 29, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}})"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"chain = prompt | model.with_retry(retry_if_exception_type=(InternalServerError,), stop_after_attempt=3, wait_exponential_jitter=True)\n",
"chain.invoke(\"bear\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.5"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Loading