diff --git a/04-MODEL/01-Chat-Models.ipynb b/04-MODEL/01-Chat-Models.ipynb new file mode 100644 index 000000000..d65c15a61 --- /dev/null +++ b/04-MODEL/01-Chat-Models.ipynb @@ -0,0 +1,824 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Chat Models\n", + "\n", + "- Author: [PangPangGod](https://github.com/pangpanggod)\n", + "- Design: []()\n", + "- Peer Review: []()\n", + "- This is a part of [LangChain Open Tutorial](https://github.com/LangChain-OpenTutorial/LangChain-OpenTutorial)\n", + "\n", + "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/langchain-ai/langchain-academy/blob/main/module-4/sub-graph.ipynb) [![Open in LangChain Academy](https://cdn.prod.website-files.com/65b8cd72835ceeacd4449a53/66e9eba12c7b7688aa3dbb5e_LCA-badge-green.svg)](https://academy.langchain.com/courses/take/intro-to-langgraph/lessons/58239937-lesson-2-sub-graphs)\n", + "\n", + "## Overview\n", + "\n", + "This tutorial covers an explanation of various Chat models (OpenAI, Anthropic, etc.) along with brief usage examples.\n", + "\n", + "### Table of Contents\n", + "\n", + "- [Overview](#overview)\n", + "- [Environment Setup](#environment-setup)\n", + "- [OpenAI](#openai)\n", + "- [Anthropic](#anthropic)\n", + "- [Perplexity](#perplexity)\n", + "- [Together AI](#together-ai)\n", + "- [Cohere](#cohere)\n", + "- [Upstage](#upstage)\n", + "- [Open LLM Leaderboard](#open-llm-leaderboard)\n", + "- [Vellum LLM Leaderboard](#vellum-llm-leaderboard)\n", + "\n", + "### References\n", + "\n", + "- [OpenAI Model Specifications](https://platform.openai.com/docs/models)\n", + "- [LangChain ChatOpenAI API reference](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.base.ChatOpenAI.html)\n", + "\n", + "- [Anthropic Model Specifications](https://docs.anthropic.com/en/docs/about-claude/models)\n", + "- [LangChain ChatAnthropic API reference](https://python.langchain.com/api_reference/anthropic/chat_models/langchain_anthropic.chat_models.ChatAnthropic.html)\n", + "\n", + "- [Perplexity Model Cards](https://docs.perplexity.ai/guides/model-cards)\n", + "- [LangChain ChatPerplexity API reference](https://api.python.langchain.com/en/latest/community/chat_models/langchain_community.chat_models.perplexity.ChatPerplexity.html)\n", + "\n", + "- [Together AI Model Specifications](https://api.together.xyz/models)\n", + "- [LangChain ChatTogether API reference](https://python.langchain.com/api_reference/together/chat_models/langchain_together.chat_models.ChatTogether.html)\n", + "\n", + "- [Cohere Model Specifications](https://docs.cohere.com/docs/models)\n", + "- [LangChain ChatCohere API reference](https://python.langchain.com/api_reference/cohere/chat_models/langchain_cohere.chat_models.ChatCohere.html)\n", + "\n", + "- [Upstage Model Specifications](https://console.upstage.ai/docs/capabilities/chat)\n", + "- [LangChain ChatUpstage API reference](https://python.langchain.com/api_reference/upstage/chat_models/langchain_upstage.chat_models.ChatUpstage.html)\n", + "\n", + "- [HuggingFace Open LLM Leaderboard](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)\n", + "- [Vellum LLM Leaderboard](https://www.vellum.ai/llm-leaderboard)\n", + "----" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Environment Setup\n", + "\n", + "Set up the environment. You may refer to [Environment Setup](https://wikidocs.net/257836) for more details.\n", + "\n", + "**[Note]**\n", + "- `langchain-opentutorial` is a package that provides a set of easy-to-use environment setup, useful functions and utilities for tutorials.\n", + "- You can checkout the [`langchain-opentutorial`](https://github.com/LangChain-OpenTutorial/langchain-opentutorial-pypi) for more details." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "\n", + "[notice] A new release of pip is available: 24.0 -> 24.3.1\n", + "[notice] To update, run: python.exe -m pip install --upgrade pip\n" + ] + } + ], + "source": [ + "%%capture --no-stderr\n", + "%pip install langchain-opentutorial" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Install required packages\n", + "from langchain_opentutorial import package\n", + "\n", + "package.install(\n", + " [\n", + " \"langsmith\",\n", + " \"langchain\",\n", + " \"langchain_openai\",\n", + " \"langchain_anthropic\",\n", + " \"langchain_community\",\n", + " \"langchain_together\",\n", + " \"langchain_cohere\",\n", + " \"langchain_upstage\",\n", + " ],\n", + " verbose=False,\n", + " upgrade=False,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you want to get automated tracing of your model calls you can also set your LangSmith API key by uncommenting below code:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Set environment variables\n", + "\n", + "# from langchain_opentutorial import set_env\n", + "\n", + "# set_env(\n", + "# {\n", + "# \"LANGCHAIN_API_KEY\": \"\",\n", + "# \"LANGCHAIN_TRACING_V2\": \"true\",\n", + "# \"LANGCHAIN_ENDPOINT\": \"https://api.smith.langchain.com\",\n", + "# \"LANGCHAIN_PROJECT\": \"01-Chat-Models\",\n", + "# }\n", + "# )" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## OpenAI\n", + "\n", + "OpenAI is an AI research and deployment company based in San Francisco, dedicated to ensuring that artificial general intelligence benefits all of humanity. Models include the GPT series of language models, such as `GPT-4` and `GPT-4o`, as well as the `DALL·E` series for image generation.\n", + "\n", + "### Model Description\n", + "\n", + "| Model | Description | Context Length | Max Output Tokens | Training Data |\n", + "|--------------------|---------------------------------------------------------------------------------------------|------------------|-------------------|----------------------|\n", + "| gpt-4o | A versatile, high-intelligence flagship model, cheaper and faster than GPT-4 Turbo. | 128,000 tokens | 16,384 tokens | Up to October 2023 |\n", + "| chatgpt-4o-latest | The continuously updated version of GPT-4o used in ChatGPT. | 128,000 tokens | 16,384 tokens | Continuously updated |\n", + "| gpt-4o-mini | A smaller, faster model with better performance than GPT-3.5 Turbo. | 128,000 tokens | 16,384 tokens | Up to October 2023 |\n", + "| gpt-4-turbo | The latest GPT-4 Turbo model with vision, JSON mode, and function calling capabilities. | 128,000 tokens | 4,096 tokens | Up to December 2023 |\n", + "| gpt-4o-realtime | A beta model optimized for real-time API use with audio and text inputs. | 128,000 tokens | 4,096 tokens | Up to October 2023 |\n", + "| gpt-4o-audio | A beta model capable of handling audio inputs and outputs via the Chat Completions API. | 128,000 tokens | 16,384 tokens | Up to October 2023 |\n", + "| gpt-3.5-turbo | Optimized for chat and non-chat tasks with natural language and code generation capabilities.| 16,385 tokens | 4,096 tokens | Up to September 2021 |\n", + "\n", + "OpenAI offers a variety of model options. \n", + "A detailed specification of these models can be found at the following link: \n", + "[OpenAI Model Specifications](https://platform.openai.com/docs/models)\n", + "\n", + "### Basic Model Options\n", + "\n", + "The basic API options are as follows:\n", + "\n", + "- `model_name` : `str` \n", + " This option allows you to select the applicable model. can be aliased as `model`.\n", + "\n", + "- `temperature` : `float` = 0.7 \n", + " This option sets the sampling `temperature`. Values can range between 0 and 2. Higher values (e.g., 0.8) make the output more random, while lower values (e.g., 0.2) make the output more focused and deterministic.\n", + "\n", + "- `max_tokens` : `int` | `None` = `None` \n", + " Specifies the maximum number of tokens to generate in the chat completion. This option controls the length of text the model can generate in one instance.\n", + "\n", + "Detailed information about the available API options can be found [here](https://python.langchain.com/api_reference/openai/chat_models/langchain_openai.chat_models.base.ChatOpenAI.html)." + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.environ.get(\"OPENAI_API_KEY\"):\n", + " os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter API key for OpenAI: \")\n", + "\n", + "from langchain_openai import ChatOpenAI\n", + "\n", + "# Create a ChatOpenAI object\n", + "model = ChatOpenAI(\n", + " model_name=\"gpt-4o-mini\", # Model name (can be aliased as 'model')\n", + " temperature=0,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The code provided assumes that your `OPENAI_API_KEY` is set in your environment variables. If you would like to manually specify your API key and also choose a different model, you can use the following code:\n", + "\n", + "``` python\n", + "model = ChatOpenAI(temperature=0, api_key=\"YOUR_API_KEY_HERE\", model=\"gpt-4o-mini\")\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Why do programmers prefer dark mode?\n", + "\n", + "Because light attracts bugs!" + ] + } + ], + "source": [ + "query = \"Tell me one joke about Computer Science\"\n", + "\n", + "# Stream the response instead of invoking it directly\n", + "response = model.stream(query)\n", + "\n", + "# Print the streamed response token by token\n", + "for token in response:\n", + " print(token.content, end=\"\", flush=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Anthropic\n", + "\n", + "Anthropic is an AI safety and research company based in San Francisco, dedicated to building reliable, interpretable, and steerable AI systems. \n", + "Their primary offering is the `Claude` family of large language models, including `Claude 3.5 Sonnet` and `Claude 3.5 Haiku`, designed for various applications such as reasoning, coding, and multilingual tasks.\n", + "\n", + "### Model Description\n", + "\n", + "| Model | Description | Context Length | Max Output Tokens | Training Data |\n", + "|--------------------|--------------------------------------------------------|------------------|-------------------|----------------------|\n", + "| Claude 3.5 Sonnet | The most intelligent model in the Claude family. | 200,000 tokens | 8,192 tokens | Up to April 2024 |\n", + "| Claude 3.5 Haiku | The fastest model with blazing speed. | 200,000 tokens | 8,192 tokens | Up to July 2024 |\n", + "| Claude 3 Opus | Powerful model for highly complex tasks. | 200,000 tokens | 4,096 tokens | Up to August 2023 |\n", + "| Claude 3 Sonnet | Balanced model offering strong utility for scaled deployments. | 200,000 tokens | 4,096 tokens | Up to August 2023 |\n", + "| Claude 3 Haiku | Fastest and most compact model for near-instant responsiveness. | 200,000 tokens | 4,096 tokens | Up to August 2023 |\n", + "\n", + "A detailed specification of these models can be found at the following link: \n", + "[Anthropic Model Specifications](https://docs.anthropic.com/en/docs/about-claude/models)\n", + "\n", + "### Basic Model Options\n", + "\n", + "The basic API options are as follows:\n", + "\n", + "- `model_name` : `str` \n", + " This option allows you to select the applicable model. can be aliased as `model`.\n", + "\n", + "- `temperature` : `float` = 0.7 \n", + " This option sets the sampling `temperature`. Values can range between 0 and 2. Higher values (e.g., 0.8) make the output more random, while lower values (e.g., 0.2) make the output more focused and deterministic.\n", + "\n", + "- `max_tokens` : `int` | `None` = `None` \n", + " Specifies the maximum number of tokens to generate in the chat completion. This option controls the length of text the model can generate in one instance.\n", + "\n", + "Detailed information about the available API options can be found [here](https://python.langchain.com/api_reference/anthropic/chat_models/langchain_anthropic.chat_models.ChatAnthropic.html)." + ] + }, + { + "cell_type": "code", + "execution_count": 27, + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.environ.get(\"ANTHROPIC_API_KEY\"):\n", + " os.environ[\"ANTHROPIC_API_KEY\"] = getpass.getpass(\"Enter API key for Anthropic: \")\n", + "\n", + "from langchain_anthropic import ChatAnthropic\n", + "\n", + "# Create a ChatAnthropic object\n", + "model = ChatAnthropic(\n", + " model_name=\"claude-3-5-haiku-latest\", # Model name (can be aliased as 'model')\n", + " temperature=0,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The code provided assumes that your `ANTHROPIC_API_KEY` is set in your environment variables. If you would like to manually specify your API key and also choose a different model, you can use the following code:\n", + "\n", + "``` python\n", + "model = ChatAnthropic(temperature=0, api_key=\"YOUR_API_KEY_HERE\", model=\"claude-3-5-haiku-latest\")\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Here's a classic computer science joke:\n", + "\n", + "Why do programmers prefer dark mode?\n", + "\n", + "Because light attracts bugs! 😄\n", + "\n", + "(This is a play on words, referencing both computer \"bugs\" (errors) and actual insects being attracted to light.)" + ] + } + ], + "source": [ + "query = \"Tell me one joke about Computer Science\"\n", + "\n", + "# Stream the response instead of invoking it directly\n", + "response = model.stream(query)\n", + "\n", + "# Print the streamed response token by token\n", + "for token in response:\n", + " print(token.content, end=\"\", flush=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Perplexity\n", + "\n", + "Perplexity AI is a conversational search engine that integrates advanced large language models (LLMs) to provide direct answers to user queries with source citations. \n", + "Their platform supports the following models, optimized for chat completion tasks, with extended context capabilities:\n", + "\n", + "### Supported Models\n", + "\n", + "| Model | Parameter Count | Context Length | Model Type |\n", + "|------------------------------------|-----------------|------------------|------------------|\n", + "| llama-3.1-sonar-small-128k-online | 8B | 127,072 tokens | Chat Completion |\n", + "| llama-3.1-sonar-large-128k-online | 70B | 127,072 tokens | Chat Completion |\n", + "| llama-3.1-sonar-huge-128k-online | 405B | 127,072 tokens | Chat Completion |\n", + "\n", + "A detailed specification of these models can be found at the following link: \n", + "[Perplexity Model Cards](https://docs.perplexity.ai/guides/model-cards)\n", + "\n", + "### Basic Model Options\n", + "\n", + "The basic API options are as follows:\n", + "\n", + "- `model` : `str` \n", + " Specifies the language model to use (e.g., `\"llama-3.1-sonar-small-128k-online\"`). This determines the performance and capabilities of the response.\n", + "\n", + "- `temperature` : `float` = 0.7 \n", + " Controls the randomness of responses. A value of 0 is deterministic, while 1 allows for the most random outputs.\n", + "\n", + "- `max_tokens` : `int` | `None` = `None` \n", + " Specifies the maximum number of tokens to generate in the chat completion. This option controls the length of text the model can generate in one instance.\n", + "\n", + "For more detailed information about the available API options, visit [Perplexity API Reference](https://api.python.langchain.com/en/latest/community/chat_models/langchain_community.chat_models.perplexity.ChatPerplexity.html)." + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.environ.get(\"PPLX_API_KEY\"):\n", + " os.environ[\"PPLX_API_KEY\"] = getpass.getpass(\"Enter API key for Perplexity: \")\n", + "\n", + "from langchain_community.chat_models import ChatPerplexity\n", + "\n", + "# Create a ChatPerplexity object\n", + "model = ChatPerplexity(\n", + " model=\"llama-3.1-sonar-large-128k-online\",\n", + " temperature=0,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The code provided assumes that your `PPLX_API_KEY` is set in your environment variables. If you would like to manually specify your API key and also choose a different model, you can use the following code:\n", + "\n", + "``` python\n", + "model = ChatPerplexity(temperature=0, pplx_api_key=\"YOUR_API_KEY_HERE\", model=\"llama-3.1-sonar-large-128k-online\")\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The 2024 Nobel Prize in Literature was awarded to South Korean author Han Kang. She was recognized \"for her intense poetic prose that confronts historical traumas and exposes the fragility of human life\"[1][3][5].\n", + "\n", + "[1] https://now.uiowa.edu/news/2024/10/international-writing-program-participant-wins-2024-nobel-prize-literature\n", + "[2] https://www.nobelprize.org/prizes/literature/2024/prize-announcement/\n", + "[3] https://www.weforum.org/stories/2024/10/nobel-prize-winners-2024/\n", + "[4] https://www.nobelprize.org/prizes/literature/2024/han/facts/\n", + "[5] https://en.wikipedia.org/wiki/2024_Nobel_Prize_in_Literature\n" + ] + } + ], + "source": [ + "# print out response\n", + "response = model.invoke(\"Who won the 2024 Nobel Prize in Literature?\")\n", + "\n", + "print(response.content)\n", + "print()\n", + "# ChatPerplexity stores the sources of knowledge and information in the additional_kwargs[\"citations\"] field.\n", + "for i, citation in enumerate(response.additional_kwargs[\"citations\"]):\n", + " print(f\"[{i+1}] {citation}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Together AI\n", + "\n", + "Together AI is a San Francisco-based company specializing in decentralized cloud services for training and deploying generative AI models. Founded in 2022, they offer a cloud platform that enables researchers, developers, and organizations to train, fine-tune, and run AI models efficiently at scale. Their services include GPU clusters featuring NVIDIA GB200, H200, and H100, and they contribute to open-source AI research, models, and datasets to advance the field.\n", + "\n", + "### Together Inference \n", + "- Offers the fastest inference stack in the industry, up to 4x faster than vLLM.\n", + "- Operates at 11x lower cost compared to GPT-4 when using Llama-3 70B.\n", + "- Features auto-scaling capabilities that adjust capacity based on API request volume.\n", + "\n", + "### Together Custom Models \n", + "- Supports customized AI model training and fine-tuning.\n", + "- Incorporates cutting-edge optimization technologies like FlashAttention-3.\n", + "- Ensures full ownership of trained models.\n", + "\n", + "### Performance Optimization \n", + "- Proprietary inference engine integrating FlashAttention-3 kernels and custom kernels.\n", + "- Implements speculative decoding algorithms like Medusa and SpecExec.\n", + "- Employs unique quantization techniques for maximum accuracy and performance.\n", + "\n", + "### Security and Privacy\n", + "- User data is not used for training new models without explicit consent.\n", + "- Provides users with complete control over data storage.\n", + "\n", + "### Supported Models \n", + "- Supports over 200 open-source models, including Google Gemma, Meta's Llama 3.3, Qwen2.5, and Mistral/Mixtral from Mistral AI.\n", + "- Enables multimodal AI models to process various types of data.\n", + "- A detailed specification of these models can be found at the following link: \n", + " [Together AI Models](https://api.together.xyz/models)\n", + "\n", + "### Basic Model Options\n", + "\n", + "The basic API options are as follows:\n", + "\n", + "- `model_name` : `str` \n", + " This option allows you to select the applicable model. can be aliased as `model`.\n", + "\n", + "- `temperature` : `float` = 0.7 \n", + " This option sets the sampling `temperature`. Values can range between 0 and 2. Higher values (e.g., 0.8) make the output more random, while lower values (e.g., 0.2) make the output more focused and deterministic.\n", + "\n", + "- `max_tokens` : `int` | `None` = `None` \n", + " Specifies the maximum number of tokens to generate in the chat completion. This option controls the length of text the model can generate in one instance.\n", + "\n", + "Detailed information about the available API options can be found [here](https://python.langchain.com/api_reference/together/chat_models/langchain_together.chat_models.ChatTogether.html).\n" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.environ.get(\"TOGETHER_API_KEY\"):\n", + " os.environ[\"TOGETHER_API_KEY\"] = getpass.getpass(\"Enter API key for Together AI: \")\n", + "\n", + "from langchain_together.chat_models import ChatTogether\n", + "\n", + "# Create a ChatPerplexity object\n", + "model = ChatTogether(\n", + " model=\"meta-llama/Llama-3.3-70B-Instruct-Turbo\",\n", + " temperature=0,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The code provided assumes that your `TOGETHER_API_KEY` is set in your environment variables. If you would like to manually specify your API key and also choose a different model, you can use the following code:\n", + "\n", + "``` python\n", + "model = ChatTogether(temperature=0, api_key=\"YOUR_API_KEY_HERE\", model=\"meta-llama/Llama-3.3-70B-Instruct-Turbo\")\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The biggest animal on the planet is the **blue whale** (Balaenoptera musculus). On average, an adult blue whale can grow up to:\n", + "\n", + "* 82 feet (25 meters) in length\n", + "* Weigh around 150-170 tons (136,000-152,000 kilograms)\n", + "\n", + "To put that into perspective, that's equivalent to the weight of about 50 elephants or a large building. Blue whales are not only the largest animals on Earth, but they are also the largest known animals to have ever existed on our planet.\n", + "\n", + "These massive creatures can be found in all of the world's oceans, from the Arctic to the Antarctic, and are known for their distinctive blue-gray color and massive size. Despite their enormous size, blue whales are incredibly streamlined and can swim at speeds of up to 30 miles (48 kilometers) per hour.\n", + "\n", + "It's worth noting that while blue whales are the largest animals, there are other large animals on the planet, such as fin whales, humpback whales, and African elephants, which are also impressive in their own right. However, the blue whale remains the largest of them all." + ] + } + ], + "source": [ + "query = \"What is the biggest Animal on The planet?\"\n", + "\n", + "# Stream the response instead of invoking it directly\n", + "response = model.stream(query)\n", + "\n", + "# Print the streamed response token by token\n", + "for token in response:\n", + " print(token.content, end=\"\", flush=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Cohere\n", + "\n", + "Cohere is a leading AI company specializing in enterprise AI solutions, enabling businesses to easily adopt and utilize AI technologies through advanced large language models (LLMs). \n", + "Their platform is tailored for natural language processing tasks, providing scalable and efficient tools for real-world applications.\n", + "\n", + "- **Founded:** 2020 \n", + "- **Key Investors:** Inovia Capital, NVIDIA, Oracle, Salesforce Ventures \n", + "- **Series C Funding:** Raised $270 million \n", + "- **Mission:** To provide an AI platform tailored for enterprise needs \n", + "\n", + "### Supported Models \n", + "| Model | Description | Context Length | Max Output Tokens |\n", + "|---------------------|-------------------------------------------------------------------------------------------------------|------------------|-------------------|\n", + "| command-r-7b | A small, fast update of the Command R+ model, excelling at RAG, tool use, agents, and multi-step reasoning. | 128,000 tokens | 4,000 tokens |\n", + "| command-r-plus | An instruction-following conversational model excelling in RAG, tool use, and multi-step reasoning. | 128,000 tokens | 4,000 tokens |\n", + "| command-r | A conversational model designed for high-quality language tasks and complex workflows like RAG and coding. | 128,000 tokens | 4,000 tokens |\n", + "| command | An instruction-following conversational model for high-quality tasks, more reliable and longer context than base models. | 4,000 tokens | 4,000 tokens |\n", + "| command-nightly | Nightly version of the command model with experimental and regularly updated features. Not for production use. | 128,000 tokens | 4,000 tokens |\n", + "| command-light | A smaller, faster version of the command model, maintaining near-equal capability with improved speed. | 4,000 tokens | 4,000 tokens |\n", + "| command-light-nightly | Nightly version of the command-light model, experimental and regularly updated. Not for production use. | 4,000 tokens | 4,000 tokens |\n", + "| c4ai-aya-expanse-8b | A highly performant 8B multilingual model serving 23 languages, designed for superior monolingual performance. | 8,000 tokens | 4,000 tokens |\n", + "| c4ai-aya-expanse-32b| A highly performant 32B multilingual model serving 23 languages, designed for superior monolingual performance. | 128,000 tokens | 4,000 tokens |\n", + "\n", + "- A detailed specification of cohere's models can be found at the following link: \n", + " [Cohere Models](https://docs.cohere.com/docs/models)\n", + "\n", + "### Basic Model Options\n", + "\n", + "The basic API options are as follows:\n", + "\n", + "- `model_name` : `str` \n", + " This option allows you to select the applicable model. can be aliased as `model`.\n", + "\n", + "- `temperature` : `float` = 0.7 \n", + " This option sets the sampling `temperature`. Values can range between 0 and 2. Higher values (e.g., 0.8) make the output more random, while lower values (e.g., 0.2) make the output more focused and deterministic.\n", + "\n", + "- `max_tokens` : `int` | `None` = `None` \n", + " Specifies the maximum number of tokens to generate in the chat completion. This option controls the length of text the model can generate in one instance.\n", + "\n", + "Detailed information about the available API options can be found [here](https://python.langchain.com/api_reference/cohere/chat_models/langchain_cohere.chat_models.ChatCohere.html)." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.environ.get(\"COHERE_API_KEY\"):\n", + " os.environ[\"COHERE_API_KEY\"] = getpass.getpass(\"Enter API key for Cohere: \")\n", + "\n", + "from langchain_cohere import ChatCohere\n", + "\n", + "# Create a ChatCohere object\n", + "model = ChatCohere(\n", + " model_name=\"command-r7b-12-2024\", # can be alisaed as 'model'\n", + " temperature=0,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The code provided assumes that your `COHERE_API_KEY` is set in your environment variables. If you would like to manually specify your API key and also choose a different model, you can use the following code:\n", + "\n", + "``` python\n", + "model = ChatCohere(temperature=0, cohere_api_key=\"YOUR_API_KEY_HERE\", model=\"command-r7b-12-2024\")\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Earth, Wind & Fire, the iconic American band, has had numerous hits throughout their illustrious career, spanning multiple genres such as R&B, soul, funk, jazz, disco, pop, and Afro pop. Here are some of their biggest hit songs:\n", + "\n", + "- \"September\": Released in 1978, \"September\" is arguably the band's most recognizable and popular song. It topped the US Billboard Hot R&B Songs chart and peaked at number eight on the Billboard Hot 100. The song's catchy melody and uplifting lyrics have made it a timeless classic, often played at celebrations and events.\n", + "\n", + "- \"Shining Star\": This was Earth, Wind & Fire's first single to reach number one on the Billboard Hot 100 in 1975. The song won a Grammy Award for Best R&B Performance by a Duo or Group with Vocals. \"Shining Star\" is known for its powerful vocals and inspiring message.\n", + "\n", + "- \"Boogie Wonderland\": A collaboration with The Emotions, this disco-funk track became a massive hit in 1979. It reached number six on the Billboard Hot 100 and is remembered for its infectious groove and energetic vocals.\n", + "\n", + "- \"After the Love Has Gone\": This ballad showcases the band's versatility and won a Grammy Award for Best R&B Song in 1980. It peaked at number two on the Billboard Hot 100 and is considered one of the band's most emotionally powerful songs.\n", + "\n", + "- \"Let's Groove\": Released in 1981, this funk and disco-influenced song became a commercial success, reaching number three on the Billboard Hot 100. \"Let's Groove\" is known for its catchy rhythm and danceable vibe.\n", + "\n", + "- \"Fantasy\": From their 1977 album \"All 'N All,\" \"Fantasy\" is another fan favorite. It peaked at number twelve on the Billboard Hot 100 and is characterized by its dreamy atmosphere and smooth vocals.\n", + "\n", + "- \"Got to Get You into My Life\": A cover of the Beatles song, Earth, Wind & Fire's version became a hit in its own right in 1978. It reached number nine on the Billboard Hot 100 and won a Grammy Award for Best Instrumental Arrangement Accompanying Vocalist(s).\n", + "\n", + "These songs are just a selection of Earth, Wind & Fire's extensive catalog of hits, showcasing their incredible musical range and enduring popularity." + ] + } + ], + "source": [ + "query = \"What are the Biggest hit songs of Earth, Wind & Fire?\"\n", + "\n", + "# Stream the response instead of invoking it directly\n", + "response = model.stream(query)\n", + "\n", + "# Print the streamed response token by token\n", + "for token in response:\n", + " print(token.content, end=\"\", flush=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Upstage\n", + "\n", + "Upstage is a South Korean startup specializing in artificial intelligence (AI) technologies, particularly large language models (LLMs) and document AI. \n", + "Their solutions are designed to deliver cost-efficient, high-performance AI capabilities across various industries.\n", + "\n", + "- **Key Product: Solar LLM** \n", + " Upstage's flagship large language model known for its speed, efficiency, and scalability. Utilizing Depth-Up Scaling (DUS) technology, Solar LLM maximizes performance and is seamlessly integrated into platforms like Amazon SageMaker JumpStart via API.\n", + "\n", + "- **Document AI Pack** \n", + " A comprehensive document processing solution powered by advanced OCR technology. This tool accurately extracts and digitizes essential information from complex documents.\n", + "\n", + "- **AskUp Seargest** \n", + " An upgraded version of the AskUp chatbot, offering personalized search and recommendation services, building upon the integration with ChatGPT.\n", + "\n", + "Upstage provides cutting-edge tools for enterprises to enhance automation, streamline workflows, and deliver AI-powered insights.\n", + "\n", + "### Supported Models \n", + "\n", + "| Model | Description | Context Length | Training Data |\n", + "|--------------|-------------------------------------------------------------------------------------------------------|------------------|----------------------|\n", + "| solar-pro | An enterprise-grade LLM designed for exceptional instruction-following and processing structured formats like HTML and Markdown. It excels in multilingual performance in English, Korean, and Japanese, with domain expertise in Finance, Healthcare, and Legal. | 32,768 tokens | Up to May 2024 |\n", + "| solar-mini | A compact 10.7B parameter LLM for businesses seeking AI solutions. Solar Mini is an instruction-following conversational model supporting English, Korean, and Japanese. It excels at fine-tuning, providing seamless integration and high-quality language processing. | 32,768 tokens | Up to December 2023 |\n", + "| solar-mini-ja| Solar Mini with enhanced capabilities in Japanese language processing. It is an instruction-following conversational model supporting Japanese as well as English and Korean. | 32,768 tokens | Up to December 2023 |\n", + "\n", + "- A detailed specification of Upstage's models can be found at the following link: \n", + " [Upstage Models](https://console.upstage.ai/docs/capabilities/chat)\n", + "\n", + "### Basic Model Options\n", + "\n", + "The basic API options are as follows:\n", + "\n", + "- `model_name` : `str` \n", + " This option allows you to select the applicable model. can be aliased as `model`.\n", + "\n", + "- `temperature` : `float` = 0.7 \n", + " This option sets the sampling `temperature`. Values can range between 0 and 2. Higher values (e.g., 0.8) make the output more random, while lower values (e.g., 0.2) make the output more focused and deterministic.\n", + "\n", + "- `max_tokens` : `int` | `None` = `None` \n", + " Specifies the maximum number of tokens to generate in the chat completion. This option controls the length of text the model can generate in one instance.\n", + "\n", + "Detailed information about the available API options can be found [here](https://python.langchain.com/api_reference/upstage/chat_models/langchain_upstage.chat_models.ChatUpstage.html)." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "import getpass\n", + "import os\n", + "\n", + "if not os.environ.get(\"UPSTAGE_API_KEY\"):\n", + " os.environ[\"UPSTAGE_API_KEY\"] = getpass.getpass(\"Enter API key for Upstage: \")\n", + "\n", + "from langchain_upstage import ChatUpstage \n", + "\n", + "# Create a ChatUpstage object\n", + "model = ChatUpstage(\n", + " model_name=\"solar-mini\", # can be alisaed as 'model'\n", + " temperature=0,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The code provided assumes that your `UPSTAGE_API_KEY` is set in your environment variables. If you would like to manually specify your API key and also choose a different model, you can use the following code:\n", + "\n", + "``` python\n", + "model = ChatUpstage(temperature=0, upstage_api_key=\"YOUR_API_KEY_HERE\", model=\"solar-mini\")\n", + "```" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Why did the computer scientist go broke?\n", + "\n", + "He invested in a company that was developing a new programming language, but it turned out to be a \"dead end\" language." + ] + } + ], + "source": [ + "query = \"Tell me one joke about Computer Science\"\n", + "\n", + "# Stream the response instead of invoking it directly\n", + "response = model.stream(query)\n", + "\n", + "# Print the streamed response token by token\n", + "for token in response:\n", + " print(token.content, end=\"\", flush=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Open LLM Leaderboard\n", + "\n", + "The Open LLM Leaderboard is a community-driven platform hosted by Hugging Face that tracks, ranks, and evaluates open-source large language models (LLMs) and chatbots. It provides datasets, score results, queries, and collections for various models, enabling users to compare performance across different benchmarks.\n", + "\n", + "By utilizing the Open LLM Leaderboard, you can identify high-performing models suitable for integration with platforms like Hugging Face and Ollama, facilitating seamless deployment and interaction with LLMs.\n", + "\n", + "For more information, visit the [Open LLM Leaderboard](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard).\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Vellum LLM Leaderboard\n", + "\n", + "The Vellum LLM Leaderboard is a platform that compares leading commercial and open-source large language models (LLMs) based on capabilities, pricing, and context window sizes. It provides insights into model performance across various tasks, including multitask reasoning, coding, and mathematics, assisting users in selecting the most suitable LLM for their specific needs.\n", + "\n", + "By utilizing the Vellum LLM Leaderboard, you can identify high-performing models suitable for integration with platforms like Hugging Face and Ollama, facilitating seamless deployment and interaction with LLMs.\n", + "\n", + "For more information, visit the [Vellum LLM Leaderboard](https://www.vellum.ai/llm-leaderboard)." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.10" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +}