SYNCHRONOUS COMMUNICATION MECHANISM BETWEEN AGENTS

● In technical terms an AI Agent is a software entity designed to perform tasks

autonomously or semi-autonomously on behalf of a user or another program. These
agents leverage artificial intelligence to make decisions, take actions, and interact
with their environment or other systems.

● The insight of multi-agent frameworks is that they simulate a team, with a mix of
generalists and specialists, that collaborate to accomplish some tasks.

● AutoGen, Crew AI, and LangGraph are leading popular frameworks that provide
tools for creating multi-agent solutions.

THE BUILDING BLOCKS OF MULTI-AGENT SYSTEMS

● Large Language Models (LLMs)

At the heart of modern multi-agent frameworks are Large Language
Models—powerful AI systems adept at understanding and generating human
language.

● Agents
Agents are autonomous entities programmed to perform specific tasks, make
decisions, and collaborate towards a shared objective.

● Tools
Tools represent specialized functions or skills that agents leverage to execute tasks.

● Processes or Flows
Processes (or flows) define how tasks should be orchestrated within a multi-agent
system, ensuring efficient task distribution and alignment with objectives.

COMPONENTS OF MULTI-AGENTS SYSTEM

LEADING MULTI-AGENT FRAMEWORKS

The choice of framework is crucial in determining the system’s scalability, autonomy,

and the level of control developers have. Below, we compare the three prominent
frameworks that we have explored, each with its unique features.

1. AutoGen
AutoGen specializes in conversational agents, providing conversation as a high-level
abstraction over multi-agent collaboration.

Key Features

● Conversational Engagement: Agents within AutoGen can engage in dialogue,

sharing messages and insights to accomplish tasks collectively.

● Customization Through Integration: Allows the integration of various components like

LLMs or human inputs, offering some degree of customizability.

AUTOGEN WORKFLOW DIAGRAM

2. CrewAI

CrewAI combines AutoGen’s autonomy with a structured, role-playing approach,

facilitating sophisticated agent interactions.

Key Features
 ● Role-Based Agent Design: Introduces customizable agents with predefined roles and
goals, supplemented by toolsets for enhanced capabilities.

● Autonomous Inter-Agent Delegation: Agents can autonomously delegate and consult

tasks among themselves, streamlining problem-solving and task management.

CREWAI WORKFLOW DIAGRAM

3. LangGraph

LangGraph is not as much as a multi-agent-framework, rather than a graph

framework that allows developers to define complex inter-agent interactions as
graphs (with fine-grained control over agent interactions).

Key Features

● Stateful Multi-Actor Applications: Supports applications involving multiple interacting

agents, maintaining state throughout the process.

● Cyclical Computation Support: Unique in its ability to introduce cycles within LLM
applications, essential for simulating agent-like behaviors

4. Comparative table of different multi-agents frameworks

Feature AutoGen Crew AI LangGraph

Type of Conversational Agents Role-Playing Agents Graph-Based Agents

Framework
Autonomy Highly Autonomous Highly Autonomous Conditionally
Autonomous

Collaboration Centralized group chat Autonomous agents with Condition-based,

roles and goals cycling graphs

Execution Managed by a Dynamic delegation, but All agents perform

dedicated agent possible to define functions
hierarchical processes

Use cases Experimentation, Development to Detailed control

prototyping, use cases production scenarios
that beget
conversational patterns

COMPARATIVE TABLE

LANGGRAPH: MULTI-AGENT WORKFLOWS

Each agent can have its own prompt, LLM, tools, and other custom code to best
collaborate with the other agents.

In this approach, each agent is a node in the graph, and their connections are
represented as an edge. The control flow is managed by edges, and they
communicate by adding to the graph's state.

Note: a very related concept here is the concept of state machines, which we
explicitly called out as a category of cognitive architectures.

There are two main considerations when thinking about different multi-agent
workflows:

1. What are the multiple independent agents?

2. How are those agents connected?

1. MULTI AGENT COLLABORATION

The different agents collaborate on a shared scratchpad of messages. This means

that all the work either of them do is visible to the other.
 COLLABORATION ARCHITECTURE

The main thing controlling the state transitions is the rule-based router.

2. AGENT SUPERVISOR

Multiple agents are connected, but they have their own independent scratchpads,
and then their final responses are appended to a global scratchpad.

An agent supervisor is responsible for routing to individual agents. It can also be

thought of an agent whose tools are other agents.

SUPERVISOR ARCHITECTURE
3. HIERARCHICAL AGENT TEAMS

The agents in the nodes are actually other langgraph objects themselves.
It is hierarchical teams because the subagents can in a way be thought of as teams.

HIERARCHICAL ARCHITECTURE

REACT FRAMEWORK

Agents use LLMs as general-purpose problem-solvers, connecting them with

external resources to answer questions or accomplish tasks.

LLM agents typically have the following main steps:

Propose action: the LLM generates text to respond directly to a user or to pass to a
function.
Execute action: your code invokes other software to do things like query a database
or call an API.
Observe: react to the response of the tool call by either calling another function or
responding to the user.

The ReAct agent is a great prototypical design for this, as it prompts the language
model using a repeated thought, act, observation loop.
This takes advantage of Chain-of-thought (CoT) prompting to make a single action
choice per step.

REACT PARADIGM

Downsides:

● It requires an LLM call for each tool invocation.

● The LLM only plans for 1 sub-problem at a time. This may lead to sub-optimal
trajectories, since it isn't forced to "reason" about the whole task.

One way to overcome these two shortcomings is through an explicit planning step.

PLANNING AGENTS

1. Plan and Execute

Based loosely on Wang, et. al.’s paper on Plan-and-Solve Prompting, and Yohei
Nakajima’s BabyAGI project, this simple architecture is emblematic of the planning
agent architecture. It consists of two basic components:

A planner, which prompts an LLM to generate a multi-step plan to complete a large

task.
Executor(s), which accept the user query and a step in the plan and invoke 1 or
more tools to complete that task.

Once execution is completed, the agent is called again with a re-planning prompt,
letting it decide whether to finish with a response or whether to generate a follow-up
plan (if the first plan didn’t have the desired effect).

PLAN AND EXECUTE PARADIGM

IMPLEMENTATION

Define Tools ⇒ Define Execution Agent ⇒ Define the State ⇒ Define Planning
Step ⇒ Define Re-Plan Step ⇒ Create the Graph

2. Reasoning without Observation (ReWOO)

In ReWOO, Xu, et. al, propose an agent that combines a multi-step planner and
variable substitution for effective tool use. It is made of 3 modules:

Planner: Generate the plan in the following format:

Worker: executes the tool with the provided arguments.
Solver: generates the answer for the initial task based on the tool observations.
 REWOO PARADIGM

IMPLEMENTATION

Define Planner ⇒ Define Worker ⇒ Define Solver ⇒ Create the Graph

3. LLMCompiler

LLMCompiler is an agent architecture designed to speed up the execution of agentic

tasks by eagerly-executed tasks within a DAG. It also saves costs on redundant
token usage by reducing the number of calls to the LLM. It has 3 main components:

Planner: Streams a DAG of tasks. Each task contains a tool, arguments, and list of
dependencies.
Task Fetching Unit: Schedules and executes the tasks as soon as they are
executable once their dependencies are met.
Joiner: Responds to the user or triggers a second plan.
 LLMCOMPILER PARADIGM

IMPLEMENTATION

Define Tools ⇒ Define Planner ⇒ DefineTask Fetching Unit ⇒ Define Joiner ⇒

Create the Graph

We worked on an example of a Multi-Agent workflow that's powered by LangGraph,

LCEL (LangChain Expression Language).

GRAPH AND STREAM VISUALIZATIONS

 RESEARCH CHAIN (TEAM)

STREAM VISUALIZATION

{'supervisor': {'next': 'Search'}}

---
{'Search': {'messages': [HumanMessage(content='The main takeaways from the paper
"Extending Llama-3\'s Context Ten-Fold Overnight" are:\n\n1. The context length of
Llama-3-8B-Instruct was extended from 8K to 80K via QLoRA fine-tuning.\n2. The entire
training cycle is super efficient, taking 8 hours on one 8xA800 (80G) GPU machine.\n3. The
resulted model exhibits superior performances across a broad range of evaluation tasks,
such as NIHS, topic retrieval, and long-context language.\n\nYou can access the paper for
more detailed information from the following sources:\n- [Hugging
Face](https://huggingface.co/papers/2404.19553)\n-
[arXiv](https://arxiv.org/abs/2404.19553)\n-
[GitHub](https://github.com/FlagOpen/FlagEmbedding/blob/master/Long_LLM/longllm_qlora/
README.md)\n- [Semantic
Scholar](https://www.semanticscholar.org/paper/Extending-Llama-3\'s-Context-Ten-Fold-Ove
rnight-Zhang-Shao/ebbee4cf985f7a9eaf2ee944f5ec8cb59a9679a0)\n- [Papers with
Code](https://paperswithcode.com/paper/extending-llama-3-s-context-ten-fold)',
name='Search')]}}
---
{'supervisor': {'next': 'FINISH'}}
---

AUTHORING CHAIN (TEAM)

STREAM VISUALIZATION

{'supervisor': {'next': 'NoteTaker'}}

---
{'NoteTaker': {'messages': [HumanMessage(content='The poem "Whispers of the Seasons"
has been written based on the outline and saved to disk as
"Whispers_of_the_Seasons_Poem". The creative process has been completed, and the
poem is now ready for any further use or publication.', name='NoteTaker')]}}
---
{'supervisor': {'next': 'FINISH'}}
---

(AUTHORING + RESEARCH) SUPER CHAIN

STREAM VISUALIZATION

{'supervisor': {'next': 'ResearchTeam'}}

---
{'ResearchTeam': {'messages': [HumanMessage(content='The North American sturgeon is a
fascinating species that is of great importance for conservation and restoration efforts. The
North American Sturgeon and Paddlefish Society (NASPS) is the North American affiliate of
the World Sturgeon Conservation Society and is dedicated to promoting the conservation
and restoration of sturgeon species in North America by developing and advancing research
pertaining to their biology, management, and utilization.\n\nSturgeon, especially the Lake
Sturgeon, are known for their unique characteristics and habitat requirements. They have
sensory whiskers called barbels in front of their mouth that help them locate food, and they
often lay their eggs on rocky substrates with specific water velocity, depth, and temperature
conditions.\n\nConservation efforts for sturgeon are crucial as they are threatened by
overfishing, poaching, habitat destruction, and the construction of dams that have altered or
blocked their annual migration to ancestral spawning grounds. Many species of sturgeon are
classified as threatened or endangered, with noticeable declines in sturgeon populations as
the demand for caviar increases.\n\nThe North American Sturgeon and Paddlefish Society
(NASPS) holds annual meetings and symposiums to discuss pertinent issues on sturgeons
and seeks opportunities for close cooperation at an international level.\n\nHere is a brief
chart summarizing the main points:\n\n| Aspect | Information
|\n|------------------------|----------------------------------------------------------------------------------------------
--------------------|\n| Conservation Efforts | NASPS is dedicated to promoting the
conservation and restoration of sturgeon species in North America. |\n| Threats
| Sturgeon are threatened by overfishing, poaching, habitat destruction, and the construction
of dams. |\n| Conservation Status | Many species of sturgeon are classified as
threatened or endangered, with noticeable declines in populations. |\n| Research Focus
| NASPS develops and advances research pertaining to the biology, management, and
utilization of sturgeon. |\n| Meetings and Symposia | NASPS holds annual meetings and
symposiums to discuss pertinent issues on sturgeons and seek international cooperation.
|\n\nFor more detailed information, you can visit the North American Sturgeon and
Paddlefish Society website at [NASPS](https://www.nasps-sturgeon.org/).\n\nThis report
provides an overview of the North American sturgeon and the conservation efforts
associated with this unique species.', name='Search')]}}
---
{'supervisor': {'next': 'PaperWritingTeam'}}
---
{'PaperWritingTeam': {'messages': [HumanMessage(content='The summary document titled
"North_American_Sturgeon_Conservation_Summary" has been successfully created and
saved. It provides an overview of the North American sturgeon, the conservation efforts by
the North American Sturgeon and Paddlefish Society (NASPS), and the threats faced by
these species. This summary can serve as a basis for further research and development of
the full conservation paper.', name='DocWriter')]}}
---
{'supervisor': {'next': 'FINISH'}}
---

EVALUATION

AGENT BASED (create a "virtual user" to simulate a conversation).

IN LANGSMITH