langchain/docs/modules/agents/examples/async_agent.ipynb

Async API for Agent

LangChain provides async support for Agents by leveraging the asyncio library.

Async methods are currently supported for the following Tools: SerpAPIWrapper and LLMMathChain. Async support for other agent tools are on the roadmap.

For Tools that have a coroutine implemented (the two mentioned above), the AgentExecutor will await them directly. Otherwise, the AgentExecutor will call the Tool's func via asyncio.get_event_loop().run_in_executor to avoid blocking the main runloop.

You can use arun to call an AgentExecutor asynchronously.

Serial vs. Concurrent Execution

In this example, we kick off agents to answer some questions serially vs. concurrently. You can see that concurrent execution significantly speeds this up.

In [1]:

import asyncio
import time

from langchain.agents import initialize_agent, load_tools
from langchain.llms import OpenAI
from langchain.callbacks.stdout import StdOutCallbackHandler
from langchain.callbacks.base import CallbackManager
from langchain.callbacks.tracers import LangChainTracer
from aiohttp import ClientSession

questions = [
    "Who won the US Open men's final in 2019? What is his age raised to the 0.334 power?",
    "Who is Olivia Wilde's boyfriend? What is his current age raised to the 0.23 power?",
    "Who won the most recent formula 1 grand prix? What is their age raised to the 0.23 power?",
    "Who won the US Open women's final in 2019? What is her age raised to the 0.34 power?",
    "Who is Beyonce's husband? What is his age raised to the 0.19 power?"
]

In [2]:

def generate_serially():
    for q in questions:
        llm = OpenAI(temperature=0)
        tools = load_tools(["llm-math", "serpapi"], llm=llm)
        agent = initialize_agent(
            tools, llm, agent="zero-shot-react-description", verbose=True
        )
        agent.run(q)

s = time.perf_counter()
generate_serially()
elapsed = time.perf_counter() - s
print(f"Serial executed in {elapsed:0.2f} seconds.")

> Entering new AgentExecutor chain...
 I need to find out who won the US Open men's final in 2019 and then calculate his age raised to the 0.334 power.
Action: Search
Action Input: "US Open men's final 2019 winner"
Observation: Rafael Nadal
Thought: I need to find out Rafael Nadal's age
Action: Search
Action Input: "Rafael Nadal age"
Observation: 36 years
Thought: I need to calculate 36 raised to the 0.334 power
Action: Calculator
Action Input: 36^0.334
Observation: Answer: 3.3098250249682484

Thought: I now know the final answer
Final Answer: Rafael Nadal, aged 36, won the US Open men's final in 2019 and his age raised to the 0.334 power is 3.3098250249682484.

> Finished chain.


> Entering new AgentExecutor chain...
 I need to find out who Olivia Wilde's boyfriend is and then calculate his age raised to the 0.23 power.
Action: Search
Action Input: "Olivia Wilde boyfriend"
Observation: Jason Sudeikis
Thought: I need to find out Jason Sudeikis' age
Action: Search
Action Input: "Jason Sudeikis age"
Observation: 47 years
Thought: I need to calculate 47 raised to the 0.23 power
Action: Calculator
Action Input: 47^0.23
Observation: Answer: 2.4242784855673896

Thought: I now know the final answer
Final Answer: Jason Sudeikis, Olivia Wilde's boyfriend, is 47 years old and his age raised to the 0.23 power is 2.4242784855673896.

> Finished chain.


> Entering new AgentExecutor chain...
 I need to find out who won the grand prix and then calculate their age raised to the 0.23 power.
Action: Search
Action Input: "Formula 1 Grand Prix Winner"
Observation: Max Verstappen
Thought: I need to find out Max Verstappen's age
Action: Search
Action Input: "Max Verstappen Age"
Observation: 25 years
Thought: I need to calculate 25 raised to the 0.23 power
Action: Calculator
Action Input: 25^0.23
Observation: Answer: 1.84599359907945
Thought: I now know the final answer
Final Answer: Max Verstappen, 25 years old, raised to the 0.23 power is 1.84599359907945.

> Finished chain.


> Entering new AgentExecutor chain...
 I need to find out who won the US Open women's final in 2019 and then calculate her age raised to the 0.34 power.
Action: Search
Action Input: "US Open women's final 2019 winner"
Observation: Bianca Andreescu defeated Serena Williams in the final, 6–3, 7–5 to win the women's singles tennis title at the 2019 US Open. It was her first major title, and she became the first Canadian, as well as the first player born in the 2000s, to win a major singles title.
Thought: I need to find out Bianca Andreescu's age.
Action: Search
Action Input: "Bianca Andreescu age"
Observation: 22 years
Thought: I now know the age of Bianca Andreescu and can calculate her age raised to the 0.34 power.
Action: Calculator
Action Input: 22^0.34
Observation: Answer: 2.8603798598506933

Thought: I now know the final answer.
Final Answer: Bianca Andreescu won the US Open women's final in 2019 and her age raised to the 0.34 power is 2.8603798598506933.

> Finished chain.


> Entering new AgentExecutor chain...
 I need to find out who Beyonce's husband is and then calculate his age raised to the 0.19 power.
Action: Search
Action Input: "Who is Beyonce's husband?"
Observation: Jay-Z
Thought: I need to find out Jay-Z's age
Action: Search
Action Input: "How old is Jay-Z?"
Observation: 53 years
Thought: I need to calculate 53 raised to the 0.19 power
Action: Calculator
Action Input: 53^0.19
Observation: Answer: 2.12624064206896

Thought: I now know the final answer
Final Answer: Jay-Z is Beyonce's husband and his age raised to the 0.19 power is 2.12624064206896.

> Finished chain.
Serial executed in 65.11 seconds.

In [4]:

async def generate_concurrently():
    agents = []
    # To make async requests in Tools more efficient, you can pass in your own aiohttp.ClientSession, 
    # but you must manually close the client session at the end of your program/event loop
    aiosession = ClientSession()
    for _ in questions:
        manager = CallbackManager([StdOutCallbackHandler()])
        llm = OpenAI(temperature=0, callback_manager=manager)
        async_tools = load_tools(["llm-math", "serpapi"], llm=llm, aiosession=aiosession, callback_manager=manager)
        agents.append(
            initialize_agent(async_tools, llm, agent="zero-shot-react-description", verbose=True, callback_manager=manager)
        )
    tasks = [async_agent.arun(q) for async_agent, q in zip(agents, questions)]
    await asyncio.gather(*tasks)
    await aiosession.close()

s = time.perf_counter()
# If running this outside of Jupyter, use asyncio.run(generate_concurrently())
await generate_concurrently()
elapsed = time.perf_counter() - s
print(f"Concurrent executed in {elapsed:0.2f} seconds.")

> Entering new AgentExecutor chain...


> Entering new AgentExecutor chain...


> Entering new AgentExecutor chain...


> Entering new AgentExecutor chain...


> Entering new AgentExecutor chain...
 I need to find out who Olivia Wilde's boyfriend is and then calculate his age raised to the 0.23 power.
Action: Search
Action Input: "Olivia Wilde boyfriend" I need to find out who Beyonce's husband is and then calculate his age raised to the 0.19 power.
Action: Search
Action Input: "Who is Beyonce's husband?"
Observation: Jay-Z
Thought: I need to find out who won the grand prix and then calculate their age raised to the 0.23 power.
Action: Search
Action Input: "Formula 1 Grand Prix Winner" I need to find out who won the US Open women's final in 2019 and then calculate her age raised to the 0.34 power.
Action: Search
Action Input: "US Open women's final 2019 winner"
Observation: Jason Sudeikis
Thought:
Observation: Max Verstappen
Thought:
Observation: Bianca Andreescu defeated Serena Williams in the final, 6–3, 7–5 to win the women's singles tennis title at the 2019 US Open. It was her first major title, and she became the first Canadian, as well as the first player born in the 2000s, to win a major singles title.
Thought: I need to find out Jason Sudeikis' age
Action: Search
Action Input: "Jason Sudeikis age" I need to find out Jay-Z's age
Action: Search
Action Input: "How old is Jay-Z?"
Observation: 53 years
Thought: I need to find out who won the US Open men's final in 2019 and then calculate his age raised to the 0.334 power.
Action: Search
Action Input: "US Open men's final 2019 winner"
Observation: Rafael Nadal defeated Daniil Medvedev in the final, 7–5, 6–3, 5–7, 4–6, 6–4 to win the men's singles tennis title at the 2019 US Open. It was his fourth US ...
Thought:
Observation: 47 years
Thought: I need to find out Max Verstappen's age
Action: Search
Action Input: "Max Verstappen Age"
Observation: 25 years
Thought: I need to find out Bianca Andreescu's age.
Action: Search
Action Input: "Bianca Andreescu age"
Observation: 22 years
Thought: I need to calculate 53 raised to the 0.19 power
Action: Calculator
Action Input: 53^0.19 I need to find out the age of the winner
Action: Search
Action Input: "Rafael Nadal age" I need to calculate 47 raised to the 0.23 power
Action: Calculator
Action Input: 47^0.23
Observation: 36 years
Thought: I need to calculate 25 raised to the 0.23 power
Action: Calculator
Action Input: 25^0.23
Observation: Answer: 2.12624064206896

Thought: I now know the age of Bianca Andreescu and can calculate her age raised to the 0.34 power.
Action: Calculator
Action Input: 22^0.34
Observation: Answer: 1.84599359907945
Thought:
Observation: Answer: 2.4242784855673896

Thought: I now need to calculate his age raised to the 0.334 power
Action: Calculator
Action Input: 36^0.334
Observation: Answer: 2.8603798598506933

Thought: I now know the final answer
Final Answer: Jay-Z is Beyonce's husband and his age raised to the 0.19 power is 2.12624064206896.

> Finished chain.
 I now know the final answer
Final Answer: Max Verstappen, 25 years old, raised to the 0.23 power is 1.84599359907945.

> Finished chain.

Observation: Answer: 3.3098250249682484

Thought: I now know the final answer
Final Answer: Jason Sudeikis, Olivia Wilde's boyfriend, is 47 years old and his age raised to the 0.23 power is 2.4242784855673896.

> Finished chain.
 I now know the final answer.
Final Answer: Bianca Andreescu won the US Open women's final in 2019 and her age raised to the 0.34 power is 2.8603798598506933.

> Finished chain.
 I now know the final answer
Final Answer: Rafael Nadal, aged 36, won the US Open men's final in 2019 and his age raised to the 0.334 power is 3.3098250249682484.

> Finished chain.
Concurrent executed in 12.38 seconds.

Using Tracing with Asynchronous Agents

To use tracing with async agents, you must pass in a custom CallbackManager with LangChainTracer to each agent running asynchronously. This way, you avoid collisions while the trace is being collected.

In [7]:

# To make async requests in Tools more efficient, you can pass in your own aiohttp.ClientSession, 
# but you must manually close the client session at the end of your program/event loop
aiosession = ClientSession()
tracer = LangChainTracer()
tracer.load_default_session()
manager = CallbackManager([StdOutCallbackHandler(), tracer])

# Pass the manager into the llm if you want llm calls traced.
llm = OpenAI(temperature=0, callback_manager=manager)

async_tools = load_tools(["llm-math", "serpapi"], llm=llm, aiosession=aiosession)
async_agent = initialize_agent(async_tools, llm, agent="zero-shot-react-description", verbose=True, callback_manager=manager)
await async_agent.arun(questions[0])
await aiosession.close()

> Entering new AgentExecutor chain...
 I need to find out who won the US Open men's final in 2019 and then calculate his age raised to the 0.334 power.
Action: Search
Action Input: "US Open men's final 2019 winner"
Observation: Rafael Nadal
Thought: I need to find out Rafael Nadal's age
Action: Search
Action Input: "Rafael Nadal age"
Observation: 36 years
Thought: I need to calculate 36 raised to the 0.334 power
Action: Calculator
Action Input: 36^0.334
Observation: Answer: 3.3098250249682484

Thought: I now know the final answer
Final Answer: Rafael Nadal, aged 36, won the US Open men's final in 2019 and his age raised to the 0.334 power is 3.3098250249682484.

> Finished chain.