Thanks to visit codestin.com
Credit goes to github.com

Skip to content

No rate limit error when using asynchronous calls for the chatgpt api #321

Closed
@userrand

Description

@userrand

Describe the bug

When using the ChatGPT api with asynchronous calls, I do not receive rate error limits, instead the code seems stuck.

To Reproduce

Here are 2 example codes that use asynchronous calls.

Note that I learned about asynchronous calls for this purpose and so have little experience.

1 Without the Python wrapper

import asyncio
from aiohttp import ClientSession

openai_api_key = "key_here"

headers = {
    "Content-Type": "application/json",
    "Authorization": "Bearer " + openai_api_key,
}
url = "https://api.openai.com/v1/chat/completions"

async def ChatBatch(prompts,temperature=0,model= "gpt-3.5-turbo") :

    async def Chat_Result(session, prompt, url=url, headers=headers):
        data = {"model": model,
            "messages": [{"role": "user", "content": prompt}],
            "temperature": temperature
        }
        async with session.post(url, headers=headers, json=data) as result:
            return await result.json()

    async def get_response():
        results = []
        async with ClientSession() as session:
            for prompt in prompts:
                result = Chat_Result(session, prompt)
                results.append(result)
            return await asyncio.gather(*results)


    # return await get_response()
    return [response['choices'][0]['message']['content'] for response in await get_response()]

test :

await ChatBatch(["Hello how are you ?"]*30)

2 With the Python Wrapper

import openai
import asyncio
from aiohttp import ClientSession

openai.api_key = "key_here"


openai.aiosession.set(ClientSession())

async def create_completion(prompt="How are you ?"):
    return await openai.ChatCompletion.acreate(messages=[{"role": "user", "content": prompt}], model="gpt-3.5-turbo")

async def main():

    return await asyncio.gather(*[create_completion() for _ in range(14)])

answers=await main()

await openai.aiosession.get().close()

answers

Code snippets

No response

OS

Linux

Python version

Python 3.9.12

Library version

openai v0.27

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions