Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Allow to modify multiple request options in Python SDK (similar to JavaScript SDK) #367

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
josealvarez97 opened this issue May 30, 2024 · 1 comment
Open

Allow to modify multiple request options in Python SDK (similar to JavaScript SDK) #367

josealvarez97 opened this issue May 30, 2024 · 1 comment
Assignees
@MarkDaoust
Labels
component:python sdk Issue/PR related to Python SDK status:triaged Issue/PR triaged to the corresponding sub-team type:feature request New feature request/enhancement

Comments

@josealvarez97
Copy link

josealvarez97 commented May 30, 2024 •
edited
Loading

Description of the feature request:

I would like the Python SDK to allow the same equivalent functionality as the JavaScript SDK regarding the possibility of modifying the request options

JavaScript SDK (can add many custom request options)

const model = googleGenAIClient.getGenerativeModel(
				{ model: "gemini-pro" },
				{
					customHeaders: {
						"x-param": "<PARAMETER>",
					},
					baseUrl: env.PROXY_URL,
					
				}
			);

PythonSDK (cannot add many custom request options)

import google.generativeai as genai

genai.configure(api_key="AIzaSyBW75RG6YjRor0pL1Eu0jWsRSd-rndTNIY")

model = genai.GenerativeModel('gemini-1.0-pro-latest')
response = model.generate_content("The opposite of hot is", 
                                  request_options= {
                                    # Only accepts timeout parameter
                                      "timeout": 10,
                                      # Anything else gives the following error
# TypeError: GenerativeServiceClient.generate_content() got an unexpected keyword argument 'customHeaders'
                                    #   "customHeaders": {
                                    #         "x-param": "<PARAMETER>",
                                    #     },
                                    # I would also like to modify the baseUrl
                                    # "baseUrl": "http://localhost:8787/",
# TypeError: GenerativeServiceClient.generate_content() got an unexpected keyword argument 'baseUrl'    
}
print(response.text)

What problem are you trying to solve with this feature?

Some people have talked about how being able to add metadata to requests in these libraries is helpful for proxies, and even tried to implement the solution themselves as you can find below:

I'm simply frustrated by the fact that it's possible to modify the request options in the JavaScript SDK, but not in the Python SDK.

If I can do it in the JavaScript SDK, why can't I do it in the Python SDK? 🤔

For the record, the Open AI Python SDK allows to do it by adding the parameter default_headers in the client configuration and extra_headers in the AI generation call:

from openai import OpenAI

client = OpenAI(
    api_key="<KEY>",
    base_url="http://localhost:8787",
    default_headers={
        "x-param": "parameter"
    }
)

chat_completion = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Say this is a test",
        }
    ],
    model="<model>",
    extra_headers={
        "x-param": "parameter"
    
    }
)

print(chat_completion.choices[0].message.content)

Any other information you'd like to share?

The documentation also seems to have implemented something similar to customHeaders in the request options (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/nodejs/latest/vertexai/requestoptions), but when I installed the Python SDK, it didn't seem to accept customHeaders as explained above. At any rate, the current Python SDK is far behind the JavaScript SDK in this regard (JS SDK accepts other parameters like baseUrl—useful for proxies).

Additional context
Add any other context or screenshots about the feature request here.

Other relevant links in this conversation may be:
@josealvarez97 josealvarez97 added component:python sdk Issue/PR related to Python SDK type:feature request New feature request/enhancement labels May 30, 2024
@singhniraj08 singhniraj08 added the status:triaged Issue/PR triaged to the corresponding sub-team label May 31, 2024
@bencipher
Copy link

What’s the update on this. I can’t run a tagging Llm chain cos of this issue. I can create the chain but can’t invoke it, get the error below:

C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Scripts\python.exe C:\Users\BEN-UNKNOWN\PycharmProjects\Pet\brazillianscraper\scratch.py
C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain\llms_init_.py:548: LangChainDeprecationWarning: Importing LLMs from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.llms import OpenAI.

To install langchain-community run pip install -U langchain-community.
warnings.warn(
C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_init_.py:29: UserWarning: Importing LLMChain from langchain root module is no longer supported. Please use langchain.chains.LLMChain instead.
warnings.warn(
C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_core_api\deprecation.py:139: LangChainDeprecationWarning: The method Chain.run was deprecated in langchain 0.1.0 and will be removed in 0.3.0. Use invoke instead.
warn_deprecated(
Traceback (most recent call last):
File "C:\Users\BEN-UNKNOWN\PycharmProjects\Pet\brazillianscraper\scratch.py", line 64, in
main()
File "C:\Users\BEN-UNKNOWN\PycharmProjects\Pet\brazillianscraper\scratch.py", line 59, in main
print(chain.run(user_personal_details))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_core_api\deprecation.py", line 168, in warning_emitting_wrapper
return wrapped(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain\chains\base.py", line 600, in run
return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_core_api\deprecation.py", line 168, in warning_emitting_wrapper
return wrapped(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain\chains\base.py", line 383, in call
return self.invoke(
^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain\chains\base.py", line 166, in invoke
raise e
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain\chains\base.py", line 156, in invoke
self._call(inputs, run_manager=run_manager)
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain\chains\llm.py", line 127, in _call
response = self.generate([inputs], run_manager=run_manager)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain\chains\llm.py", line 139, in generate
return self.llm.generate_prompt(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_core\language_models\chat_models.py", line 677, in generate_prompt
return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_core\language_models\chat_models.py", line 534, in generate
raise e
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_core\language_models\chat_models.py", line 524, in generate
self._generate_with_cache(
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_core\language_models\chat_models.py", line 749, in _generate_with_cache
result = self._generate(
^^^^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_google_genai\chat_models.py", line 760, in generate
response: GenerateContentResponse = chat_with_retry(
^^^^^^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_google_genai\chat_models.py", line 189, in chat_with_retry
return chat_with_retry(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\tenacity_init.py", line 330, in wrapped_f
return self(f, *args, **kw)
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\tenacity_init.py", line 467, in call
do = self.iter(retry_state=retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\tenacity_init.py", line 368, in iter
result = action(retry_state)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\tenacity_init.py", line 390, in
self._add_action_func(lambda rs: rs.outcome.result())
^^^^^^^^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures_base.py", line 401, in __get_result
raise self.exception
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\tenacity_init.py", line 470, in call
result = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_google_genai\chat_models.py", line 187, in _chat_with_retry
raise e
File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_google_genai\chat_models.py", line 171, in _chat_with_retry
return generation_method(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: GenerativeServiceClient.generate_content() got an unexpected keyword argument 'function_call'

Process finished with exit code 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@MarkDaoust MarkDaoust

Labels
component:python sdk Issue/PR related to Python SDK status:triaged Issue/PR triaged to the corresponding sub-team type:feature request New feature request/enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@MarkDaoust @bencipher @josealvarez97 @singhniraj08