Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Bug: Conflicting dependencies with Pydantic V2 #2926

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
barreeeiroo opened this issue Aug 4, 2023 · 3 comments
Closed

Bug: Conflicting dependencies with Pydantic V2 #2926

barreeeiroo opened this issue Aug 4, 2023 · 3 comments
Labels
parser Parser (Pydantic) utility

Comments

@barreeeiroo
Copy link
Contributor

barreeeiroo commented Aug 4, 2023

Expected Behaviour

When defining a requirements.txt file with dependencies, it should be possible to define it like this:

aws-lambda-powertools[all]~=2.22.0
pydantic[email]~=2.1.1

Running pip install -r requirements.txt should install both aws-lambda-powertools and pydantic packages. AWS Lambda Powertools is compatible with Pydantic V2 thanks to #2733.

Current Behaviour

Running pip install -r requirements.txt raises the following error:

ERROR: Cannot install aws-lambda-powertools[all]==2.22.0 and pydantic[email]==2.1.1 because these package versions have conflicting dependencies.

The conflict is caused by:

  • aws-lambda-powertools[all] 2.22.0 depends on pydantic<2.0.0 and >=1.8.2; extra == "parser" or extra == "all"
  • pydantic[email] 2.1.1 depends on pydantic 2.1.1 (Installed)

Code snippet

aws-lambda-powertools[all]~=2.22.0
pydantic[email]~=2.1.1

pip install -r requirements.txt

Possible Solution

I was surprised when finding this error because, based on #2672 and #2733, it seems like AWS Lambda Powertools was already compatible with Pydantic V2.

However, when looking at the pyproject.toml file I found out about the dependency being still in V1:

[tool.poetry.dependencies]
...
pydantic = { version = "^1.8.2", optional = true }

Shouldn't this be set to be either ^1.8.2 or ^2.0.3 (based on Pydantic V2 usage docs)?

So, a possible solution could be:

[tool.poetry.dependencies]
...
pydantic = { version = ">=1.8.2, !=2.0.0, !=2.0.1, !=2.0.2, <3.0.0", optional = true }

Ideally, it would be nice to have something like ^1.8.2 || ^2.0.3, but not sure if that's allowed in Poetry.

Steps to Reproduce

  1. Create a requirements.txt file.
  2. Include both AWS Lambda Powertools and Pydantic V2 as direct dependencies.
  3. Try to install dependencies through pip install -r requirements.txt.

Powertools for AWS Lambda (Python) version

2.22.0

AWS Lambda function runtime

3.11

Packaging format used

Lambda Layers, PyPi

Debugging logs

No response

@barreeeiroo barreeeiroo added bug Something isn't working triage Pending triage from maintainers labels Aug 4, 2023
@leandrodamascena leandrodamascena moved this from Triage to Working on it in Powertools for AWS Lambda (Python) Aug 4, 2023
@leandrodamascena leandrodamascena added parser Parser (Pydantic) utility and removed bug Something isn't working triage Pending triage from maintainers labels Aug 4, 2023
@leandrodamascena
Copy link
Contributor

Hello @barreeeiroo! Thank you for opening this issue! This is actually not a bug. We've kept Pydantic v1 as the default in Powertools to avoid issues with dependency resolutions for customers who are using Powertools with Pydantic v1 and don't want to switch to Pydantic v2.

To add the use of Pydantic v2 with Powertools, you must remove the optional dependency for Pydantic. In you case you can change your file:

aws-lambda-powertools[aws-xray-sdk, fastjsonschema]~=2.22.0
pydantic[email]~=2.1.1

Reference: https://docs.powertools.aws.dev/lambda/python/latest/utilities/parser/#using-pydantic-v2

Thanks

@leandrodamascena leandrodamascena moved this from Working on it to Pending customer in Powertools for AWS Lambda (Python) Aug 4, 2023
@barreeeiroo
Copy link
Contributor Author

barreeeiroo commented Aug 5, 2023

Hi @leandrodamascena, you're totally right. I've updated the requirements.txt file to your suggestion and it's now working (I didn't think about removing the parser extra), sorry about the not-bug.

As a possible improvement though, maybe the documentation regarding Pydantic V2 could be improved a little bit. I have already checked the link you mention before (Using Pydantic V2) and my assumption was that I "just" had to include the pydantic v2 dependency, which is what I did. I didn't think about removing the parser extra from aws-lambda-powertools.
Now that I read more carefully the RFC, I can see that you mentioned "customers should refrain from using [all] or [parser] when bringing Pydantic v2 as part of their dependencies" but couldn't actually find it in the documentation where its mentioned how to use Parser with Pydantic V2 (that's why I assumed I only had to include Pydantic V2 as dependency, without any additional task).

In the documentation, right below where it says "Add aws-lambda-powertools and pydantic>=2.0.3 as a dependency in your preferred tool", maybe a warning alert box could be created stating something like "Make sure your aws-lambda-powertools dependency doesn't include the [all] or [parser] extras to avoid the dependency conflict.", just like the RFC said.
I can take care of adding it if you'd like to add that clarification about Pydantic V2 usage.


EDIT: requirements.txt should be:

aws-lambda-powertools[tracer,validation]~=2.22.0

@github-project-automation github-project-automation bot moved this from Pending customer to Coming soon in Powertools for AWS Lambda (Python) Aug 5, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Aug 5, 2023

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

@leandrodamascena leandrodamascena moved this from Coming soon to Shipped in Powertools for AWS Lambda (Python) Aug 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
parser Parser (Pydantic) utility
Projects
Status: Shipped
Development

No branches or pull requests

2 participants