-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
integration with typing annotations for declarative #7535
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
not sure if I like this idea, I would need to think a bit abut it. |
for relationships, we usually will need the left-annotation to be the explicit part: user: Mapped["User"] = relationship() This is because we can't annotate relationship to produce the correct type in the other direction, when the class argument is a string: # can't type this as Mapped[User] without explicit type
user = relationship("User") So i was thinking of being consistent, or at least allowing consistency, to have the mapping start up from the "x: Mapped[y]" side. for the type lookup, we already have this mapping right here: https://github.com/sqlalchemy/sqlalchemy/blob/main/lib/sqlalchemy/sql/sqltypes.py#L2998 we already do this, nobody needs to override it or anything. if you have a specific type you want to use with mapped[int] then you specify it in the mapped_column(), no differently than people do now anyway with "cast()" or whatever. |
there's the general problem of str->Unicode for databases with explicit unicode datatypes and data conversion issues like pyodbc. so we might need to improve that part. |
i'd propose a subclass of str: |
I think it would be useful to provide an override at the registry level. Like for pg I would like to map str to Text, not varchar and float to double_precision. Also maybe folks like to map int to biging |
The mapping we have is also not really complete since it's not dialect specific, like uuid is missing. Also I'm not sure if we can support also list->array and extract the type from a list annotation to do list[str] -> Array(Text) |
UUID is just going to be missing, this would be a very basic typing-saver only. re: override registry it can be something local to the registry() object, not a global. |
well then if there's a registry override there's where you put your UUID :) |
things like Array are also very unusual edge cases |
sure, that was my suggestion, since that's for declarative one we do have a registry always available, so it should not be that much of an issue to support. we can have a constructor argument called
indeed, it would be there |
Mike Bayer referenced this issue: WIP for ORM typing https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/3495 |
introduces: 1. new mapped_column() helper 2. DeclarativeBase helper 3. declared_attr has been re-typed 4. rework of Mapped[] to return InstrumentedAtribute for class get, so works without Mapped itself having expression methods 5. ORM constructs now generic on [_T] also includes some early typing work, most of which will be in later commits: 1. URL and History become typing.NamedTuple 2. come up with type-checking friendly way of type checking cy extensions, where type checking will be applied to the py versions, just needed to come up with a succinct conditional pattern for the imports References: #6810 References: #7535 References: #7562 Change-Id: Ie5d9a44631626c021d130ca4ce395aba623c71fb
I think this pep would be very useful to specify a particular sqlalchemy type for a column https://www.python.org/dev/peps/pep-0593/ We could have something like class Foo:
foo: int
bar: Annotated[str, String(42)] This would also solve the type map issue we talked above ( or could be used as an alternative / in conjunction with it ) |
possibly. it looks like "yet another syntax" so far, as opposed to "bar: mapped_column(String(42))" but something to consider |
Maybe it could be in place of other syntaxes? This seems the official python way of doing what this kind of things. |
the Column construct accepts a dozen different kinds of parameters and arguments. Where do things like ForeignKey(), "key", "name" etc. get specified if Annotated[str, String] takes the place of Column /mapped_column? also it feels redundant to have to type things like Annotated[str, TEXT] - mapped_column(TEXT) is more succinct. there is more variability in a single Python type to multiple database types, than the other way around (the use case for the "type map", which would be much less prominent since it is usually not needed). |
that is, I think the canonical way to map declaratively is going to stay mostly like it is: class User(Base):
__tablename__ = 'user'
id = mapped_column(Integer, primary_key=True)
name = mapped_column(String, nullable=False)
some_date = mapped_column(DateTime)
addresses = relationship(lambda: Address, back_populates="user", uselist=True) the above is what people are used to, it's extremely close to what's documented everywhere and we can derive all the typings above without any annotations. obviously relationship() will usually not be able to use lambda like that so still some exception there. |
Ok so I remembered that we wanted to allow mapping without specifying mapped_column / relationship.
I think annotated allows any type after the first one (and also allows multiple arguments) so I guess Probably just something to take into consideration if we want to experiment
I just though of this, and most likely the type checker do not like it but we could make _T = TypeVar('_T')
class relationship(Generic[_T]):
@overload
def __new__(self, thing, uselist=False, ....) -> _T
@overload
def __new__(self, thing, uselist=True, ....) -> List[_T]
def __new__(self, thing, ...) -> _T
...
return RelationshipProperty(...) so that we could use it as if TYPE_CHECKING:
from .other import Address
class User(Base):
__tablename__ = 'user'
id = mapped_column(Integer, primary_key=True)
name = mapped_column(String, nullable=False)
some_date = mapped_column(DateTime)
addresses = relationship['Address']('Address', back_populates="user", uselist=True) |
Mike Bayer referenced this issue: establish mypy / typing approach for v2.0 https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/3562 |
Mike Bayer has proposed a fix for this issue in the main branch: establish mypy / typing approach for v2.0 https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/3562 |
docs for all this stuff are going to be a separate commit. typing affects a lot of things so this commit is just grouping all of it together towards the goal of expediency. |
The Mypy plugin is not maintainable long-term and will be replaced by new APIs that allow for typing to work inline without the need for plugins. Change-Id: Icc7a203df1d0b19bde2fd852719b7b7215774c58 References: #7535
The Mypy plugin is not maintainable long-term and will be replaced by new APIs that allow for typing to work inline without the need for plugins. Change-Id: Icc7a203df1d0b19bde2fd852719b7b7215774c58 References: sqlalchemy#7535
large patch to get ORM / typing efforts started. this is to support adding new test cases to mypy, support dropping sqlalchemy2-stubs entirely from the test suite, validate major ORM typing reorganization to eliminate the need for the mypy plugin. * New declarative approach which uses annotation introspection, fixes: sqlalchemy#7535 * Mapped[] is now at the base of all ORM constructs that find themselves in classes, to support direct typing without plugins * Mypy plugin updated for new typing structures * Mypy test suite broken out into "plugin" tests vs. "plain" tests, and enhanced to better support test structures where we assert that various objects are introspected by the type checker as we expect. as we go forward with typing, we will add new use cases to "plain" where we can assert that types are introspected as we expect. * For typing support, users will be much more exposed to the class names of things. Add these all to "sqlalchemy" import space. * Column(ForeignKey()) no longer needs to be `@declared_attr` if the FK refers to a remote table * composite() attributes mapped to a dataclass no longer need to implement a `__composite_values__()` method * with_variant() accepts multiple dialect names Change-Id: I22797c0be73a8fbbd2d6f5e0c0b7258b17fe145d Fixes: sqlalchemy#7535 Fixes: sqlalchemy#7551 References: sqlalchemy#6810
Uh oh!
There was an error while loading. Please reload this page.
this is a continuation of the thinking from sqlalchemy/sqlalchemy2-stubs#170 with some experimentation at sqlalchemy/sqlalchemy2-stubs@a436c38 .
Current thinking:
mapped_column()
. This will basically becolumn_property()
with some helpers.Mapped
directly. via a subclass_DeclartativeMappedPlaceholder
, something like that. The__get__
__set__
etc methods can raise NotImplementedError for instance level access, since under normal use there will never be an instance of a class with these attributes set, they will be replaced byInstrumentedAttribute
__annotations__
, if arguments are not present. this shouldn't be that hard to accomplish. Yes it means I'd like a lookup of Mapped[int] -> Column(Integer), stuff like that. Big deal.__annotations__
only returned strings, as was the issue in PEP 563, PEP 649 and pydantic pydantic/pydantic#2678 , we could still work with that because we already do relationship() using strings, and column types are trivial. looks like that area of python is getting a lot of attention though so there should be good behaviors there.__init__()
being added. There's no way to have a class with attributes where the attributes magically create an__init__
method, unless you use dataclasses, which the type checkers have added hardcoded rules to accomplish. same with__table__
and all that. Propose a stub class that people will use, like:if someone wants their
__init__
to have full typing information, they have to write it out. I dont see any way around that other than trying to trick the static checkers into thinking we are using dataclasses and that does not seem like a wise choice.so here's a mapping:
so I think that can work? but im not sure, if mapped_column() is typed as
Mapped[Any]
can I declare that asMapped[int]
? I think so? I can't get to testing the above until I get a good chunk of SQLAlchemy 2.0 typed; right now things are a mess with it not being typed and the stubs being more wrong every day.so that's one way, now can it also work the other way? I think to some extent at least? here's that:
so here, I think it works if we type things as:
similar idea for relationship, etc. with the constructs all working in both ways it should make for a lot more typing happening automatically at least? not sure. this is seeming easier than it did previously which is making me not trust that im not missing something. anyway, thinking this. need to get lots of annotations into 2.0 main before we can fluently play with this stuff.
The text was updated successfully, but these errors were encountered: