How can one create mixin dataclasses with native dataclass mapping? #9211

skilkis · 2023-02-01T17:29:20Z

skilkis
Feb 1, 2023

Previously with the declarative dataclass mapping it was quite simple to create mixins to inherit certain fields and support type-hinting. However, when I attempt to do the same with native dataclass mapping, I run into the issue that inherited fields are in reverse order at runtime. For example, the code below raises a TypeError: non-default argument 'name' follows default argument.

from __future__ import annotations

from typing import ClassVar, Optional

from sqlalchemy import Identity, Integer, String, Text
from sqlalchemy.orm import (
    Mapped,
    MappedAsDataclass,
    declarative_mixin,
    declared_attr,
    mapped_column as column,
    registry,
)

models = registry()


@declarative_mixin
class Model(MappedAsDataclass):
    __tablename__: ClassVar[str]

    id: Mapped[int] = column(Integer, Identity(), init=False, primary_key=True)

    @declared_attr.directive
    @classmethod
    def __tablename__(cls) -> str:
        return cls.__name__.lower()


@declarative_mixin
class NamedModel(Model):
    name: Mapped[str] = column(String(255), nullable=False, index=True)


@declarative_mixin
class DescribedModel(NamedModel):
    description: Mapped[Optional[str]] = column(
        Text,
        nullable=True,
        default=None,
    )


@models.mapped_as_dataclass
class Book(DescribedModel):
    publisher: Mapped[Optional[str]] = column(
        String(255),
        nullable=True,
        default=None,
    )


book = Book()

The desired declarative mapping approach is as follows:

from __future__ import annotations

from dataclasses import dataclass, field
from typing import ClassVar, Optional

from sqlalchemy import Column, Identity, Integer, String, Text
from sqlalchemy.orm import declarative_mixin, declared_attr, registry

SQL = "sql"

models = registry()


@declarative_mixin
@dataclass
class Model:
    __tablename__: ClassVar[str]
    __sa_dataclass_metadata_key__ = SQL

    id: int = field(
        init=False,
        metadata={SQL: Column(Integer, Identity(), primary_key=True)},
    )

    @declared_attr.directive
    @classmethod
    def __tablename__(cls) -> str:
        return cls.__name__.lower()


@declarative_mixin
@dataclass
class NamedModel(Model):
    name: str = field(
        metadata={
            SQL: lambda: Column(String(255), nullable=False, index=True)
        },
    )


@declarative_mixin
@dataclass
class DescribedModel(NamedModel):
    description: Optional[str] = field(
        default=None,
        metadata={SQL: Column(Text, nullable=True)},
    )


@models.mapped
@dataclass
class Book(DescribedModel):
    publisher: Optional[str] = field(
        default=None,
        metadata={SQL: Column(String(255), nullable=True)},
    )

How could the mixin approach given above with the declarative dataclass mapping be achieved with the new native mapping approach? Thanks a bunch in advance 🚀

zzzeek · 2023-02-01T17:32:18Z

zzzeek
Feb 1, 2023
Maintainer

good news, this is fixed and was released less than five minutes ago in 2.0.1. See https://docs.sqlalchemy.org/en/20/changelog/changelog_20.html#change-c35566767e596bb56954f885049a247d / #9179

0 replies

CaselIT · 2023-02-01T18:59:07Z

CaselIT
Feb 1, 2023
Maintainer

There is something strange coing on when a base class is not used. I think it's a bug, but I'm not certain:

from __future__ import annotations
import dataclasses as dc

import sqlalchemy as sa
from sqlalchemy import orm
from sqlalchemy.orm import Mapped

models = orm.registry()

@orm.declarative_mixin
class Model(orm.MappedAsDataclass):

    id: Mapped[int] = orm.mapped_column(
        sa.Integer, sa.Identity(), init=False, primary_key=True
    )

    @orm.declared_attr.directive
    @classmethod
    def __tablename__(cls) -> str:
        return cls.__name__.lower()

@models.mapped_as_dataclass
class Book(Model):
    publisher: Mapped[str | None] = orm.mapped_column(
        sa.String(255),
        nullable=True,
        default=None,
    )

class Base(orm.DeclarativeBase):
    pass

class Book2(Model, Base):
    publisher: Mapped[str | None] = orm.mapped_column(
        sa.String(255),
        nullable=True,
        default=None,
    )

print("with decorator")
print(dc.is_dataclass(Book))
print(Book())
print("with base decorator")
print(dc.is_dataclass(Book2))
print(Book2())

this prints

with decorator
True
Book(id=<sqlalchemy.orm.properties.MappedColumn object at 0x0000028A60A29540>, publisher=<sqlalchemy.orm.properties.MappedColumn object at 0x0000028A60A2A140>)
with base decorator
True
Book2(id=None, publisher=None)

when not using a base class something is wrong

5 replies

zzzeek Feb 1, 2023
Maintainer

so the class here is being made into a dataclass twice because it both inherits from orm.MappedAsDataclass and also has the @models.mapped_as_dataclass decorator on it. These can't be mixed because the superclass vs. the decorator apply the mapping at different times.

The case as written can't really work, to see it "work", I added this fake- patch

diff --git a/lib/sqlalchemy/orm/decl_api.py b/lib/sqlalchemy/orm/decl_api.py
index e3e2611da1..02b483ba30 100644
--- a/lib/sqlalchemy/orm/decl_api.py
+++ b/lib/sqlalchemy/orm/decl_api.py
@@ -622,7 +622,7 @@ class MappedAsDataclass(metaclass=DCTransformDeclarative):
 
         super().__init_subclass__()
 
-        if not _is_mapped_class(cls):
+        if not _is_mapped_class(cls) and not getattr(cls, "_dont_make_a_dataclass", False):
             _ClassScanMapperConfig._apply_dataclasses_to_any_class(
                 current_transforms, cls
             )

then add that

@models.mapped_as_dataclass
class Book(Model):
    _dont_make_a_dataclass = True

    publisher: Mapped[str | None] = orm.mapped_column(
        sa.String(255),
        nullable=True,
        default=None,
    )

otherwise the class is turned into a dataclass by the MappedAsDataclass superclass before it's mapped, which will have bad effects

this is a patch which can guard against this, however it relies on non-documented dataclasses API. There's no other way to do this AFAICT though

diff --git a/lib/sqlalchemy/orm/decl_base.py b/lib/sqlalchemy/orm/decl_base.py
index a858f12cb9..28bb81d249 100644
--- a/lib/sqlalchemy/orm/decl_base.py
+++ b/lib/sqlalchemy/orm/decl_base.py
@@ -1036,6 +1036,13 @@ class _ClassScanMapperConfig(_MapperConfig):
         if not dataclass_setup_arguments:
             return
 
+        if "__dataclass_fields__" in self.cls.__dict__:
+            raise exc.InvalidRequestError(
+                f"Class {self.cls} is already a dataclass; ensure that "
+                "base classes / decorator styles of establishing dataclasses "
+                "are not being mixed"
+            )
+
         manager = instrumentation.manager_of_class(self.cls)
         assert manager is not None

CaselIT Feb 1, 2023
Maintainer

is_dataclass is litterally that check:

# The name of an attribute on the class where we store the Field
# objects.  Also used to check if a class is a Data Class.
_FIELDS = '__dataclass_fields__'

def is_dataclass(obj):
    """Returns True if obj is a dataclass or an instance of a
    dataclass."""
    cls = obj if isinstance(obj, type) else type(obj)
    return hasattr(cls, _FIELDS)

I can add that check

CaselIT Feb 1, 2023
Maintainer

mmm. It's actually different, since it uses hasattr

CaselIT Feb 1, 2023
Maintainer

I've pushed at https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/4413

zzzeek Feb 1, 2023
Maintainer

yup, hasattr() is not the same as "in __dict__" and dataclasses assigns very different meaning to each

skilkis · 2023-02-01T19:31:00Z

skilkis
Feb 1, 2023
Author

@zzzeek @CaselIT thank you for the very quick reply. I've tried the approach using Base with 2.0.1 and it is working fine. Also running into the same issue with the id column if registry.mapped_as_dataclass is used instead

2 replies

CaselIT Feb 1, 2023
Maintainer

yes looking at it it seems a bug, But I'm 100% sure if that's the correct usage (if not then we are probably missing an error)

zzzeek Feb 1, 2023
Maintainer

just FYI the is_dataclass() function is not useful for this kind of case since it returns True if any class in its hierarchy is a dataclass. this is not the same thing as every individual class being a dataclass which is what you need for attributes on each class to be correctly included in constructors etc.

skilkis · 2023-02-02T16:18:02Z

skilkis
Feb 2, 2023
Author

@zzzeek I am getting a similar error today where a mixin dataclass field declared with declared_attr is overridden when the base class using the mixin is subclassed. Minimal repro is as follows:

import dataclasses
from typing import Any, Dict, Optional

from sqlalchemy import ForeignKey, Identity, Integer, String, orm
from sqlalchemy.orm import Mapped, mapped_column


class Base(orm.DeclarativeBase):
    pass


class Model(orm.MappedAsDataclass):
    @orm.declared_attr.directive
    @classmethod
    def __tablename__(cls) -> str:
        return cls.__name__.lower()

    @orm.declared_attr.directive
    @classmethod
    def __mapper_args__(cls) -> Dict[str, Any]:
        return {
            "polymorphic_identity": cls.__name__,
            "polymorphic_on": "polymorphic_type",
        }

    @orm.declared_attr
    @classmethod
    def polymorphic_type(cls) -> Mapped[str]:
        return mapped_column(
            String,
            insert_default=cls.__name__,
            init=False,
        )


class Book(Model, Base):
    id: Mapped[int] = mapped_column(
        Integer,
        Identity(),
        primary_key=True,
        init=False,
    )


fields = dataclasses.fields(Book)
print(Book, {field.name: f"init={field.init}" for field in fields})


class Novel(Book):
    id: Mapped[int] = mapped_column(
        ForeignKey("book.id"),
        primary_key=True,
        init=False,
    )
    description: Mapped[Optional[str]]

If you use a debugger you can see that polymorphic_type becomes a new field in Novel where init=True. I was able to narrow down this behavior to how _ClassScanMapperConfig._setup_dataclasses_transforms injects gathered annotations, and adds polymorphic_type into Novel.__annotations__. The subsequent dataclass transformation on Novel then replaces the previous Book.polymorphic_type field with a new one where init=True.

4 replies

zzzeek Feb 2, 2023
Maintainer

I hvae a good idea how that might happen, which is hopefully it and would be easy to fix

zzzeek Feb 2, 2023
Maintainer

nope, that's not what's happening. what you said is happening, is what's happening.

skilkis Feb 2, 2023
Author

Thank you for looking into it! It's insane how quick you respond 💯 Very much appreciated

zzzeek Feb 2, 2023
Maintainer

I've added #9226 for this. I fortunately came up with how it can be fixed but i need to make the solution more robust.

zzzeek · 2025-09-13T20:00:42Z

zzzeek
Sep 13, 2025
Maintainer

chiming in here as it seems we have some more recent issues that touch upon the patterns here, so also see #12733, #12854

in 2.1 with the patch at https://gerrit.sqlalchemy.org/c/sqlalchemy/sqlalchemy/+/6013 we can do mixins with the decorators, but you have to stick to all decorators or all superclasses. so @CaselIT 's example becomes:

from __future__ import annotations
import dataclasses as dc

import sqlalchemy as sa
from sqlalchemy import orm
from sqlalchemy.orm import Mapped

models = orm.registry()

@orm.unmapped_dataclass
class DecoratorModel:

    id: Mapped[int] = orm.mapped_column(
        sa.Integer, sa.Identity(), init=False, primary_key=True
    )

    @orm.declared_attr.directive
    @classmethod
    def __tablename__(cls) -> str:
        return cls.__name__.lower()


@orm.mapped_as_dataclass(models)
class Book(DecoratorModel):
    publisher: Mapped[str | None] = orm.mapped_column(
        sa.String(255),
        nullable=True,
        default=None,
    )


class MixinModel(orm.MappedAsDataclass):

    id: Mapped[int] = orm.mapped_column(
        sa.Integer, sa.Identity(), init=False, primary_key=True
    )

    @orm.declared_attr.directive
    @classmethod
    def __tablename__(cls) -> str:
        return cls.__name__.lower()

class Base(orm.DeclarativeBase):
    pass

class Book2(MixinModel, Base):
    publisher: Mapped[str | None] = orm.mapped_column(
        sa.String(255),
        nullable=True,
        default=None,
    )

print("with decorator")
print(dc.is_dataclass(Book))
print(Book())
print("with base classes")
print(dc.is_dataclass(Book2))
print(Book2())

as we learned in #12733 we can't really mix the decorator and mixin class styles right now due to the mechanics of how class initialization works.

0 replies

Uh oh!

How can one create mixin dataclasses with native dataclass mapping? #9211

Uh oh!

Uh oh!

skilkis Feb 1, 2023

Replies: 5 comments · 11 replies

Uh oh!

zzzeek Feb 1, 2023 Maintainer

Uh oh!

CaselIT Feb 1, 2023 Maintainer

Uh oh!

zzzeek Feb 1, 2023 Maintainer

Uh oh!

CaselIT Feb 1, 2023 Maintainer

Uh oh!

CaselIT Feb 1, 2023 Maintainer

Uh oh!

CaselIT Feb 1, 2023 Maintainer

Uh oh!

zzzeek Feb 1, 2023 Maintainer

Uh oh!

Uh oh!

skilkis Feb 1, 2023 Author

Uh oh!

CaselIT Feb 1, 2023 Maintainer

Uh oh!

zzzeek Feb 1, 2023 Maintainer

Uh oh!

skilkis Feb 2, 2023 Author

Uh oh!

zzzeek Feb 2, 2023 Maintainer

Uh oh!

zzzeek Feb 2, 2023 Maintainer

Uh oh!

skilkis Feb 2, 2023 Author

Uh oh!

zzzeek Feb 2, 2023 Maintainer

Uh oh!

zzzeek Sep 13, 2025 Maintainer

skilkis
Feb 1, 2023

Replies: 5 comments 11 replies

zzzeek
Feb 1, 2023
Maintainer

CaselIT
Feb 1, 2023
Maintainer

zzzeek Feb 1, 2023
Maintainer

CaselIT Feb 1, 2023
Maintainer

CaselIT Feb 1, 2023
Maintainer

CaselIT Feb 1, 2023
Maintainer

zzzeek Feb 1, 2023
Maintainer

skilkis
Feb 1, 2023
Author

CaselIT Feb 1, 2023
Maintainer

zzzeek Feb 1, 2023
Maintainer

skilkis
Feb 2, 2023
Author

zzzeek Feb 2, 2023
Maintainer

zzzeek Feb 2, 2023
Maintainer

skilkis Feb 2, 2023
Author

zzzeek Feb 2, 2023
Maintainer

zzzeek
Sep 13, 2025
Maintainer