Codestin Search App

greg0ire · 2024-03-14T16:20:18Z

When using orm:schema-tool:update together with --complete, all assets not described by the current metadata are dropped.

This is an issue when using doctrine/migrations, because not using that option is deprecated, and because when it is used, the table that holds the migrations is dropped as well since it is not described by ORM metadata.

A solution to that is configuring an asset filter that filters out the metadata table except when running commands inside the migrations namespace.

How can I test this?

composer config repositories.greg0ire vcs https://github.com/greg0ire/DoctrineMigrationsBundle
composer require doctrine/doctrine-migrations-bundle "dev-complete-compat as 3.3.0"

greg0ire · 2024-03-14T16:23:19Z

+        <service id="doctrine_migrations.schema_filter_listener" class="Doctrine\Migrations\EventListener\SchemaFilterListener">
+            <tag name="kernel.event_listener" event="console.command" method="onConsoleCommand" />
+            <tag name="doctrine.dbal.schema_filter" />
+        </service>


This is the part I am not sure about, because I have been enable to test it. I've tried accessing the schema filter manager definition in order to inspect it after the compiler pass, like this:

public function testItRegistersTheSchemaFilterListener(): void { $container = $this->getContainer([]); $container->getDefinition('doctrine.dbal.schema_asset_filter_manager')->setPublic(true); $container->compile(); $container->get('doctrine.dbal.schema_asset_filter_manager'); }

That last line results in

Symfony\Component\DependencyInjection\Exception\ServiceNotFoundException: The "doctrine.dbal.schema_asset_filter_manager" service or alias has been removed or inlined when the container was compiled. You should either make it public, or stop using the container directly and use dependency injection instead.

Any ideas how to test this?

doctrine.dbal.schema_asset_filter_manager is defined as an abstract. The container builder removes unused and/or abstract definitions during its compilation causing the service to be hidden.

An example fix, not per se the wanted fix, could be:

$container ->getDefinition('doctrine.dbal.schema_asset_filter_manager') ->setAbstract(false) ->setArgument(0, []) ->setPublic(true);

OK… I guess since it's abstract, I should try to retrieve a concrete definition inheriting from it. I inspected the container, and there does not seem to be any :(

greg0ire · 2024-03-14T19:09:30Z

Cc @FlyingDR :)

christian-kolb · 2025-01-10T07:05:09Z

Anything I can do to get this over the finish line? Would love to be able to remove the workaround we've got in our app 🙂

greg0ire · 2025-01-10T10:53:33Z

You could test it on your application and confirm whether it works or not.

christian-kolb · 2025-01-10T13:52:48Z

@greg0ire Ok, I will. Are the conflicts which are mentioned by the PR going to be a stumbling block for it?

christian-kolb · 2025-01-10T14:12:18Z

@greg0ire Unfortunately it already fails on cache:clear:

 php bin/console cache:clear
[2025-01-10T15:06:13.071685+01:00] console.CRITICAL: Error thrown while running command "cache:clear". Message: "Class "Doctrine\Migrations\EventListener\SchemaFilterListener" not found" {"exception":"[object] (Error(code: 0): Class \"Doctrine\\Migrations\\EventListener\\SchemaFilterListener\" not found at /var/www/html/api/var/cache/dev/ContainerXRKMlYx/App_KernelDevDebugContainer.php:703)","command":"cache:clear","message":"Class \"Doctrine\\Migrations\\EventListener\\SchemaFilterListener\" not found"} []
[2025-01-10T15:06:13.071685+01:00] console.CRITICAL: Error thrown while running command "cache:clear". Message: "Class "Doctrine\Migrations\EventListener\SchemaFilterListener" not found" {"exception":"[object] (Error(code: 0): Class \"Doctrine\\Migrations\\EventListener\\SchemaFilterListener\" not found at /var/www/html/api/var/cache/dev/ContainerXRKMlYx/App_KernelDevDebugContainer.php:703)","command":"cache:clear","message":"Class \"Doctrine\\Migrations\\EventListener\\SchemaFilterListener\" not found"} []
[2025-01-10T15:06:13.072295+01:00] console.DEBUG: Command "cache:clear" exited with code "1" {"command":"cache:clear","code":1} []
[2025-01-10T15:06:13.072369+01:00] php.CRITICAL: Uncaught Error: Class "Doctrine\Migrations\EventListener\SchemaFilterListener" not found {"exception":"[object] (Error(code: 0): Class \"Doctrine\\Migrations\\EventListener\\SchemaFilterListener\" not found at /var/www/html/api/var/cache/dev/ContainerXRKMlYx/App_KernelDevDebugContainer.php:703)"} []
[2025-01-10T15:06:13.072369+01:00] php.CRITICAL: Uncaught Error: Class "Doctrine\Migrations\EventListener\SchemaFilterListener" not found {"exception":"[object] (Error(code: 0): Class \"Doctrine\\Migrations\\EventListener\\SchemaFilterListener\" not found at /var/www/html/api/var/cache/dev/ContainerXRKMlYx/App_KernelDevDebugContainer.php:703)"} []
Symfony\Component\ErrorHandler\Error\ClassNotFoundError^ {#1415
  #message: """
    Attempted to load class "SchemaFilterListener" from namespace "Doctrine\Migrations\EventListener".\n
    Did you forget a "use" statement for "Doctrine\Bundle\MigrationsBundle\EventListener\SchemaFilterListener"?
    """
  #code: 0
  #file: "./var/cache/dev/ContainerXRKMlYx/App_KernelDevDebugContainer.php"
  #line: 703
  trace: {
    ./var/cache/dev/ContainerXRKMlYx/App_KernelDevDebugContainer.php:703 {
      ContainerXRKMlYx\App_KernelDevDebugContainer::getDoctrine_Dbal_DefaultConnectionService($container)^
      › $b->setSchemaManagerFactory(($container->privates['doctrine.dbal.legacy_schema_manager_factory'] ??= new \Doctrine\DBAL\Schema\LegacySchemaManagerFactory()));
      › $b->setSchemaAssetsFilter(new \Doctrine\Bundle\DoctrineBundle\Dbal\SchemaAssetsFilterManager([new \Doctrine\Bundle\DoctrineBundle\Dbal\RegexSchemaAssetFilter('~^(?!(?:domain_messages)|(?:domain_snapshots)|(?:cache_items)|(?:projection_handler_event_lock))~'), ($container->privates['doctrine_migrations.schema_filter_listener'] ??= new \Doctrine\Migrations\EventListener\SchemaFilterListener())]));
      › $b->setMiddlewares([new \Doctrine\DBAL\Logging\Middleware(($container->privates['monolog.logger.doctrine'] ?? self::getMonolog_Logger_DoctrineService($container))), $c]);
    }
    ./var/cache/dev/ContainerXRKMlYx/App_KernelDevDebugContainer.php:1379 { …}
    ./var/cache/dev/ContainerXRKMlYx/App_KernelDevDebugContainer.php:1363 { …}
    ./var/cache/dev/ContainerXRKMlYx/App_KernelDevDebugContainer.php:653 { …}
    ./var/cache/dev/ContainerXRKMlYx/App_KernelDevDebugContainer.php:556 { …}
    ./var/cache/dev/ContainerXRKMlYx/getConsoleProfilerListenerService.php:23 { …}
    ./var/cache/dev/ContainerXRKMlYx/App_KernelDevDebugContainer.php:506 { …}
    ./var/cache/dev/ContainerXRKMlYx/App_KernelDevDebugContainer.php:768 { …}
    ./vendor/symfony/event-dispatcher/EventDispatcher.php:221 { …}
    ./vendor/symfony/event-dispatcher/EventDispatcher.php:70 { …}
    ./vendor/symfony/event-dispatcher/Debug/TraceableEventDispatcher.php:252 { …}
    ./vendor/symfony/event-dispatcher/Debug/TraceableEventDispatcher.php:116 { …}
    ./vendor/symfony/console/Application.php:1044 { …}
    ./vendor/symfony/framework-bundle/Console/Application.php:123 { …}
    ./vendor/symfony/console/Application.php:316 { …}
    ./vendor/symfony/framework-bundle/Console/Application.php:77 { …}
    ./vendor/symfony/console/Application.php:167 { …}
    ./bin/console:38 { …}
  }
}

The service is mentioned in services.xml as Doctrine\Migrations\EventListener\SchemaFilterListener but the namespace in the class seems to be Doctrine\Bundle\MigrationsBundle\EventListener.

Fixing it to be the following seems to be necessary:

<service id="doctrine_migrations.schema_filter_listener" class="Doctrine\Bundle\MigrationsBundle\EventListener\SchemaFilterListener">

But even with this added, when running: php bin/console doctrine:schema:update --dump-sql it results in:

DROP TABLE migration_versions;

christian-kolb · 2025-01-10T14:13:07Z

@greg0ire Is there anything that needs to be configured for it to work?

greg0ire · 2025-01-10T14:28:53Z

Ok, I will. Are the conflicts which are mentioned by the PR going to be a stumbling block for it?

No.

Thanks for your suggestion, I applied it. I will need to fix the conflicts for this to get merged though.

Is there anything that needs to be configured for it to work?

No, I don't think so…

greg0ire · 2025-01-11T10:50:05Z


-        if (! isset($config['services'][MetadataStorage::class])) {
+        if (isset($config['services'][MetadataStorage::class])) {
+            $container->removeDefinition('doctrine_migrations.schema_filter_listener');


When a user defines their own storage class, there is no telling what they are going to do… they might even not store it in a database. At this point they're one their own.

christian-kolb · 2025-01-11T12:17:56Z

@greg0ire Awesome, now it's working. Now I get what I should get:

[OK] Nothing to update - your database is already in sync with the current entity metadata.

greg0ire · 2025-01-11T12:36:49Z

Thanks for helping moving this forward! Let's get this reviewed now.

SenseException · 2025-01-15T23:14:04Z

+    public function testItFiltersNothingWhenDisabled(): void
+    {
+        $listener = new SchemaFilterListener('doctrine_migration_versions');
+        $listener->disable();
+
+        self::assertTrue($listener(new Table('doctrine_migration_versions')));
+        self::assertTrue($listener(new Table('some_other_table')));
+    }


I don't understand this use case. When do you need to disable it in userland by calling the public disable-method?

🤔 I suspect I might have written that one before writing the next use case. I'll remove it and make the method private, if it's needed in userland, people can make it public later.

When using orm:schema-tool:update together with --complete, all assets not described by the current metadata is dropped. This is an issue when using doctrine/migrations, because not using that option is deprecated, and because when it is used, the table that holds the migrations is dropped as well since it is not described by ORM metadata. A solution to that is configuring an asset filter that filters out the metadata table except when running commands inside the migrations namespace.

kevinpapst · 2025-01-18T00:03:39Z

EDIT: posted as new bug report in #586

haxus01 · 2025-01-24T11:37:46Z

-            if ($storageConfiguration['table_name'] !== null) {
+            if ($storageConfiguration['table_name'] === null) {
+                $filterDefinition->addArgument('doctrine_migration_versions');
+            } else {


After implementing these changes, running the migration results in the following exception:
SQLSTATE[42P07]: Duplicate table: 7 ERROR: relation "doctrine_migration_versions" already exists.

greg0ire commented Mar 14, 2024

View reviewed changes

greg0ire force-pushed the complete-compat branch 2 times, most recently from feca975 to 2cfbb72 Compare March 14, 2024 16:29

greg0ire changed the base branch from 3.3.x to 3.4.x March 14, 2024 16:32

greg0ire force-pushed the complete-compat branch 2 times, most recently from 8ad6f64 to 656c9ee Compare March 14, 2024 16:45

greg0ire mentioned this pull request Mar 14, 2024

doctrine:schema:validate returns DROP TABLE doctrine_migration_versions; doctrine/migrations#1406

Closed

greg0ire force-pushed the complete-compat branch from 656c9ee to c356a12 Compare March 15, 2024 06:45

greg0ire commented Jan 10, 2025

View reviewed changes

Comment thread Resources/config/services.xml Outdated

greg0ire commented Jan 10, 2025

View reviewed changes

Comment thread EventListener/SchemaFilterListener.php Outdated

greg0ire force-pushed the complete-compat branch from 8d4e01b to 2e7a3f7 Compare January 11, 2025 10:48

greg0ire commented Jan 11, 2025

View reviewed changes

greg0ire force-pushed the complete-compat branch from 2e7a3f7 to 065d980 Compare January 11, 2025 10:51

greg0ire marked this pull request as ready for review January 11, 2025 12:35

greg0ire requested review from SenseException, derrabus and stof January 11, 2025 12:35

greg0ire added the Improvement label Jan 11, 2025

SenseException requested changes Jan 15, 2025

View reviewed changes

greg0ire force-pushed the complete-compat branch from 065d980 to c85cd25 Compare January 16, 2025 07:38

greg0ire requested a review from SenseException January 16, 2025 07:41

SenseException approved these changes Jan 16, 2025

View reviewed changes

greg0ire merged commit 45d1b74 into doctrine:3.4.x Jan 16, 2025

greg0ire deleted the complete-compat branch January 16, 2025 20:19

greg0ire added this to the 3.4.0 milestone Jan 16, 2025

github-actions Bot mentioned this pull request Jan 16, 2025

Merge release 3.4.0 into 4.0.x #581

Merged

greg0ire mentioned this pull request Jan 21, 2025

add filterAssetNames to get asset tables doctrine/dbal#6729

Closed

haxus01 reviewed Jan 24, 2025

View reviewed changes

Uh oh!

Conversation

greg0ire commented Mar 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How can I test this?

Uh oh!

greg0ire Mar 14, 2024

Choose a reason for hiding this comment

Uh oh!

wmouwen Mar 17, 2024

Choose a reason for hiding this comment

Uh oh!

greg0ire Jan 11, 2025

Choose a reason for hiding this comment

Uh oh!

greg0ire commented Mar 14, 2024

Uh oh!

christian-kolb commented Jan 10, 2025

Uh oh!

greg0ire commented Jan 10, 2025

Uh oh!

christian-kolb commented Jan 10, 2025

Uh oh!

christian-kolb commented Jan 10, 2025

Uh oh!

christian-kolb commented Jan 10, 2025

Uh oh!

Uh oh!

greg0ire commented Jan 10, 2025

Uh oh!

Uh oh!

greg0ire Jan 11, 2025

Choose a reason for hiding this comment

Uh oh!

christian-kolb commented Jan 11, 2025

Uh oh!

greg0ire commented Jan 11, 2025

Uh oh!

Uh oh!

SenseException Jan 15, 2025

Choose a reason for hiding this comment

Uh oh!

greg0ire Jan 16, 2025

Choose a reason for hiding this comment

Uh oh!

kevinpapst commented Jan 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

haxus01 Jan 24, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

greg0ire commented Mar 14, 2024 •

edited

Loading

kevinpapst commented Jan 18, 2025 •

edited

Loading