Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

DanielSiepmann
Copy link
Contributor

@DanielSiepmann DanielSiepmann commented Feb 14, 2024

The typo3/testing-framework creates functional test instances
as "classic" mode instances, writing a PackageStates.php file
composed using composer information, provided and handled by the
internal ComposerPackageManager class. Extensions in the state
file are not sorted based on their dependencies and suggestions.

To mitigate this and having a correctly sorted extension state
file, the PackageCollection service has been introduced with
the goal to resort the extension state file after the initial
write to provide the correct sorted extension state.

The extension sorting within the state file is important, as the
TYPO3 Core uses this sorting to loop over extensions to collect
information from the extensions, for example TCA, TCAOverrides,
ext_localconf.php and other things. Package sorting is here very
important to allow extensions to reconfigure or change the stuff
from other extensions, which is guaranteed in normal "composer"
and "classic" mode instances.

For "classic" mode instances, only the ext_emconf.php file is
taken into account and "composer.json" as the source of thruth
for "composer" mode instances since TYPO3 v12, which made the
ext_emconf.php file obsolete for extensions only installed
with composer.

Many agencies removed the optional ext_emconf.php file for
project local path extensions like a sitepackage to remove
the maintenance burden, which is a valid case.

Sadly, the PackageCollection adopted from the TYPO3 Core
PackageManager did not reflected this case and failed for
extensions to properly sort only on extension dependencies
and keys due the mix of extension key and composer package
name handling. Extension depending on another extension
failed to be sorted correctly with following exception:

UnexpectedValueException: The package "extension_key"
depends on "composer/package-key" which is not present
in the system.

This change modifies the PackageCollection implementation
to take only TYPO3 extension and system extension into account
for dependency resolving and sorting, using ext_emconf.php
depends/suggest information as first source and falling back
to composer require and suggestion information.

Resolves: #541
Releases: main, 8

@oliverklee
Copy link
Contributor

Should this only go into the main branch, or would it make sense to backport it to 7.x as well?

@DanielSiepmann
Copy link
Contributor Author

I'm fine with both. I personally need this in one project where we apply this as a patch.
I'd leave this up to the maintainer to decide.

@DanielSiepmann DanielSiepmann marked this pull request as draft February 15, 2024 06:38
@DanielSiepmann
Copy link
Contributor Author

Looks like there is somehow an issue with loading order within acceptance tests … I'll investiage.

@DanielSiepmann DanielSiepmann force-pushed the 541-support-composer-only branch 2 times, most recently from 912733d to db0d6a2 Compare February 15, 2024 07:57
@DanielSiepmann DanielSiepmann marked this pull request as ready for review February 15, 2024 07:58
@liayn
Copy link
Contributor

liayn commented Oct 14, 2024

Hm... Feature?
I would rather call this a serious bugfix...

All of our internal extensions skip emconf (obviously) so this hits us on every upgrade from v11 to v12 of each extension. :-(

@sbuerk sbuerk force-pushed the 541-support-composer-only branch from db0d6a2 to 2304f73 Compare October 15, 2024 09:06
@sbuerk
Copy link
Collaborator

sbuerk commented Oct 15, 2024

on it ..

@sbuerk
Copy link
Collaborator

sbuerk commented Oct 15, 2024

I guess there is more work to do, and need more time to investigate this further. So no quick merge of this (not saying that it is a bad change !).

Thanks @DanielSiepmann for a first solution suggestion and the tests !!!! That helps to investigate that more in detail.

Continue on this the next days. Stay tuned !

@liayn
Copy link
Contributor

liayn commented Oct 15, 2024

@sbuerk Thanks a bunch. I just wonder what we changed in Core v12 that this hits so hard? I mean this worked flawlessly - at least for us - with Core v11 and TF 7

@sbuerk
Copy link
Collaborator

sbuerk commented Oct 15, 2024

@sbuerk Thanks a bunch. I just wonder what we changed in Core v12 that this hits so hard? I mean this worked flawlessly - at least for us - with Core v11 and TF 7

TF8 added the sorting of packages to ensure that extension in functional instances has the correct sorting, normally TYPO3 would take care of in a legacy instance. As the package state is written by tf after symlinking to it, the state file must be sorted. For that, a custom sorting thing has been implemented in TF to take care of that.

With TF7 / v11 or v12 you have to take care to mention extension in the correct order yourself.

For example, of you have a extension which requries to be executed after another extension. For example having suggest defined so your extension runs after for example EXT:dashboard to register widgets etc.

The PackageCollection just assums (use code which assums) that extension still have ext_emconf.php in the extension.

There are lot more details to this, but we need to address some things in this PackageCollection now.

@sbuerk sbuerk force-pushed the 541-support-composer-only branch from 2304f73 to f0aa291 Compare November 11, 2024 20:06
@sbuerk sbuerk force-pushed the 541-support-composer-only branch from f0aa291 to 312d5a0 Compare November 27, 2024 08:48
The `typo3/testing-framework` creates functional test instances
as "classic" mode instances, writing a `PackageStates.php` file
composed using composer information, provided and handled by the
internal `ComposerPackageManager` class. Extensions in the state
file are not sorted based on their dependencies and suggestions.

To mitigate this and having a correctly sorted extension state
file, the `PackageCollection` service has been introduced with
the goal to resort the extension state file after the initial
write to provide the correct sorted extension state.

The extension sorting within the state file is important, as the
TYPO3 Core uses this sorting to loop over extensions to collect
information from the extensions, for example TCA, TCAOverrides,
ext_localconf.php and other things. Package sorting is here very
important to allow extensions to reconfigure or change the stuff
from other extensions, which is guaranteed in normal "composer"
and "classic" mode instances.

For "classic" mode instances, only the `ext_emconf.php` file is
taken into account and "composer.json" as the source of thruth
for "composer" mode instances since TYPO3 v12, which made the
`ext_emconf.php` file obsolete for extensions only installed
with composer.

Many agencies removed the optional `ext_emconf.php` file for
project local path extensions like a sitepackage to remove
the maintenance burden, which is a valid case.

Sadly, the `PackageCollection` adopted from the TYPO3 Core
`PackageManager` did not reflected this case and failed for
extensions to properly sort only on extension dependencies
and keys due the mix of extension key and composer package
name handling. Extension depending on another extension
failed to be sorted correctly with following exception:

    UnexpectedValueException: The package "extension_key"
    depends on "composer/package-key" which is not present
    in the system.

This change modifies the `PackageCollection` implementation
to take only TYPO3 extension and system extension into account
for dependency resolving and sorting, using `ext_emconf.php`
depends/suggest information as first source and falling back
to `composer` require and suggestion information.

Resolves: TYPO3#541
Releases: main, 8
@sbuerk sbuerk force-pushed the 541-support-composer-only branch from 312d5a0 to 626f37f Compare November 27, 2024 16:22
@sbuerk sbuerk changed the title [FEATURE] Add support for composer.json only extensions [BUGFIX] Respect composer mode only extension in FunctionalTestCase Nov 27, 2024
@sbuerk
Copy link
Collaborator

sbuerk commented Nov 27, 2024

@DanielSiepmann Modified your commit/pull-request.

Note, testest the very same commit with #634 against TYPO3 Core with https://review.typo3.org/c/Packages/TYPO3.CMS/+/87253 and also
in a nightly.

Is the change okay for you ?

@d-s-codappix
Copy link

Sure. I've replaced all ext_emconf.php files of our test fixture extensions with corresponding composer.json files and tested that state with the latest version of this PR. This version works fine for us in our v12 project with TF 8.

@sbuerk sbuerk merged commit c3d94c4 into TYPO3:main Nov 28, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for dependencies to TYPO3 Extensions without ext_emconf.php
6 participants