Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Dependency Injection inifinite loop handling circular references consuming memory #39015

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
soyuka opened this issue Nov 6, 2020 · 4 comments
Closed

Dependency Injection inifinite loop handling circular references consuming memory #39015

soyuka opened this issue Nov 6, 2020 · 4 comments
Labels
Bug DependencyInjection Status: Needs Review

Comments

@soyuka
Copy link
Contributor

soyuka commented Nov 6, 2020 •
edited
Loading

Symfony version(s) affected: 5.2.x

Description
Clearing the cache with latest symfony breaks on api-platform/core

How to reproduce

git clone [email protected]:api-platform/core
cd core
# If you have moreutils, or just add the minimum-stability dev to the composer.json
jq '. + {"minimum-stability": "dev"}' composer.json | sponge composer.json
composer install
php tests/Fixtures/app/console cache:warmup -vv

This will lead to an out of memory error (be sure you don't use memory_limit=-1)

Possible Solution

Revert the code changed in PhpDumper (see https://github.com/symfony/symfony/pull/38980/files#diff-82d7ebdd6477cecb404b0d3f100a9026e950529edb9a665460ae0330dfad7080L454)

        foreach ($edges as $edge) {
            $node = $edge->getDestNode();
            $id = $node->getId();
+            if (!$node->getValue() instanceof Definition || $sourceId === $id || $edge->isLazy() || $edge->isWeak()) {
-            if (!$node->getValue() instanceof Definition || $sourceId === $id || $edge->isWeak()) {
                continue;
            }

If you reproduced and want to try the fix, just rm -rf tests/Fixtures/app/var/cache/ and use the command again.

ref: #38980
@soyuka soyuka added the Bug label Nov 6, 2020
@soyuka soyuka mentioned this issue Nov 6, 2020
Merged
@jderusse jderusse mentioned this issue Nov 6, 2020
Closed
@jderusse
Copy link
Member

jderusse commented Nov 6, 2020

I optimized the way the loop are detected (see #39019 )
Tell me of it fixes your issue.

Note: this is not an infinite loops, but an exponential loop that consume a huge amount of memory

@jderusse jderusse mentioned this issue Nov 6, 2020
Merged
@soyuka
Copy link
Contributor Author

soyuka commented Nov 8, 2020 •
edited
Loading

Note: this is not an infinite loops, but an exponential loop that consume a huge amount of memory

Using 8gb of memory also breaks on my issue.

The patch in #39021 does not fix the issue.

Some more details on the error:

php -dmemory_limit=1g tests/Fixtures/app/console cache:warmup --no-optional-warmers -vvv

PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 262144 bytes) in /home/soyuka/forks/core/tests/Fixtures/app/var/cache/test/ContainerKApm16H/EntityManager_9a5be93.php on line 163
Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 262144 bytes) in /home/soyuka/forks/core/tests/Fixtures/app/var/cache/test/ContainerKApm16H/EntityManager_9a5be93.php on line 163

The code there looks like this:

    public function getConfiguration()
    {
        $this->initializer6977e && ($this->initializer6977e->__invoke($valueHoldera8e22, $this, 'getConfiguration', array(), $this->initializer6977e) || 1) && $this->valueHoldera8e22 = $valueHoldera8e22;
        return $this->valueHoldera8e22->getConfiguration();
    }

It's always the same EntityManager related code that consumes memory indefinitely.

@jderusse
Copy link
Member

jderusse commented Nov 8, 2020 •
edited
Loading

Hmm, this breaks since #3812 because now, a lazy service used a proxy. Which end in an infinity loop where the proxy is injected inside the proxy.

this is fixed by https://github.com/symfony/symfony/compare/85d8004602f0b4e18999ade5cb8ecabfa6a06f42..d3b951b767e70a40c80ddbb9ebfcfe7518f14c81 could you give a try @soyuka ?

@nicolas-grekas this second issue, shows that some lazy services MUST stop the loop detection, do you have other similar case in mind?

@soyuka
Copy link
Contributor Author

soyuka commented Nov 8, 2020 •
edited
Loading

https://github.com/symfony/symfony/compare/85d8004602f0b4e18999ade5cb8ecabfa6a06f42..d3b951b767e70a40c80ddbb9ebfcfe7518f14c81 fixes the issue.

@fabpot fabpot closed this as completed Nov 9, 2020
fabpot added a commit that referenced this issue Nov 9, 2020
@fabpot
bug #39021 [DependencyInjection] Optimize circular collection by remo…
0ae674a 
…ving flattening (jderusse)

This PR was merged into the 4.4 branch.

Discussion
----------

 [DependencyInjection] Optimize circular collection by removing flattening

| Q             | A
| ------------- | ---
| Branch?       | 4.4
| Bug fix?      | yes
| New feature?  | no
| Deprecations? | no
| Tickets       | Fix #39015
| License       | MIT
| Doc PR        | -

Alternative to #39019

Commits
-------

e649f47 Optimize circular collection by removing flattening
@jderusse jderusse mentioned this issue Nov 11, 2020
Merged
@PhilETaylor PhilETaylor mentioned this issue Nov 11, 2020
Closed
@mmarton mmarton mentioned this issue Nov 13, 2020
Closed
@jderusse jderusse mentioned this issue Nov 20, 2020
Merged
nicolas-grekas added a commit that referenced this issue Nov 27, 2020
@nicolas-grekas
bug #39129 [DependencyInjection] Fix circular in DI with lazy + byCon…
7d7fa3e 
…truct loop (jderusse)

This PR was squashed before being merged into the 4.4 branch.

Discussion
----------

[DependencyInjection] Fix circular in DI with lazy + byContruct loop

| Q             | A
| ------------- | ---
| Branch?       | 4.4
| Bug fix?      | yes
| New feature?  | no
| Deprecations? | no
| Tickets       | Fix #39120
| License       | MIT
| Doc PR        | -

This fix another issue lazy service.
It partially revert #38980 and #39021

Initially, we trusted lazy services to be lazy and not beeing called while building the services graph
=> bug #38970 when lazy deps is injected in a factory, it may be consumed directly to build the object before the graph is fully built
Fixed by #38980 => lazy service are considered as "normal service"
=> bug #39015 some loop are not resolvable with "normal service", but it shouldn't be an issue when servie proxifyied
Fixed by #39021 => lazy service are considered as "normal service" except when proxyfied
=> bug #39120 some loop are not resolvable with "normal service", but it shouldn't be an issue because the lazy service is injected in the constructor and user

Fixed by this PR => that revert to the initial state. lazy service are trusted.
But now, The IterratorArgument injected in a factory (single exception) is not more considered as lazy

Commits
-------

54af139 [DependencyInjection] Fix circular in DI with lazy + byContruct loop
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug DependencyInjection Status: Needs Review
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@fabpot @jderusse @soyuka @xabbuh @carsonbot