[DependencyInjection] Self-referenced 'service_container' service breaks garbage collection #11422

sun · 2014-07-19T19:14:17Z

Q	A
Bug fix?	yes
New feature?	no
BC breaks?	no
Deprecations?	no
Tests pass?	yes
Needs merge to	2.4
Fixed tickets	—
License	MIT
Doc PR	—

Problem

Container::__construct() sets the service_container service ID to a self-reference of $this.
This breaks PHP garbage collection (as well as clone).

Background

As explained in Reference Counting Basics, PHP is not able to destruct and garbage collect variables that contain a reference to itself, because the refcount is always > 0.

Example Use-Case

A precompiled container that is cloned for multiple tests (because ::compile() can be slow).

$precompiled = new ContainerBuilder();
// ...
$precompiled->compile();
foreach ($precompiled->getServiceIds() as $id) {
  if ($precompiled->initialized($id)) {
    $precompiled->set($id, null);
  }
}
$precompiled_clean = clone $precompiled;
$precompiled = NULL;
// ...
foreach ($tests as $test) {
  $container = clone $precompiled_clean;
  // Required without this PR:
  $container->set('service_container', $container);
  // ...
}

Actual result [master]:
All references to the original $precompiled container are not destructed and cleaned up, because $precompiled still references itself, which can cause terribly hard to debug defects due to stale reference pointers in memory (especially with ContainerAware services, or things like e.g. PDO connections), as well as high memory consumption, etc.
Expected result [PR]:
No references to $precompiled are left; everything is shut down, cleaned up, and garbage-collected upon $precompiled = NULL;

Proposed solution

Hard-code a special behavior for the service ID service_container.
Remove the self-reference.
For now (until next major release), ensure that all Container methods work identically to before — instead of throwing exceptions when e.g. trying to set service_container to something that isn't $this… (which is technically possible right now, but the operation doesn't remotely make sense)

API changes

Per (3) above, this PR is backwards-compatible. However, for the sake of full disclosure, users may experience the following super-micro changes compared to HEAD:

The following method no longer exists in a dumped Container, because it is no longer a registered service:

    protected function getServiceContainerService()
    {
        throw new RuntimeException('You have requested a synthetic service ("service_container"). The DIC does not know how to construct this service.');
    }

But given the exception, you weren't able to call it anyway.

The order/position of service_container in the array returned by Container::getServiceIds() may be different.

But that array has no particular order to begin with.

(Only the component's own unit tests are expecting an identical result at times, which is why I added the hard-coded ID via $id[] last, instead of initializing it first with $id already.)
$container->set('service_container', $not_the_container); no longer works.

sun · 2014-07-19T19:34:28Z

FWIW, PhpDumper uses similar shortcuts for service_container → $this already, but PhpDumper::addFrozenConstructor() still contains the following:

        $code .= <<<EOF
…
        \$this->set('service_container', \$this);

I'm not sure whether it is safe to also remove that call from the dumped constructor. It is a no-op for Container::set(), but not necessarily for custom implementations.

sun · 2014-07-19T20:20:47Z

Travis test failure (only on 5.5) is not related to this PR. The second to last Travis build succeeded.

See symfony/symfony#11422

sun · 2014-07-24T17:52:18Z

Hm, I thought this would be a straightforward improvement/quick fix… 😉 Curious, did someone look at this already?

fabpot · 2014-07-25T14:56:32Z

src/Symfony/Component/DependencyInjection/Container.php

@@ -261,6 +269,9 @@ public function has($id)
     */
    public function get($id, $invalidBehavior = self::EXCEPTION_ON_INVALID_REFERENCE)
    {
+        if ($id === 'service_container') {


Unfortunately, the service id is case insensitive (see code after your changes.) I don't think people are using anything else, but that's a slight BC break... which we can probably ignore.

Any performance impact?

Yeah, I wasn't sure whether it's worth to move the condition into the loop. For maximum BC, I guess we should…

Comparing a string variable with a fixed static string is hardly measurable, in the realm of microseconds.
(btw, would be great to remove all of the strtolower() futzing in an upcoming release; the total cost of that can be up to 100ms…)

Moved the condition into the loop now.

fabpot · 2014-07-25T14:56:54Z

Looks good to me.

fabpot · 2014-07-25T15:01:44Z

This could be merged in 2.3.

xabbuh · 2014-07-26T09:49:10Z

src/Symfony/Component/DependencyInjection/Container.php

@@ -197,6 +195,12 @@ public function set($id, $service, $scope = self::SCOPE_CONTAINER)

        $id = strtolower($id);

+        if ($id === 'service_container') {


Should be 'service_container' === $id (and below too), shouldn't it?

Sorry, wasn't aware that Symfony enforces yoda conditions everywhere; fixed.

…llection (and clone).

See symfony/symfony#11422

fabpot · 2014-08-02T06:04:32Z

👍

fabpot · 2014-08-02T06:06:17Z

Thank you @sun.

…service breaks garbage collection (sun) This PR was submitted for the master branch but it was merged into the 2.3 branch instead (closes #11422). Discussion ---------- [DependencyInjection] Self-referenced 'service_container' service breaks garbage collection | Q | A | ------------- | --- | Bug fix? | yes | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Needs merge to | 2.4 | Fixed tickets | — | License | MIT | Doc PR | — #### Problem 1. `Container::__construct()` sets the `service_container` service ID to a self-reference of `$this`. 1. This breaks PHP garbage collection (as well as `clone`). #### Background * As explained in [Reference Counting Basics](http://php.net/manual/en/features.gc.refcounting-basics.php), PHP is not able to destruct and garbage collect variables that contain a reference to itself, because the `refcount` is always > 0. #### Example Use-Case A precompiled container that is cloned for multiple tests (because `::compile()` can be slow). ```php $precompiled = new ContainerBuilder(); // ... $precompiled->compile(); foreach ($precompiled->getServiceIds() as $id) { if ($precompiled->initialized($id)) { $precompiled->set($id, null); } } $precompiled_clean = clone $precompiled; $precompiled = NULL; // ... foreach ($tests as $test) { $container = clone $precompiled_clean; // Required without this PR: $container->set('service_container', $container); // ... } ``` * **Actual result [master]:** All references to the original `$precompiled` container are not destructed and cleaned up, because `$precompiled` still references itself, which can cause terribly hard to debug defects due to stale reference pointers in memory (especially with `ContainerAware` services, or things like e.g. PDO connections), as well as high memory consumption, etc. * **Expected result [PR]:** No references to `$precompiled` are left; everything is shut down, cleaned up, and garbage-collected upon `$precompiled = NULL;` #### Proposed solution 1. Hard-code a special behavior for the service ID `service_container`. 1. Remove the self-reference. 1. For now (until next major release), ensure that all `Container` methods work identically to before — instead of throwing exceptions when e.g. trying to set `service_container` to something that isn't `$this`… (which is technically possible right now, but the operation doesn't remotely make sense) #### API changes Per (3) above, this PR is backwards-compatible. However, for the sake of full disclosure, users may experience the following super-micro changes compared to HEAD: 1. The following method no longer exists in a dumped Container, because it is no longer a registered service: ```php protected function getServiceContainerService() { throw new RuntimeException('You have requested a synthetic service ("service_container"). The DIC does not know how to construct this service.'); } ``` But given the exception, you weren't able to call it anyway. 1. The order/position of `service_container` in the array returned by `Container::getServiceIds()` may be different. But that array has no particular order to begin with. (Only the component's own unit tests are expecting an identical result at times, which is why I added the hard-coded ID via `$id[]` last, instead of initializing it first with `$id` already.) 1. `$container->set('service_container', $not_the_container);` no longer works. Commits ------- 440322e Fixed self-reference in 'service_container' service breaks garbage collection (and clone).

hacfi · 2015-02-08T06:52:59Z

Is there a PR already to fix the code marked as @todo?

jakzal · 2015-02-08T12:42:07Z

@hacfi there's none yet.

… to allow garbage collection Follow up of symfony#11422

This issue has been fixed in symfony#11422. Due to a bad merge in 3bed1b7 it partially appeared again in Symfony 2.6 or higher.

This PR was merged into the 2.6 branch. Discussion ---------- [DependencyInjection] resolve circular reference | Q | A | ------------- | --- | Bug fix? | yes | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | #11422, #14445 | License | MIT | Doc PR | This issue has been fixed in #11422. Due to a bad merge in 3bed1b7 it partially appeared again in Symfony 2.6 or higher. Commits ------- 8b3b3ce [DependencyInjection] resolve circular reference

This PR was merged into the 2.3 branch. Discussion ---------- [DependencyInjection] Fixed missing tests | Q | A | ------------- | --- | Bug fix? | no | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | - | License | MIT | Doc PR | - Follow up of #11422 and #14446 Commits ------- 2892902 Fixed tests

githoober · 2015-09-16T04:04:26Z

I am getting Uncaught exception 'Symfony\Component\DependencyInjection\Exception\InvalidArgumentException' with message 'The service definition "service_container" does not exist in \Symfony\Component\DependencyInjection\ContainerBuilder::getDefinition when injecting a service with an optional dependency on a 'service_container' into another service. Using the service with this dependency directly works.

<service id="override" class="Override">
  <!--- this call breaks -->
        <call method="setExplicitTransactionMediator">
            <argument type="service" id="explicit_transaction_mediator"/>
        </call>
</service>

<service id="explicit_transaction_mediator" class="ExplicitTransactionMediator" public="false">
        <argument type="collection">
            <argument type="constant">ExplicitTransactionMediator::EVENT_ON_COMMIT</argument>
        </argument>
        <call method="addCallback">
            <argument type="constant">ExplicitTransactionMediator::EVENT_ON_COMMIT</argument>
            <argument type="collection">
                <argument type="service" id="explicit_transaction_colleague"/>
                <argument type="string">postFlush</argument>
            </argument>
        </call>
</service>

<service id="explicit_transaction_colleague" class="ExplicitTransactionColleague">
        <argument type="collection">
            <argument type="service" id="change_listener"/>
        </argument>
 </service>

<service class="Change" id="change_listener">
        <argument type="service" id="entitymanager"/>
        <argument type="service" id="logger"/>
        <argument type="service" id="config"/>
        <call method="setContainer">
            <argument type="service" id="service_container"/>
        </call>
</service>

Is there a way to reference service container differently, so it does not break, or it is a bug introduced in this ticket?

stof · 2015-09-16T08:50:53Z

@githoober Can you give the stack trace of the exception ?

stof · 2015-09-16T08:51:20Z

And please open a dedicated issue for that instead of commenting on a closed PR

sun added a commit to sun/drupal that referenced this pull request Jul 20, 2014

Fixed precompiled container leaks into tests.

e59141a

See symfony/symfony#11422

sun added a commit to sun/drupal that referenced this pull request Jul 21, 2014

Fixed precompiled container leaks into tests.

de52e73

See symfony/symfony#11422

sun added a commit to sun/drupal that referenced this pull request Jul 21, 2014

Fixed precompiled container leaks into tests.

1ec587e

See symfony/symfony#11422

sun mentioned this pull request Jul 24, 2014

Allow a test to identify whether it runs in isolation sebastianbergmann/phpunit#1350

Closed

fabpot reviewed Jul 25, 2014
View reviewed changes

xabbuh reviewed Jul 26, 2014
View reviewed changes

Fixed self-reference in 'service_container' service breaks garbage co…

e017c38

…llection (and clone).

sun added a commit to sun/drupal that referenced this pull request Jul 26, 2014

Fixed precompiled container leaks into tests.

cec6f23

See symfony/symfony#11422

sun mentioned this pull request Jul 27, 2014

Allow a test to identify whether it runs in isolation sebastianbergmann/phpunit#1360

Merged

sun added the DependencyInjection label Jul 31, 2014

sun added a commit to sun/drupal that referenced this pull request Aug 1, 2014

Fixed precompiled container leaks into tests.

db9fd21

See symfony/symfony#11422

fabpot closed this Aug 2, 2014

hacfi added a commit to hacfi/symfony that referenced this pull request Apr 22, 2015

[DependencyInjection] Fix self-referenced 'service_container' service…

f1a8c1c

… to allow garbage collection Follow up of symfony#11422

hacfi mentioned this pull request Apr 22, 2015

[DependencyInjection] Fix self-referenced 'service_container' service #14445

Closed

xabbuh added a commit to xabbuh/symfony that referenced this pull request Apr 22, 2015

[DependencyInjection] resolve circular reference

8b3b3ce

This issue has been fixed in symfony#11422. Due to a bad merge in 3bed1b7 it partially appeared again in Symfony 2.6 or higher.

xabbuh mentioned this pull request Apr 22, 2015

[DependencyInjection] resolve circular reference #14446

Merged

dosten mentioned this pull request Apr 23, 2015

[DependencyInjection] Fixed missing tests #14457

Merged

		@@ -197,6 +195,12 @@ public function set($id, $service, $scope = self::SCOPE_CONTAINER)

		$id = strtolower($id);

		if ($id === 'service_container') {

Uh oh!

[DependencyInjection] Self-referenced 'service_container' service breaks garbage collection #11422

[DependencyInjection] Self-referenced 'service_container' service breaks garbage collection #11422

Uh oh!

Conversation

sun commented Jul 19, 2014

Problem

Background

Example Use-Case

Proposed solution

API changes

Uh oh!

sun commented Jul 19, 2014

Uh oh!

sun commented Jul 19, 2014

Uh oh!

sun commented Jul 24, 2014

Uh oh!

fabpot Jul 25, 2014

Choose a reason for hiding this comment

Uh oh!

fabpot Jul 25, 2014

Choose a reason for hiding this comment

Uh oh!

sun Jul 25, 2014

Choose a reason for hiding this comment

Uh oh!

sun Jul 25, 2014

Choose a reason for hiding this comment

Uh oh!

fabpot commented Jul 25, 2014

Uh oh!

fabpot commented Jul 25, 2014

Uh oh!

xabbuh Jul 26, 2014

Choose a reason for hiding this comment

Uh oh!

sun Jul 26, 2014

Choose a reason for hiding this comment

Uh oh!

fabpot commented Aug 2, 2014

Uh oh!

fabpot commented Aug 2, 2014

Uh oh!

hacfi commented Feb 8, 2015

Uh oh!

jakzal commented Feb 8, 2015

Uh oh!

githoober commented Sep 16, 2015

Uh oh!

stof commented Sep 16, 2015

Uh oh!

stof commented Sep 16, 2015

Uh oh!

Uh oh!