[DI] Check for privates before shared services #22866

ro0NL · 2017-05-23T07:38:25Z

Q	A
Branch?	3.2
Bug fix?	yes
New feature?	no
BC breaks?	no
Deprecations?	no
Tests pass?	yes
Fixed tickets	#22801 (comment), #22801 (comment)
License	MIT
Doc PR	symfony/symfony-docs#...

cc @stof

nicolas-grekas · 2017-05-23T07:47:25Z

CI is red :)

stof · 2017-05-23T08:27:42Z

@nicolas-grekas this is because of composer/composer#6342

ro0NL · 2017-05-23T08:29:31Z

Kinda clueless here... trying to fix WebProfilerExtensionTest::testToolbarConfig

patched assertSaneContainer so it skips private id's (hardcoded locally to test :)

however these 2 assertions pass for all provided debug modes, indiviudally;

        $this->assertSaneContainer($this->getDumpedContainer(), '', array('web_profiler.csp.handler'));

        if ($listenerInjected) {
            $this->assertSame($listenerEnabled, $this->container->get('web_profiler.debug_toolbar')->isEnabled());
        }

the combination seems to trigger a deprecation 😕

ro0NL · 2017-05-23T08:31:16Z

first patch.. lets see what happens :)

ro0NL · 2017-05-23T08:39:27Z

testStopwatchExtensionAvailability actually gets a private service 😠

$container->get('twig.extension.debug.stopwatch')->getTokenParsers()

is there a clean way to silence expected deprecations.. or should we do something else here.

@stof see deprecations at appveyor ;)

stof · 2017-05-23T09:06:52Z

well, the test should either not access a private service, or be marked as legacy. If the test is not meant to be removed in 4.0, marking it as legacy is not an option, and the test should be rewritten instead.
This is good that I catched the mistake in the deprecation trigger in the other PR, as this allowed us to catch mistakes in our own testsuite which was using deprecated APIs without getting the notice...

ro0NL · 2017-05-23T09:10:35Z

Updated.. this should do. But i dont like it really :) but i guess we have no choice as we never decided how privates reflects a compiled containerbuilder.

@stof to clarify.. the root issue is ContainerBuilder::get() calling parent::get

stof · 2017-05-23T09:12:40Z

src/Symfony/Bundle/TwigBundle/Tests/DependencyInjection/TwigExtensionTest.php

        $this->compileContainer($container);

-        $tokenParsers = $container->get('twig.extension.debug.stopwatch')->getTokenParsers();
+        $tokenParsers = $container->get('test_debug_stopwatch')->stopwatch->getTokenParsers();


a better solution would be to use $container->get('twig')->getExtension(StopwatchExtension::class) to access the extension, which avoids registering an extra service

seems to break stuff :/

1) Symfony\Bundle\TwigBundle\Tests\DependencyInjection\TwigExtensionTest::testStopwatchExtensionAvailability with data set "debug-and-stopwatch-enabled" (true, true, true) Symfony\Component\DependencyInjection\Exception\ServiceNotFoundException: You have requested a non-existent service "twig.loader".

reverted the whole change, as with privates being empty it keeps working :)

this test should fail whenever we decide to change behavior.

stof · 2017-05-23T09:14:16Z

src/Symfony/Component/DependencyInjection/Container.php

@@ -262,6 +262,10 @@ public function get($id, $invalidBehavior = self::EXCEPTION_ON_INVALID_REFERENCE
            if (isset($this->aliases[$id])) {
                $id = $this->aliases[$id];
            }
+            if (isset($this->privates[$id]) && !$this instanceof ContainerBuilder) {


why the check on ContainerBuilder ? this is useless ($this->privates will be empty for them anyway)

not sure.. https://github.com/symfony/symfony/blob/v3.2.8/src/Symfony/Component/DependencyInjection/ContainerBuilder.php#L577

but this definitely fixes tests :)

i think ContainerBuilder needs to behave the same as Container, after compilation. But that could be a separate PR i guess. The changelog doesnt mention ContainerBuilder really...

well, once compiled, the array is not empty anymore, but we do want the deprecation in this case. So please remove the extra check you added there (which was not there before moving the notice btw, so you are the one adding a difference in the behavior)

nicolas-grekas · 2017-05-23T09:32:55Z

I think we should move this to 3.3: it will trigger more notices in 3.2 apps and break some strict CI for no real benefit.

ro0NL · 2017-05-23T09:45:28Z

3.2 is truly buggy though.. (@stof's comment in previous PR) i think the ContainerBuilder check will prevent those unexpected notices actually; while fixing Container.

otherwise, could you just commit the changelog against 3.2 :)) but you probably prefer a separate PR for that :P

stof · 2017-05-23T09:55:12Z

@ro0NL the ContainerBuilder check does not solve anything in 3.2.
The issue was that we were missing deprecations when getting a private service which was already instantiated previously (due to being a dependency of another service you got earlier). This happens both for a ContainerBuilder and the dumped container.
Your check fixes the testsuite only because the testsuite does not do any such bad get calls on a dumped container (as most tests don't bother about dumping the container and then loading the dumped container to run assertions)

ro0NL · 2017-05-23T15:09:10Z

@stof im still a bit confused.. (ive reverted the containerbuilder checks).

moving the private check up (above checking for shared services), truly fixes your comments from #22801, i.e. https://github.com/symfony/symfony/pull/22866/files#diff-8aeba2b88d0347062017ee160a6beb49R408 fails otherwise.

imo. it needs a fix for 3.2 (properly deprecating private services), but im missing something regarding the container builder. AFAIK the issue is ContainerBuilder::get() => parent::get().

Updating this call to get() still triggers the deprecation for private service twig.extension.debug.stopwatch. Im curious whats the root issue here, or at least what the proper fix is for either 3.2 or 3.3.

ro0NL · 2017-05-23T15:20:50Z

Think this approach https://github.com/symfony/symfony/blob/v3.2.8/src/Symfony/Component/DependencyInjection/Dumper/PhpDumper.php#L1488 needs to be done in container builder after compilation... ie. bypasssing get() internally for privates.

nicolas-grekas · 2017-05-23T16:17:08Z

My 2 cts: ContainerBuilder should not care about private or not. Thus, calling parent::get() shouldn't matter because $this->privates should not be populated.
I don't know where things fail right now but if we stick to this policy and tests fail, then I'd say these tests are broken, nothing else.

nicolas-grekas · 2017-05-23T16:20:10Z

For me,

symfony/src/Symfony/Component/DependencyInjection/ContainerBuilder.php

Line 577 in 7fbda2e

$this->privates[$id] = true;

should be removed, that's the bug.
ContainerBuilder is a bootstrap-stage thing, and private/public is a "compiled-container" only thing.

ro0NL · 2017-05-23T17:02:59Z

Agree. I figured it out ;) i tend to qualify it a bug though.. as my perception is compiled containerbuilder === compiled container === dumped compiled container, but for now we're good.

would be nice if ContainerBuilder::compile() returns a new (prepped) Container, that would allow to even further simplify (killing isCompiled and such), but that's quite a step though.

Squahsed&rebased. Ready for review :)

Sorry for the inconvenience, but this should do.

stof · 2017-05-23T17:14:44Z

@nicolas-grekas currently, you can keep using a ContainerBuilder after compilation (which is what some projects do as they don't dump the container to the cache, for instance Behat)

ro0NL · 2017-05-23T17:38:47Z

may i recall #20643 :) as containerbuilder is open for anything keeping privates empty is in line with that, and so a bugfix.

but the behavior is still wrong, however currently it's half right, half wrong.

nicolas-grekas

👍

stof · 2017-05-24T11:19:01Z

We discussed this with @nicolas-grekas and here is the summary of changes we would want to get:

add a hidden argument in getServiceIds to allow getting only public ids (i.e. opting in for the new behavior), and trigger a deprecation notice when getting the list with private ids in it (we have some existing code adding similar hidden arguments)
retarget the PR to 3.3 (due to the above previous deprecation)
try to keep the existing code filling the privates array in the ContainerBuilder, and debug test failures instead. As the 3.4 branch will allow using 4.0 deps, accessing private services in tests will be forbidden anyway. I suspect several failures are related to getServiceIds and so could be solved by the first point.

@ro0NL can you work on it ? Otherwise, I can take this PR over this evening to finish it

ro0NL · 2017-05-24T11:38:36Z

@stof i will finish it tonight :) so no changes for 3.2 right?

try to keep the existing code filling the privates array in the ContainerBuilder

so we dont allow fetching privates using containerbuilder right (after compiling). getServiceIds will be the least problem :) it needs to bypass get() for references etc. internally. ie. what the phpdumper does.

ro0NL · 2017-05-24T11:46:08Z

@stof Allow edits from maintainers is enabled.. so feel free to add commits though :)

fabpot · 2017-05-24T14:23:08Z

Thank you @ro0NL.

@stof

This PR was merged into the 3.2 branch. Discussion ---------- [DI] Check for privates before shared services | Q | A | ------------- | --- | Branch? | 3.2 | Bug fix? | yes | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | #22801 (comment), #22801 (comment) | License | MIT | Doc PR | symfony/symfony-docs#...  cc @stof Commits ------- 4f683a9 [DI] Check for privates before shared services

nicolas-grekas · 2017-05-24T14:40:46Z

3.2 now merged into 3.3 so that next steps can happen on 3.3

@stof

This PR was merged into the 3.2 branch. Discussion ---------- [DI] Check for privates before shared services | Q | A | ------------- | --- | Branch? | 3.2 | Bug fix? | yes | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | symfony#22801 (comment), symfony#22801 (comment) | License | MIT | Doc PR | symfony/symfony-docs#...  cc @stof Commits ------- 4f683a9 [DI] Check for privates before shared services

nicolas-grekas added this to the 3.2 milestone May 23, 2017

stof reviewed May 23, 2017

View reviewed changes

stof requested changes May 23, 2017

View reviewed changes

nicolas-grekas modified the milestones: 3.3, 3.2 May 23, 2017

[DI] Check for privates before shared services

4f683a9

nicolas-grekas approved these changes May 23, 2017

View reviewed changes

ro0NL mentioned this pull request May 24, 2017

[DI] Mark service_container a private service #22806

Closed

fabpot merged commit 4f683a9 into symfony:3.2 May 24, 2017

ro0NL deleted the di/privates-3.2 branch May 24, 2017 14:48

This was referenced May 29, 2017

Release v3.2.9 #22947

Merged

Release v3.3.0 #22949

Merged

Uh oh!

[DI] Check for privates before shared services #22866

[DI] Check for privates before shared services #22866

Uh oh!

Conversation

ro0NL commented May 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nicolas-grekas commented May 23, 2017

Uh oh!

stof commented May 23, 2017

Uh oh!

ro0NL commented May 23, 2017

Uh oh!

ro0NL commented May 23, 2017

Uh oh!

ro0NL commented May 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stof commented May 23, 2017

Uh oh!

ro0NL commented May 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stof May 23, 2017

Choose a reason for hiding this comment

Uh oh!

ro0NL May 23, 2017

Choose a reason for hiding this comment

Uh oh!

ro0NL May 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ro0NL May 23, 2017

Choose a reason for hiding this comment

Uh oh!

stof May 23, 2017

Choose a reason for hiding this comment

Uh oh!

ro0NL May 23, 2017

Choose a reason for hiding this comment

Uh oh!

ro0NL May 23, 2017

Choose a reason for hiding this comment

Uh oh!

stof May 23, 2017

Choose a reason for hiding this comment

Uh oh!

nicolas-grekas commented May 23, 2017

Uh oh!

ro0NL commented May 23, 2017

Uh oh!

stof commented May 23, 2017

Uh oh!

ro0NL commented May 23, 2017

Uh oh!

ro0NL commented May 23, 2017

Uh oh!

nicolas-grekas commented May 23, 2017

Uh oh!

nicolas-grekas commented May 23, 2017

Uh oh!

ro0NL commented May 23, 2017

Uh oh!

stof commented May 23, 2017

Uh oh!

ro0NL commented May 23, 2017

Uh oh!

nicolas-grekas left a comment

Choose a reason for hiding this comment

Uh oh!

stof commented May 24, 2017

Uh oh!

ro0NL commented May 24, 2017

Uh oh!

ro0NL commented May 24, 2017

Uh oh!

fabpot commented May 24, 2017

Uh oh!

ro0NL commented May 23, 2017 •

edited

Loading

ro0NL commented May 23, 2017 •

edited

Loading

ro0NL commented May 23, 2017 •

edited

Loading

ro0NL May 23, 2017 •

edited

Loading