[DI] Improve performance of removing/inlining passes #27471

nicolas-grekas · 2018-06-01T17:59:46Z

Q	A
Branch?	master
Bug fix?	no
New feature?	no
BC breaks?	no
Deprecations?	yes
Tests pass?	yes
Fixed tickets	#27386
License	MIT
Doc PR	-

Here is an optimization to reclaim some compilation time by optimizing the analysis of unused and inlined services.

This PR removes any use case for RepeatedPass, instead:

RemoveUnusedDefinitionsPass works in one run, removing all private services that are unreachable from public services
InlineServiceDefinitionsPass reduces the number of nodes to analyze per iteration using AnalyzeServiceReferencesPass on a duplicated container internally.

https://blackfire.io/profiles/compare/00723822-6c09-431c-b98d-4a4197d044fc/graph?settings%5Bdimension%5D=wt&settings%5Bdisplay%5D=focused&settings%5BtabPane%5D=nodes&selected=Symfony%5CComponent%5CDependencyInjection%5CCompiler%5CRepeatedPass%3A%3Aprocess&callname=Symfony%5CComponent%5CDependencyInjection%5CCompiler%5CRepeatedPass%3A%3Aprocess

nicolas-grekas · 2018-06-03T08:13:24Z

@tarlepp since you reported a slowdown on Slack, can you please try this PR and report any results?

staabm · 2018-06-03T08:18:17Z

The blackfire link in the desc doesnt work

nicolas-grekas · 2018-06-03T08:24:53Z

@staabm updated thanks.

Note that our test suite is pretty nice on this topic, coverage is pretty great. I also used this patch to compile Blackfire's container, and it produces exactly same output. Confidence is good on my side :)

chalasr

UPGRADE notes are missing

chalasr · 2018-06-04T11:59:15Z

src/Symfony/Component/DependencyInjection/Compiler/RepeatablePassInterface.php

@@ -16,6 +16,8 @@
 * RepeatedPass.
 *
 * @author Johannes M. Schmitt <[email protected]>
+ *
+ * @deprecated since Symfony 4.1.


chalasr · 2018-06-04T12:00:47Z

src/Symfony/Component/DependencyInjection/Compiler/RepeatedPass.php

@@ -11,13 +11,17 @@

 namespace Symfony\Component\DependencyInjection\Compiler;

+@trigger_error(sprintf('The "%s" class is deprecated since Symfony 4.1.', RepeatedPass::class), E_USER_DEPRECATED);


nicolas-grekas · 2018-06-04T21:11:18Z

Do we need upgrade notes here? This is mostly internal thing IMHO.

stof · 2018-06-05T08:18:47Z

src/Symfony/Component/DependencyInjection/Compiler/InlineServiceDefinitionsPass.php

+        $srcIdsCount = 0;
+        foreach ($this->graph->getNode($id)->getInEdges() as $edge) {
+            $srcId = $edge->getSourceNode()->getId();
+            $this->connectedIds[$srcId] = true;
            if ($edge->isWeak()) {


if an edge is weak, other edges will not be marked as connected here. Is is expected ?

stof · 2018-06-05T08:19:45Z

src/Symfony/Component/DependencyInjection/Compiler/InlineServiceDefinitionsPass.php

        }

-        if (count(array_unique($ids)) > 1) {
+        if (1 !== \count($srcIds)) {


what is the difference between \count($srcIds) and $srcIdsCount ? If $srcIdsCount is the number of references (and so a single service referencing it multiple times is counted multiple times), the variable name is not clear.

stof · 2018-06-05T08:22:45Z

src/Symfony/Component/DependencyInjection/Compiler/AnalyzeServiceReferencesPass.php

@@ -101,6 +112,19 @@ protected function processValue($value, $isRoot = false)
                ContainerInterface::IGNORE_ON_UNINITIALIZED_REFERENCE === $value->getInvalidBehavior()
            );

+            if ($this->inExpression) {
+                $this->inExpression = false;


why making it false again ? Wouldn't this break in case an expression is referencing multiple services ?

stof · 2018-06-05T08:25:32Z

src/Symfony/Component/DependencyInjection/Compiler/ReplaceAliasByActualDefinitionPass.php

@@ -32,6 +32,7 @@ class ReplaceAliasByActualDefinitionPass extends AbstractRecursivePass
     */
    public function process(ContainerBuilder $container)
    {
+        $this->enableExpressionProcessing();


Are you sure about that ? Replacing a reference inside the expression won't change the expression anyway. It will keep using the alias.

stof · 2018-06-05T08:27:55Z

src/Symfony/Component/DependencyInjection/Compiler/ResolveReferencesToAliasesPass.php

@@ -27,6 +27,7 @@ class ResolveReferencesToAliasesPass extends AbstractRecursivePass
     */
    public function process(ContainerBuilder $container)
    {
+        $this->enableExpressionProcessing();


Are you sure about that ? Replacing a reference inside the expression won't change the expression anyway. It will keep using the alias.

nicolas-grekas · 2018-06-05T09:34:36Z

@stof agreed, thanks for the review, comments addressed.

src/Symfony/Component/DependencyInjection/Compiler/AbstractRecursivePass.php

...mfony/Bundle/FrameworkBundle/Tests/DependencyInjection/Compiler/CachePoolClearerPassTest.php

src/Symfony/Component/DependencyInjection/Compiler/AnalyzeServiceReferencesPass.php

Tobion · 2018-06-05T19:29:08Z

Thank you @nicolas-grekas.

…nicolas-grekas) This PR was merged into the 4.2-dev branch. Discussion ---------- [DI] Improve performance of removing/inlining passes | Q | A | ------------- | --- | Branch? | master | Bug fix? | no | New feature? | no | BC breaks? | no | Deprecations? | yes | Tests pass? | yes | Fixed tickets | #27386 | License | MIT | Doc PR | - Here is an optimization to reclaim some compilation time by optimizing the analysis of unused and inlined services. This PR removes any use case for `RepeatedPass`, instead: - `RemoveUnusedDefinitionsPass` works in one run, removing all private services that are unreachable from public services - `InlineServiceDefinitionsPass` reduces the number of nodes to analyze per iteration using `AnalyzeServiceReferencesPass` on a duplicated container internally. https://blackfire.io/profiles/compare/00723822-6c09-431c-b98d-4a4197d044fc/graph?settings%5Bdimension%5D=wt&settings%5Bdisplay%5D=focused&settings%5BtabPane%5D=nodes&selected=Symfony%5CComponent%5CDependencyInjection%5CCompiler%5CRepeatedPass%3A%3Aprocess&callname=Symfony%5CComponent%5CDependencyInjection%5CCompiler%5CRepeatedPass%3A%3Aprocess ![image](https://user-images.githubusercontent.com/243674/40884496-c31e780e-6714-11e8-8218-967c4b25b9ce.png) Commits ------- cf375e5 [DI] Improve performance of removing/inlining passes

nicolas-grekas added this to the 4.1 milestone Jun 1, 2018

carsonbot added Status: Needs Review DependencyInjection Bug labels Jun 1, 2018

nicolas-grekas mentioned this pull request Jun 1, 2018

[DI] do not log an inlined service as removed #27469

Closed

nicolas-grekas force-pushed the di-perf branch 6 times, most recently from 5095d4f to 2f0c343 Compare June 3, 2018 08:11

nicolas-grekas added the Deprecation label Jun 3, 2018

nicolas-grekas force-pushed the di-perf branch from 2f0c343 to 2bd4ffb Compare June 3, 2018 08:20

nicolas-grekas changed the base branch from master to 4.1 June 3, 2018 08:35

nicolas-grekas force-pushed the di-perf branch 2 times, most recently from 0bf53a9 to ae832d5 Compare June 3, 2018 21:19

nicolas-grekas changed the base branch from 4.1 to master June 3, 2018 21:27

nicolas-grekas added Performance and removed Bug labels Jun 3, 2018

nicolas-grekas modified the milestones: 4.1, next Jun 3, 2018

nicolas-grekas force-pushed the di-perf branch 2 times, most recently from 21effb3 to 75114e9 Compare June 4, 2018 06:41

carsonbot removed the Status: Needs Review label Jun 4, 2018

Tobion approved these changes Jun 4, 2018

View reviewed changes

carsonbot added Status: Reviewed and removed Status: Reviewed labels Jun 4, 2018

carsonbot added the Status: Needs Work label Jun 4, 2018

chalasr requested changes Jun 4, 2018

View reviewed changes

nicolas-grekas force-pushed the di-perf branch from 75114e9 to ead5a79 Compare June 4, 2018 17:23

stof reviewed Jun 5, 2018

View reviewed changes

nicolas-grekas force-pushed the di-perf branch from ead5a79 to 21bdddb Compare June 5, 2018 09:34

nicolas-grekas force-pushed the di-perf branch from 21bdddb to 03517d3 Compare June 5, 2018 09:35

stof reviewed Jun 5, 2018

View reviewed changes

src/Symfony/Component/DependencyInjection/Compiler/AbstractRecursivePass.php Outdated Show resolved Hide resolved

nicolas-grekas force-pushed the di-perf branch from 03517d3 to 1706579 Compare June 5, 2018 09:55

Taluu reviewed Jun 5, 2018

View reviewed changes

...mfony/Bundle/FrameworkBundle/Tests/DependencyInjection/Compiler/CachePoolClearerPassTest.php Show resolved Hide resolved

src/Symfony/Component/DependencyInjection/Compiler/AnalyzeServiceReferencesPass.php Outdated Show resolved Hide resolved

[DI] Improve performance of removing/inlining passes

cf375e5

nicolas-grekas force-pushed the di-perf branch from 1706579 to cf375e5 Compare June 5, 2018 12:00

chalasr approved these changes Jun 5, 2018

View reviewed changes

carsonbot added Status: Reviewed and removed Status: Needs Work labels Jun 5, 2018

stof approved these changes Jun 5, 2018

View reviewed changes

Tobion merged commit cf375e5 into symfony:master Jun 5, 2018

nicolas-grekas deleted the di-perf branch June 5, 2018 19:35

nicolas-grekas mentioned this pull request Jun 6, 2018

[DI] Remove dead code in RemoveUnusedDefinitionsPass #27506

Closed

ciaranmcnulty mentioned this pull request Jul 2, 2018

Fatal error in RemoveUnusedDefinitionsPass when building container with injected services #27802

Closed

nicolas-grekas modified the milestones: next, 4.2 Nov 1, 2018

This was referenced Nov 3, 2018

Release v4.2.0-BETA1 #29072

Closed

Release v4.2.0-BETA1 #29073

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DI] Improve performance of removing/inlining passes #27471

[DI] Improve performance of removing/inlining passes #27471

nicolas-grekas commented Jun 1, 2018 •

edited

Loading

nicolas-grekas commented Jun 3, 2018

staabm commented Jun 3, 2018

nicolas-grekas commented Jun 3, 2018

chalasr left a comment

chalasr Jun 4, 2018

chalasr Jun 4, 2018

nicolas-grekas commented Jun 4, 2018

stof Jun 5, 2018

stof Jun 5, 2018

stof Jun 5, 2018

stof Jun 5, 2018

stof Jun 5, 2018

nicolas-grekas commented Jun 5, 2018

Tobion commented Jun 5, 2018

		@@ -11,13 +11,17 @@

		namespace Symfony\Component\DependencyInjection\Compiler;

		@trigger_error(sprintf('The "%s" class is deprecated since Symfony 4.1.', RepeatedPass::class), E_USER_DEPRECATED);

[DI] Improve performance of removing/inlining passes #27471

[DI] Improve performance of removing/inlining passes #27471

Conversation

nicolas-grekas commented Jun 1, 2018 • edited Loading

nicolas-grekas commented Jun 3, 2018

staabm commented Jun 3, 2018

nicolas-grekas commented Jun 3, 2018

chalasr left a comment

Choose a reason for hiding this comment

chalasr Jun 4, 2018

Choose a reason for hiding this comment

chalasr Jun 4, 2018

Choose a reason for hiding this comment

nicolas-grekas commented Jun 4, 2018

stof Jun 5, 2018

Choose a reason for hiding this comment

stof Jun 5, 2018

Choose a reason for hiding this comment

stof Jun 5, 2018

Choose a reason for hiding this comment

stof Jun 5, 2018

Choose a reason for hiding this comment

stof Jun 5, 2018

Choose a reason for hiding this comment

nicolas-grekas commented Jun 5, 2018

Tobion commented Jun 5, 2018

nicolas-grekas commented Jun 1, 2018 •

edited

Loading