Memory leak when using route annotation #32220

goetas · 2019-06-27T13:27:58Z

Symfony version(s) affected: v4.3.2 (tested also 3.4 and 4.2, same issue)

Description
When a route is defined using annotations, there is a memory leak.
If the route is defined using YAML, there is no memory leak.

How to reproduce
I have created a basic symfony app available in https://github.com/goetas/symfony-mem-leak
The travis build showing the memory leak https://travis-ci.org/goetas/symfony-mem-leak/jobs/551313488

The test app has a basic controller as follows:

class DefaultController
{
    /**
     * @Route("/leak",name="getss")
     * @return Response
     */
    public function leak()
    {
        return new Response('hello');
    }

    public function noleak()
    {
        return new Response('hello');
    }
}

Requests to leak will result in a memory leak, requests to noleak will have no leaks.

The repo includes 2 scripts bin/runner-no-leak and bin/runner-no-leak that call 5000 times the two routes.

When bin/runner-leak is called, memory goes up.

In larger application memory goes up faster... apparently depending on the output of the method..

The text was updated successfully, but these errors were encountered:

stof · 2019-06-27T13:33:11Z

Well, your no-leak case is actually using more than 2x the memory from the start, so maybe it leaked everything even faster (and then stopped leaking more)...

goetas · 2019-06-27T13:35:32Z

Well, your no-leak case is actually using more than 2x the memory from the start, so maybe it leaked everything even faster (and then stopped leaking more)...

That is just because the first command warms up the cache

goetas · 2019-06-27T13:37:14Z

I have committed right now a warmup command too

https://travis-ci.org/goetas/symfony-mem-leak/jobs/551321942

goetas · 2019-06-27T14:58:40Z

Tacked down the issue to the PhpFilesAdapter, used but the annotation reader.

Replacing it with a ArrayAdapter the leak disappears.

goetas · 2019-06-27T15:36:02Z

I have reduced it to a minimal version that leaks on my machine, but does not on travis.

goetas · 2019-06-27T20:25:22Z

Found the issue... https://bugs.php.net/bug.php?id=76982 😭

PhpFilesAdapter uses exactly this technique to load files from the cache

goetas · 2019-06-27T20:29:47Z

I think symfony's PhpFilesAdapter should avoid using closures in the cached file

goetas · 2019-06-27T20:34:27Z

Introduced in symfony with #27543 (but it is a PHP bug rather than symfony)

nicolas-grekas · 2019-06-27T21:50:32Z

Thanks for the details, I found a work around in #32236
Could you please confirm it's fixing the mem leak?

goetas · 2019-06-28T08:06:43Z

Thanks for working on this!
I have verified your PR and seems working.
Here is the build on Travis and here the branch that uses your cache-fix branch

goetas · 2019-06-28T08:08:30Z

Can somebody with the right privileges change the label on this issue from "routing" to "cache"?

goetas · 2019-06-28T09:38:00Z

Unfortunately it seems the PHP bug (https://bugs.php.net/bug.php?id=76982) has opened some pandora box of memory leaks....

Here (build and code) the same type of memory leak applied to the symfony dependency injection container... that recently started to use closures and includes quite intensively.

Can somebody prove me wrong please?

goetas · 2019-06-28T10:12:08Z

It seems I'm not wrong... hoped to be...

Here (code and build) of a compiled container leaking memory....

Things get way worse if the service is configured as shared: false

@goetas

This PR was merged into the 4.2 branch. Discussion ---------- [Cache] work aroung PHP memory leak | Q | A | ------------- | --- | Branch? | 4.2 | Bug fix? | yes | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | #32220 | License | MIT | Doc PR | - Works around https://bugs.php.net/76982, as identified by @goetas in the linked issue. Commits ------- 5d55b91 [Cache] work aroung PHP memory leak

nicolas-grekas · 2019-06-28T14:39:12Z

I had a look at your reproducers, my opinion right now is that we don't care: to hit the bug, one needs to create a container instance in a loop. This is not a realistic use case to me: the container should be booted once and reused as often as possible. That's what php-pm and similar strategies do. Swoole integrations should too if they don't.

goetas · 2019-06-28T14:44:24Z

my opinion right now is that we don't care: to hit the bug, one needs to create a container instance in a loop. This is not a realistic use case to me: the container should be booted once and reused as often as possible.

In this case (with non shared services), the leak happens with one single instance on the container already booted.

I understand that in this case symfony can not do much, it is more a general PHP issue.
My learning from this is that PHP closures are not that perfect as they seem, and that they should be used with care until some issues are not fixed.

nicolas-grekas · 2019-06-28T14:49:59Z

I don't see any memory leak with this case.
Note that using range() is allocating a big array. Not sure it's the best in a script that wants to track memory side effects of other parts.

goetas · 2019-06-28T15:00:58Z

I don't see any memory leak with this case.

You are right... I got lot in the last hours of debugging. It seems the issue is only when creating multiple containers. Things not common in PHP-FPM applications, but a common thing for lambda, swoole or reactphp

Note that using range() is allocating a big array. Not sure it's the best in a script that wants to track memory side effects of other parts.

I'm not testing memory usage, but memory leaks. It does not matter now much memory is using, it should not increase on each iteration. But that is another topic

nicolas-grekas · 2019-06-28T15:05:22Z

but a common thing for lambda, swoole or reactphp

If that's true, that's a bug in the integration of SF with those: the same kernel is reused in e.g. php-pm, which does it correctly, using reactphp.

I think we can close this issue, thanks again for investigating!

goetas · 2019-06-28T15:07:17Z

I agree that this can be closed.

Hope just that somebody will start to work on https://bugs.php.net/bug.php?id=76982

I think that there is a lot of RAM world-wide just because of that bug 😃

zmitic · 2019-06-30T10:33:45Z

@goetas

Things not common in PHP-FPM applications, but a common thing for lambda, swoole or reactphp

I am using https://github.com/k911/swoole-bundle and didn't notice memory leaks in production mode, dev does have some but not a big problem.

Do you know how to replicate memory leak under swoole?

goetas · 2019-06-30T15:43:04Z

I do not know how you handle requests in swoole.
Do you get a fresh kernel on each request (or call boot/shutdown)? If yes, then your app should be affected. If you have a single global di container for the whole app, then you can consider yourself safe.

Regarding the cache issue, it was introduced with symfony 4.2, if you have an older version, you are safe from this too

goetas mentioned this issue Jun 27, 2019

Memory leak not displayed in the dump BitOne/php-meminfo#94

Closed

xabbuh added Bug Routing labels Jun 27, 2019

carsonbot added the Status: Needs Review label Jun 27, 2019

nicolas-grekas mentioned this issue Jun 27, 2019

[Cache] work aroung PHP memory leak #32236

Merged

fabpot added Cache and removed Routing labels Jun 28, 2019

nicolas-grekas closed this as completed Jun 28, 2019

driesvints mentioned this issue Apr 9, 2020

Memory leaks in PHPUnit laravel/framework#30736

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Memory leak when using route annotation #32220

Memory leak when using route annotation #32220

goetas commented Jun 27, 2019

stof commented Jun 27, 2019

goetas commented Jun 27, 2019

goetas commented Jun 27, 2019 •

edited

Loading

goetas commented Jun 27, 2019

goetas commented Jun 27, 2019

goetas commented Jun 27, 2019

goetas commented Jun 27, 2019 •

edited

Loading

goetas commented Jun 27, 2019

nicolas-grekas commented Jun 27, 2019

goetas commented Jun 28, 2019

goetas commented Jun 28, 2019

goetas commented Jun 28, 2019

goetas commented Jun 28, 2019 •

edited

Loading

nicolas-grekas commented Jun 28, 2019 •

edited

Loading

goetas commented Jun 28, 2019 •

edited

Loading

nicolas-grekas commented Jun 28, 2019

goetas commented Jun 28, 2019

nicolas-grekas commented Jun 28, 2019 •

edited

Loading

goetas commented Jun 28, 2019

zmitic commented Jun 30, 2019

goetas commented Jun 30, 2019

Memory leak when using route annotation #32220

Memory leak when using route annotation #32220

Comments

goetas commented Jun 27, 2019

stof commented Jun 27, 2019

goetas commented Jun 27, 2019

goetas commented Jun 27, 2019 • edited Loading

goetas commented Jun 27, 2019

goetas commented Jun 27, 2019

goetas commented Jun 27, 2019

goetas commented Jun 27, 2019 • edited Loading

goetas commented Jun 27, 2019

nicolas-grekas commented Jun 27, 2019

goetas commented Jun 28, 2019

goetas commented Jun 28, 2019

goetas commented Jun 28, 2019

goetas commented Jun 28, 2019 • edited Loading

nicolas-grekas commented Jun 28, 2019 • edited Loading

goetas commented Jun 28, 2019 • edited Loading

nicolas-grekas commented Jun 28, 2019

goetas commented Jun 28, 2019

nicolas-grekas commented Jun 28, 2019 • edited Loading

goetas commented Jun 28, 2019

zmitic commented Jun 30, 2019

goetas commented Jun 30, 2019

goetas commented Jun 27, 2019 •

edited

Loading

goetas commented Jun 27, 2019 •

edited

Loading

goetas commented Jun 28, 2019 •

edited

Loading

nicolas-grekas commented Jun 28, 2019 •

edited

Loading

goetas commented Jun 28, 2019 •

edited

Loading

nicolas-grekas commented Jun 28, 2019 •

edited

Loading