Allow manipulation of base path when warming cache #3079

samuelvogel · 2012-01-10T10:17:16Z

It makes sense to allow a manipulation of the base path for cache warming for several reasons:

You use php-fpm in production and have it configured to run inside a chroot, since CLI used for cache warming does not have the chroot, paths will differ
You checkout new versions of your application and warm your cache before transfering it to the actual docroot to make the upgrade process atomic

The problem is when you warm the cache in a directory that is later moved or only accessible by a different path, then paths in app/cache/prod/templates.php for example will be wrong, and Twig imports will break.

The text was updated successfully, but these errors were encountered:

nfx · 2012-09-02T20:35:54Z

I was thinking on implementing similar feature for deployment of sf2 based app. Simple sed-replacing was not actually an option, as most of the paths were stored in php-serialized files.

This feature would take much time to test and implement, as it depends not on single bundle.

samuelvogel · 2013-08-09T14:25:15Z

Actually we are doing it with sed currently, maybe this is of use for you:

WORKSPACE="$(mktemp -d)"
FINAL=/var/www/
find $WORKSPACE/app/cache -type f -exec sed -i "s#$WORKSPACE#$FINAL#g" '{}' \;

nfx · 2013-08-13T21:36:27Z

@samuelvogel so, there are no serialised files and nothing gets corrupted?

samuelvogel · 2013-08-13T21:39:49Z

Nothing gets corrupted, we are using this in production!

nfx · 2013-08-14T09:45:02Z

then add entry to manual please ;)

johnrivelt · 2014-05-07T08:15:53Z

Same Issue, I can't run both web and console together.
I am using OPENBSD, NGINX, PHP-FPM.
In cached appProdProjectContainer.php file,
getTwig_LoaderService() called absolute path.
If I generate cache file from web, path is like "/htdocs/".
But if I generate cache file from php app/console "/var/www/htdocs/"

Any solutions ?

johnrivelt · 2014-05-12T05:35:33Z

I found solution without coding in php side.
NGINX is using chroot as default and it indicate folder like "/var/www".
PHP-FPM has option like chroot="/var/www" to fit NGINX default chroot.

Steps

When start NGINX server, use -u option to disable chroot function.
Comment chroot="/var/www" line in php-fpm.conf
restart nginx, php-fpm

Then path will be changed to absolute path.

samuelvogel · 2014-05-12T08:23:09Z

@johnrivelt: Well this is not really a solution, you are just disabling the chroot of PHP-FPM...

What we are doing currently is emptying the /app/cache/ folder and calling the page once via HTTP to regenerate the cache files. This is not an atomic deployment, but it works!

johnrivelt · 2014-05-12T09:05:00Z

@samuelvogel

Your solution will work for web access but not from console command in my
case which call twig templating service.

I confirmed that problem is
->addPath method's path in cached file.

Because the cache generated from web has web root path is same with SAPI
chroot.

My solving way is to make php-fpm root path same with CLI root.
Then no matter where you generate cache, it will work for both.

I am assuming your case might be related chroot path or cache folder
permission.

John Cho
On May 12, 2014 6:23 PM, "Samuel Vogel" [email protected] wrote:

@johnrivelt https://github.com/johnrivelt: Well this is not really a
solution, you are just disabling the chroot...

What we are doing currently is emptying the /app/cache/ folder and
calling the page once via HTTP to regenerate the cache files. This is not
an atomic deployment, but it works!

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/3079#issuecomment-42806661
.

…tainer (fabpot) This PR was merged into the 2.3 branch. Discussion ---------- [HttpKernel] removed absolute paths from the generated container | Q | A | ------------- | --- | Bug fix? | yes | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | first step to resolve #6484, #3079, and #9238 | License | MIT | Doc PR | n/a This PR converts absolute paths to relative ones in the dumped container. The code is a bit "ugly", but it gets the job done and I'm not sure that there is a more elegant way without breaking everything. Commits ------- c1450b4 [HttpKernel] removed absolute paths from the generated container

…e generated container (nicolas-grekas) This PR was merged into the 2.3 branch. Discussion ---------- [DependencyInjection] make paths relative to __DIR__ in the generated container | Q | A | ------------- | --- | Bug fix? | yes | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | #6484, #3079, partially #9238, #10894, #10999 | License | MIT | Doc PR | n/a This is an alternative approach to #10999 for removing absolute paths from the generated container: instead of trying to fix the container file after it has been dumped, telling to the PhpDumper where its output will be written allows it to replace parts of strings by an equivalent value based on `__DIR__`. This should be safe, thus the PR is on 2.3. Commits ------- edd7057 [DependencyInjection] make paths relative to __DIR__ in the generated container

nicolas-grekas · 2014-12-02T12:35:47Z

Fixed by #12784

kostiklv · 2015-01-06T11:51:06Z

@fabpot, @nicolas-grekas, this issue was not fully fixed by #12784. Specifically, app/cache/*/templates.php still contains absolute paths

tdierolf · 2015-04-08T11:01:17Z

I agree. Currently we're running into problems having absolute paths in app/cache/*/templates.php.

We're building release packages during a Jenkins build on a dedicated build machine which we then deploy to production machines. But the cache file app/cache/*/templates.php contains the absolute paths from the build on the Jenkins machine.

We can omit the TemplatePathsCacheWarmer with the --no-optional-warmer option as it is an optional warmer or just delete app/cache/*/templates.php files before deploying. However, having correct templates paths cached in production would still be a significant performance boost.

FaKleiser · 2015-08-05T11:58:41Z

I agree, too. There are still absolute paths in app/cache/*/templates.php. We are currently running Symfony 2.7.2.

pasdeloup · 2016-06-29T19:31:50Z

It should be possible to adopt the same strategy than @nicolas-grekas on #12784 : after generating absolute path, match the result with DIR to make it relative.
Did anyone try it? Is there a problem like a BC break? I can try but I don't want to waste time if it has already been done and doesn't work.

…s cache (tgalopin) This PR was merged into the 3.2-dev branch. Discussion ---------- [FrameworkBundle] Use relative paths in templates paths cache | Q | A | ------------- | --- | Branch? | master | Bug fix? | yes | New feature? | yes | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | #3079 | License | MIT | Doc PR | - This implements the usage of relative paths instead of absolute ones in `var/cache/*/templates.php`, important for ability to build the cache in a different context than where it will be used. This PR transforms the following `templates.php`: ``` php <?php return array ( ':default:index.html.twig' => '/home/tgalopin/www/symfony-standard/app/Resources/views/default/index.html.twig', '::base.html.twig' => '/home/tgalopin/www/symfony-standard/app/Resources/views/base.html.twig', ); ``` Into: ``` php <?php return array ( ':default:index.html.twig' => __DIR__.'/../../../app/Resources/views/default/index.html.twig', '::base.html.twig' => __DIR__.'/../../../app/Resources/views/base.html.twig', ); ``` I also added tests for the TemplateCachePathsWarmer and improved tests for the TemplateLocator. Commits ------- 6f6139c [FrameworkBundle] Use relative paths in templates paths cache

tgalopin · 2016-08-23T17:59:51Z

Should be fixed now, if you have an issue don't hesitate to report it here.

tdierolf · 2016-09-22T12:32:20Z

Why has #19687 not been released yet?

nicolas-grekas · 2016-09-22T12:45:04Z

It is merged into master, and master will be tagged as 3.2 by the end of november, according to the release policy.

tdierolf · 2016-09-23T08:37:57Z

Ah ok, I see. Thank you very much!

jakzal mentioned this issue Dec 23, 2013

Using Symfony2 in a PHP-FPM chroot-ed environment. #6484

Closed

fabpot mentioned this issue May 12, 2014

[HttpKernel] removed absolute paths from the generated container #10894

Merged

fabpot closed this as completed May 17, 2014

fabpot mentioned this issue May 26, 2014

[HttpKernel] removed absolute paths from the generated container #10999

Closed

nicolas-grekas mentioned this issue Nov 30, 2014

[DependencyInjection] make paths relative to __DIR__ in the generated container #12784

Merged

nicolas-grekas reopened this Jan 6, 2015

rvanginneken mentioned this issue Oct 13, 2015

[TEMPLATING] relative paths in template cache #16233

Closed

1 task

javiereguiluz added Hack Day and removed Hack Day labels Mar 10, 2016

tgalopin mentioned this issue Aug 20, 2016

[FrameworkBundle] Use relative paths in templates paths cache #19687

Merged

fabpot closed this as completed Aug 23, 2016

Uh oh!

Allow manipulation of base path when warming cache #3079

Allow manipulation of base path when warming cache #3079

Comments

samuelvogel commented Jan 10, 2012

nfx commented Sep 2, 2012

Uh oh!

samuelvogel commented Aug 9, 2013

Uh oh!

nfx commented Aug 13, 2013

Uh oh!

samuelvogel commented Aug 13, 2013

Uh oh!

nfx commented Aug 14, 2013

Uh oh!

johnrivelt commented May 7, 2014

Uh oh!

johnrivelt commented May 12, 2014

Uh oh!

samuelvogel commented May 12, 2014

Uh oh!

johnrivelt commented May 12, 2014

Uh oh!

nicolas-grekas commented Dec 2, 2014

Uh oh!

kostiklv commented Jan 6, 2015

Uh oh!

tdierolf commented Apr 8, 2015

Uh oh!

FaKleiser commented Aug 5, 2015

Uh oh!

pasdeloup commented Jun 29, 2016

Uh oh!

tgalopin commented Aug 23, 2016

Uh oh!

tdierolf commented Sep 22, 2016

Uh oh!

nicolas-grekas commented Sep 22, 2016

Uh oh!

tdierolf commented Sep 23, 2016

Uh oh!