Implement service-based Resource (cache) validation #15738

mpdude · 2015-09-09T12:42:31Z

Q	A
Bug fix?	no
New feature?	yes
BC breaks?	no
Deprecations?	yes
Tests pass?	yes
Fixed tickets	#7230, #15692, #7782
License	MIT
Doc PR	symfony/symfony-docs#5136

Overview

Currently, any metadata passed to ConfigCache (namely implementations of ResourceInterface) is serialized to disk. When the ConfigCache is validated, the metadata is unserialized and queried through ResourceInterface::isFresh() to determine whether the cache is fresh. That way, ResourceInterface implementations cannot interact with services, for example a database connection.

This PR introduces the new concept of ResourceCheckers. Services implementing ResourceCheckerInterface can be tagged as config_cache.resource_checker with an optional priority.

Clients that wish to use ConfigCache can then obtain an instance from the config_cache_factory service (which implements ConfigCacheFactoryInterface). The factory will take care of injecting resource checkers into the ConfigCache instance so that they can be used for cache validation.

Checking cache metadata is easy for ResourceCheckers:

First, the ResourceCheckerInterface::supports() implementation is passed the metadata object in question. If the checker cannot handle the type of resource passed, supports() should return false.
Otherwise, the ResourceCheckerInterface::isFresh() method will be called and given the resource as well as the timestamp at which the cache was initialized. If that method returns false, the cache is considered stale. If it returns true, the resource is considered unchanged and will not be passed to any additional checkers.

BC and migration path

This PR does not (intend to) break BC but it comes with deprecations. The main reason is that ResourceInterface contains an isFresh() method that does not make sense in the general case of resources.

Thus, ResourceInterface::isFresh() is marked as deprecated and should be removed in Symfony 3.0. Resource implementations that can (or wish to) be validated in that simple manner can implement the SelfCheckingResourceInterface sub-interface that still contains (and will keep) the isFresh() method. The change should be as simple as changing the extends list.

Apart from that, ResourceInterface will be kept as the base interface for resource implementations. It is used in several @api interfaces and thus cannot easily be substituted.

For the Symfony 2.x series, a BCResourceInterfaceChecker will be kept that performs validation through ResourceInterface::isFresh() but will trigger a deprecation warning. The remedy is to either implement a custom ResourceChecker with a priority higher than -1000; or to switch to the aforementioned SelfCheckingResourceInterface which is used at a priority of -990 (without deprecation warning).

The ConfigCache and ConfigCacheFactory classes can be used as previously but do not feature checker-based cache validation.

Outlook and closing remarks:

This PR supersedes #7230, #15692 and works at least in parts towards the goal of #7176.

The ResourceCheckerInterface, ...ConfigCache and ...ConfigCacheFactory no longer need to be aware of the debug flag. The different validation rules applied previously are now just a matter of ResourceChecker configuration (i. e. "no checkers" in prod).

It might be possible to remove the debug flag from Symfony's Router and/or Translator classes in the future as well because it was only passed on to the ConfigCache there.

stof · 2015-09-09T15:50:46Z

src/Symfony/Component/Config/Resource/BCResourceInterfaceChecker.php

+
+    public function isFresh(ResourceInterface $metadata, $timestamp)
+    {
+        trigger_error('Resource checking through ResourceInterface::isFresh() is deprecated since 2.8 and will be removed in 3.0', E_USER_DEPRECATED);


missing @ (which would make the testsuite fail btw)

fabpot · 2015-09-14T06:15:36Z

The UPGRADE-2.8.md and 3.0 file should be updated with instruction on how to upgrade.

mpdude · 2015-09-14T09:29:31Z

I added entries in the UPGRADING files and hope that's what you expect.

Currently, any metadata passed to ConfigCache (namely implementations of ResourceInterface) is serialized to disk. When the ConfigCache is validated, the metadata is unserialized and queried through ResourceInterface::isFresh() to determine whether the caceh is fresh. That way, ResourceInterface implementations cannot interact with services, for example a database connection. This PR introduces the new concept of ResourceCheckers. Services implementing ResourceCheckerInterface can be tagged as "config_cache.resource_checker" with an optional priority. Clients that wish to use ConfigCache can then obtain an instance from the config_cache_factory service (which implements ConfigCacheFactoryInterface). The factory will take care of injecting resource checkers into the ConfigCache instance so that they can be used for cache validation. Checking cache metadata is easy for ResourceCheckers: - First, the ResourceCheckerInterface::supports() implementation is passed the metadata object in question. If the checker cannot handle the type of resource passed, support() should return false. - Otherwise, the ResourceCheckerInterface::isFresh() method will be called and given the resource as well as the timestamp at which the cache was initialized. If that method returns false, the cache is considered stale. If it returns true, the resource is considered valid and will *not* be passed to any additional checkers. BC and migration path: This PR does not (intend to) break BC but it comes with deprecations. The main reason is that ResourceInterface contains an isFresh() method that does not make sense in the general case of resources. Thus, ResourceInterface::isFresh() is marked as deprecated and shall be removed in Symfony 3.0. Resource implementations that can (or wish to) be validated in that simple manner can implement the SelfCheckingResourceInterface sub-interface that still contains (and will keep) the isFresh() method. The change should be as simple as changing the "extends" list. Apart from that, ResourceInterface will be kept as the base interface for resource implementations. Note that ResourceInterface is used in several @api interfaces and thus cannot easily be substituted. For the Symfony 2.x series, a BCResourceInterfaceChecker will be kept that performs validation through ResourceInterface::isFresh() but will trigger a deprecation warning. The remedy is to either implement a custom ResourceChecker with a priority higher than -1000; or to switch to the aforementioned SelfCheckingResourceInterface which is used at a priority of -990 (without deprecation warning). The ConfigCache and ConfigCacheFactory classes can be used as previously but do not feature checker-based cache validation. Outlook and closing remarks: This PR supersedes #7230, #15692 and works at least in parts towards the goal of #7176. The ResourceCheckerInterface, ...ConfigCache and ...ConfigCacheFactory no longer need to be aware of the "debug" flag. The different validation rules applied previously are now just a matter of ResourceChecker configuration (i. e. "no checkers" in prod). It might be possible to remove the "debug" flag from Symfony's Router and/or Translator classes in the future as well because it was only passed on to the ConfigCache there.

…ngResourceInterface

stloyd · 2015-09-14T11:45:43Z

src/Symfony/Component/Config/Tests/Resource/ResourceStub.php

+        $this->fresh = $isFresh;
+    }
+
+    public function __toString() {


{ should be in next line.

Hey FabBot, are you sleeping :)?

stloyd · 2015-09-14T16:44:37Z

src/Symfony/Bundle/FrameworkBundle/DependencyInjection/Compiler/ConfigCachePass.php

+        krsort($resourceCheckers);
+        $resourceCheckers = call_user_func_array('array_merge', $resourceCheckers);
+
+        $container->getDefinition('config_cache_factory')->replaceArgument(0, $resourceCheckers);


Shouldn't this be findDefinition()?

Well, that service is defined and not an alias. Do you think it should be one?

mpdude · 2015-09-16T07:23:17Z

@fabpot Anything else you'd like to see changed? Once this is merged I'd like to continue discussing #15719 and #7796, both would need to be addressed in 2.8/3.0 because of deprecations/removals.

mpdude · 2015-09-23T08:58:48Z

Build failure due to transient test on Windows, added to the list at #15617 (comment)

mpdude · 2015-09-23T09:00:15Z

ping @fabpot

fabpot · 2015-09-23T09:12:28Z

👍

fabpot · 2015-09-24T11:40:05Z

ping @symfony/deciders

dunglas · 2015-09-24T17:16:18Z

👍

fabpot · 2015-09-25T06:33:14Z

Thank you @mpdude.

mpdude · 2015-09-25T07:23:04Z

Thank you @fabpot. Now let's get #15719 resolved (a quick win IMO) and we have a simple and clean ResourceInterface in 3.0.

stof · 2015-09-25T10:18:10Z

This PR is breaking tests

mpdude · 2015-09-25T10:31:39Z

Where?

stof · 2015-09-25T10:40:21Z

Ah, I understand the issue: this change is not yet merged into master, and http-kernel 2.8 allows config at both 2.8 ad 3.0.
@fabpot can you merge branches to fix the build ?

fabpot · 2015-09-25T10:59:30Z

done

aitboudad · 2015-09-27T12:09:56Z

src/Symfony/Bundle/FrameworkBundle/Tests/Translation/TranslatorTest.php

@@ -104,34 +104,6 @@ public function testTransWithCachingWithInvalidLocale()
        $translator->trans('foo');
    }

-    public function testLoadResourcesWithCaching()


any reason why this test fails ?

$translator->setConfigCacheFactory(new \Symfony\Component\Config\ResourceCheckerConfigCacheFactory([ new \Symfony\Component\Config\Resource\SelfCheckingResourceChecker(), new \Symfony\Component\Config\Resource\BCResourceInterfaceChecker() ]));

Don't know, but maybe you want to use the default ConfigCacheFactory in that case?

I used ResourceCheckerConfigCacheFactory, the problems occurs when we want to work with the debug mode, is there a way to pass debug mode for ResourceCheckerConfigCacheFactory like the old one ?

carsonbot added the Status: Needs Review label Sep 9, 2015

mpdude mentioned this pull request Sep 9, 2015

Introduce MetadataValidators to test arbitrary metadata for freshness. #15692

Closed

stof reviewed Sep 9, 2015
View reviewed changes

mpdude mentioned this pull request Sep 10, 2015

Deprecate ResourceInterface::getResource() #15719

Merged

mpdude added 5 commits September 14, 2015 13:27

Fix deprecation notice in test

64b9224

Bump require-dev dep because TranslatorCacheTest now needs SelfChecki…

6d81c69

…ngResourceInterface

Feedback from GH review

ab13ef4

Add UPGRADING notes

d1cb1fd

stloyd reviewed Sep 14, 2015
View reviewed changes

CS fix / License added

a845b3c

stloyd reviewed Sep 14, 2015
View reviewed changes

mpdude mentioned this pull request Sep 18, 2015

Using a service as a router resource #15742

Closed

This was referenced Sep 24, 2015

[Config] Allow for service-based Resource (cache) validation #7230

Closed

Refactor behaviour triggered by a flag into separate classes. #7782

Closed

fabpot closed this in d60428c Sep 25, 2015

fabpot reopened this Sep 25, 2015

fabpot closed this Sep 25, 2015

xabbuh mentioned this pull request Sep 25, 2015

[Config] Ability to use a service to check for the freshness of cache entries symfony/symfony-docs#5723

Closed

aitboudad mentioned this pull request Sep 27, 2015

FrameworkBundle translator tests are failing for deps=high #15935

Closed

aitboudad reviewed Sep 27, 2015
View reviewed changes

mpdude mentioned this pull request Sep 28, 2015

Remove failing test to fix #15935 #15943

Merged

fabpot mentioned this pull request Nov 16, 2015

Release v2.8.0-BETA1 #16550

Merged

mpdude mentioned this pull request Feb 5, 2018

Write a short cookbook entry how to make use of the ConfigCacheFactory in 2.7 symfony/symfony-docs#5136

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement service-based Resource (cache) validation #15738

Implement service-based Resource (cache) validation #15738

mpdude commented Sep 9, 2015

stof Sep 9, 2015

fabpot commented Sep 14, 2015

mpdude commented Sep 14, 2015

stloyd Sep 14, 2015

mpdude Sep 14, 2015

stloyd Sep 14, 2015

mpdude Sep 14, 2015

mpdude commented Sep 16, 2015

mpdude commented Sep 23, 2015

mpdude commented Sep 23, 2015

fabpot commented Sep 23, 2015

fabpot commented Sep 24, 2015

dunglas commented Sep 24, 2015

fabpot commented Sep 25, 2015

mpdude commented Sep 25, 2015

stof commented Sep 25, 2015

mpdude commented Sep 25, 2015

stof commented Sep 25, 2015

fabpot commented Sep 25, 2015

aitboudad Sep 27, 2015

mpdude Sep 27, 2015

aitboudad Sep 27, 2015

Implement service-based Resource (cache) validation #15738

Implement service-based Resource (cache) validation #15738

Conversation

mpdude commented Sep 9, 2015

Overview

BC and migration path

Outlook and closing remarks:

Choose a reason for hiding this comment

fabpot commented Sep 14, 2015

mpdude commented Sep 14, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mpdude commented Sep 16, 2015

mpdude commented Sep 23, 2015

mpdude commented Sep 23, 2015

fabpot commented Sep 23, 2015

fabpot commented Sep 24, 2015

dunglas commented Sep 24, 2015

fabpot commented Sep 25, 2015

mpdude commented Sep 25, 2015

stof commented Sep 25, 2015

mpdude commented Sep 25, 2015

stof commented Sep 25, 2015

fabpot commented Sep 25, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment