[Cache] Add stampede protection via probabilistic early expiration #27009

nicolas-grekas · 2018-04-22T20:21:45Z

Q	A
Branch?	master
Bug fix?	no
New feature?	yes
BC breaks?	no
Deprecations?	yes
Tests pass?	yes
Fixed tickets	-
License	MIT
Doc PR

This PR implements probabilistic early expiration on top of $cache->get($key, $callback);

It adds a 3rd arg to CacheInterface::get:

float $beta A float that controls the likelyness of triggering early expiration. 0 disables it, INF forces immediate expiration. The default is implementation dependend but should typically be 1.0, which should provide optimal stampede protection.

javiereguiluz · 2018-04-23T06:44:46Z

src/Symfony/Component/Cache/Adapter/PhpArrayAdapter.php

+     */
+    public function get(string $key, callable $callback, float $beta = null)
+    {
+        if (!\is_string($key)) {


Just asking: above we have string $key in the method argument ... will this if (!is_string($key)) code be ever executed if we pass a non-string argument?

javiereguiluz · 2018-04-23T06:47:08Z

src/Symfony/Component/Cache/Adapter/ProxyAdapter.php

                $innerItem->set(null);

                return $item;
            },
            null,
            CacheItem::class
        );
+        $this->setInnerItem = \Closure::bind(
+            function (CacheItemInterface $innerItem, array $item) {
+                if ($stats = $item["\0*\0newStats"]) {


Not sure it it'd help much, but maybe move \0*\0 to private const PREFIX = '\0*\0'; in this class?

Not sure also, using the const would just add indirection IMHO, for a purely internal thing.
Other comments addressed thanks.

javiereguiluz · 2018-04-23T06:48:09Z

src/Symfony/Component/Cache/CacheInterface.php

+{
+    /**
+     * @param callable(CacheItemInterface $item):mixed $callback Should return the computed value for the given key/item
+     * @param float                                    $beta     A float that controls the likelyness of triggering early expiration.


likelyness -> likeness

javiereguiluz · 2018-04-23T06:48:51Z

src/Symfony/Component/Cache/CacheInterface.php

+    /**
+     * @param callable(CacheItemInterface $item):mixed $callback Should return the computed value for the given key/item
+     * @param float                                    $beta     A float that controls the likelyness of triggering early expiration.
+     *                                                           0 disables it, INF forces immediate expiration. The default is implementation dependend


dependend -> dependent

javiereguiluz · 2018-04-23T06:49:34Z

src/Symfony/Component/Cache/CacheItem.php


 /**
 * @author Nicolas Grekas <[email protected]>
 */
 final class CacheItem implements CacheItemInterface
 {
+    /**
+     * References the unix timestamp stating when the item will expire, as integer.


unix -> Unix

nicolas-grekas · 2018-04-23T08:54:16Z

ping @Nyholm FYI: here, I deprecate getPreviousTags() and replace it by getMetadata()

src/Symfony/Component/Cache/CacheInterface.php

nicolas-grekas · 2018-04-24T10:03:19Z

I don't like unclear default values. Why not 1.0 directly?

I designed it this way so that the interface is generic enough. Hardcoding 1.0 as default would prevent any other implementations from providing a different default. Yes, there are no other implementations than the ones in this PR, but that's not a reason to break the abstraction provided by the interface.
Note that PSR-6 does the same for default expiry of items.

Tobion · 2018-04-24T10:21:08Z

src/Symfony/Component/Cache/CacheInterface.php

+{
+    /**
+     * @param callable(CacheItemInterface $item):mixed $callback Should return the computed value for the given key/item
+     * @param float|null                               $beta     A float that controls the likeness of triggering early expiration.


typo: likeliness

I did the same mistake as you, but @javiereguiluz made me fix it: it's really "likeness".

They have different meanings. You mean likeliness here.
likeliness = probability
likeness = resemblance

indeed, sorry for the misunderstanding (on my side)! (fixed)

nicolas-grekas · 2018-05-21T17:57:41Z

Rebased, PR ready for review.

nicolas-grekas · 2018-05-28T12:36:57Z

PR is ready.

The protection needs storing expiry+computation time. This is done by encoding these values in the key of a wrapping array:
$persistedValue = array("\x9D".$expiry.$computeTime."\x5F" => $cachedValue);

by using this magic-numbered structure, we are able to persist both raw + wrapped values in the same store, providing forward/backward compat at the storage level.

Stampede protection is always enabled when accessing values through the new CacheInterface::get() method (one should use the PSR-6 interface if they don't want the overhead of storing expiry+compute-time, thus opting out from stampede protection.)

Nyholm · 2018-05-28T13:12:55Z

Does the getPreviousTags really need to be deprecated? Wouldn't it be good to still have that? As far as I know that would make us compatible with the "soon to be"-PSR

nicolas-grekas · 2018-05-28T13:18:10Z

That's the thing we need to discuss :)
So, my current pov is that accessing tags is useful, but also is accessing the expiry. One use case for the expiry is the one present in this PR, another one is for HTTP caches, allowing to compute the max-age directive when serving cached values.
Which means if we spend some effort having an updated PSR, it could be worth doing it once for all.
This getMetadata() method would be my proposal: an array with keyed values, thus extensible, but still having some keys reserved by const name at least (tags, expiry - computation time could be a custom extension - or in the PSR).

nicolas-grekas · 2018-05-31T18:30:08Z

Rebased

nicolas-grekas · 2018-06-06T17:50:30Z

ping @symfony/deciders

Nyholm

I've looked at this PR. It is real hard to understand what you are doing. Even though I know what you are trying to do. :/

I do not mind the API changes but I would be happy to see some more comments.

Nyholm · 2018-06-08T19:34:25Z

src/Symfony/Component/Cache/Adapter/AbstractAdapter.php

                $item->isHit = $isHit;
                $item->defaultLifetime = $defaultLifetime;
+                if (\is_array($v) && 1 === \count($v) && 10 === \strlen($k = \key($v)) && "\x9D" === $k[0] && "\0" === $k[5] && "\x5F" === $k[9]) {


This line looks really complicated. I cannot tell what it is doing. Maybe add a comment?

Comments added, I hope it will be more clear.

Nyholm · 2018-06-08T20:10:49Z

src/Symfony/Component/Cache/Adapter/ProxyAdapter.php

                $innerItem->set(null);

                return $item;
            },
            null,
            CacheItem::class
        );
+        $this->setInnerItem = \Closure::bind(
+            function (CacheItemInterface $innerItem, array $item) {
+                if (isset(($metadata = $item["\0*\0newMetadata"])[CacheItem::METADATA_TAGS])) {


Same here, Im really trying to understand this, but I can't =)

Comment added, better?

fabpot

with minor comments

fabpot · 2018-06-10T10:07:51Z

src/Symfony/Component/Cache/Adapter/AbstractAdapter.php

                $item->isHit = $isHit;
                $item->defaultLifetime = $defaultLifetime;
+                // Detect wrapped values that encode for their expiry and creation duration
+                // For compacity, these values are packed in the key of an array using magic numbers


for compactness?

fabpot · 2018-06-10T10:09:44Z

src/Symfony/Component/Cache/Adapter/AbstractAdapter.php

                $item->isHit = $isHit;
                $item->defaultLifetime = $defaultLifetime;
+                // Detect wrapped values that encode for their expiry and creation duration
+                // For compacity, these values are packed in the key of an array using magic numbers
+                if (\is_array($v) && 1 === \count($v) && 10 === \strlen($k = \key($v)) && "\x9D" === $k[0] && "\0" === $k[5] && "\x5F" === $k[9]) {


Magic 10 here, not sure if we want to make it explicit (as we would want to make 5 and 9 explicit as well).

fabpot · 2018-06-10T10:10:38Z

src/Symfony/Component/Cache/Adapter/AbstractAdapter.php

                    }
+                    // For compacity, expiry and creation duration are packed in the key of a array, using magic numbers as separators


compactness as well?

fabpot · 2018-06-10T10:12:05Z

src/Symfony/Component/Cache/Adapter/ProxyAdapter.php

                $item->isHit = $innerItem->isHit();
                $item->defaultLifetime = $defaultLifetime;
                $item->innerItem = $innerItem;
                $item->poolHash = $poolHash;
+                // Detect wrapped values that encode for their expiry and creation duration
+                // For compacity, these values are packed in the key of an array using magic numbers


nicolas-grekas · 2018-06-10T15:12:38Z

comments addressed thanks

fabpot · 2018-06-10T16:26:45Z

src/Symfony/Component/Cache/CacheItem.php

@@ -140,10 +157,24 @@ public function tag($tags)
     * Returns the list of tags bound to the value coming from the pool storage if any.
     *
     * @return array
+     *
+     * @deprecated since Symfony 4.2. Use the "getMetadata()" method instead.


4.2, use the

fabpot · 2018-06-11T07:06:14Z

Thank you @nicolas-grekas.

…y expiration (nicolas-grekas) This PR was merged into the 4.2-dev branch. Discussion ---------- [Cache] Add stampede protection via probabilistic early expiration | Q | A | ------------- | --- | Branch? | master | Bug fix? | no | New feature? | yes | BC breaks? | no | Deprecations? | yes | Tests pass? | yes | Fixed tickets | - | License | MIT | Doc PR | This PR implements [probabilistic early expiration](https://en.wikipedia.org/wiki/Cache_stampede#Probabilistic_early_expiration) on top of `$cache->get($key, $callback);` It adds a 3rd arg to `CacheInterface::get`: > float $beta A float that controls the likelyness of triggering early expiration. 0 disables it, INF forces immediate expiration. The default is implementation dependend but should typically be 1.0, which should provide optimal stampede protection. Commits ------- 13523ad [Cache] Add stampede protection via probabilistic early expiration

…lculations (nicolas-grekas) This PR was merged into the 4.2-dev branch. Discussion ---------- [Cache] Use sub-second accuracy for internal expiry calculations | Q | A | ------------- | --- | Branch? | master | Bug fix? | no | New feature? | not really | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | - | License | MIT | Doc PR | - Embeds #26929, #27009 and #27028, let's focus on the 4th commit for now. This is my last significant PR in the Cache series :) By using integer expiries internally, our current implementations are sensitive to abrupt transitions when time() goes to next second: `$s = time(); sleep(1); echo time() - $s;` *can* display 2 from time to time. This means that we do expire items earlier than required by the expiration settings on items. This also means that there is no way to have a sub-second expiry. For remote backends, that's fine, but for ArrayAdapter, that's a limitation we can remove. This PR replaces calls to `time()` by `microtime(true)`, providing more accurate timing measurements internally. Commits ------- 08554ea [Cache] Use sub-second accuracy for internal expiry calculations

nicolas-grekas added this to the next milestone Apr 22, 2018

carsonbot added Status: Needs Review Cache Feature Deprecation labels Apr 22, 2018

nicolas-grekas force-pushed the cache-stats branch from 8e556be to 0b647d2 Compare April 23, 2018 05:28

javiereguiluz reviewed Apr 23, 2018

View reviewed changes

nicolas-grekas force-pushed the cache-stats branch 3 times, most recently from c2e16bf to 6ae48dc Compare April 23, 2018 08:48

nicolas-grekas force-pushed the cache-stats branch 2 times, most recently from 7a76d9f to af895ff Compare April 23, 2018 19:36

Tobion reviewed Apr 24, 2018

View reviewed changes

src/Symfony/Component/Cache/CacheInterface.php Show resolved Hide resolved

nicolas-grekas force-pushed the cache-stats branch from af895ff to aca75fa Compare April 24, 2018 06:54

nicolas-grekas mentioned this pull request Apr 24, 2018

[Cache] Prevent stampede at warmup using apcu_entry() for locking #27028

Closed

nicolas-grekas force-pushed the cache-stats branch 2 times, most recently from 99634fc to 472b593 Compare April 24, 2018 10:07

Tobion reviewed Apr 24, 2018

View reviewed changes

nicolas-grekas force-pushed the cache-stats branch 2 times, most recently from 5e7db51 to d01a58b Compare April 24, 2018 14:36

nicolas-grekas mentioned this pull request Apr 24, 2018

[Cache] Use sub-second accuracy for internal expiry calculations #27031

Merged

nicolas-grekas mentioned this pull request May 18, 2018

[Cache]: Get invalid cache items #27294

Closed

nicolas-grekas force-pushed the cache-stats branch from d01a58b to 5076b6d Compare May 21, 2018 17:55

nicolas-grekas force-pushed the cache-stats branch from 5076b6d to 7ec3d50 Compare May 28, 2018 12:22

nicolas-grekas force-pushed the cache-stats branch from a81d3ba to fdce538 Compare May 31, 2018 18:28

Nyholm reviewed Jun 8, 2018

View reviewed changes

nicolas-grekas force-pushed the cache-stats branch from fdce538 to f6ca712 Compare June 9, 2018 08:41

fabpot approved these changes Jun 10, 2018

View reviewed changes

carsonbot added Status: Reviewed and removed Status: Needs Review labels Jun 10, 2018

nicolas-grekas force-pushed the cache-stats branch from f6ca712 to fe26414 Compare June 10, 2018 15:12

fabpot requested changes Jun 10, 2018

View reviewed changes

carsonbot added Status: Needs Work and removed Status: Reviewed labels Jun 10, 2018

[Cache] Add stampede protection via probabilistic early expiration

13523ad

nicolas-grekas force-pushed the cache-stats branch from fe26414 to 13523ad Compare June 10, 2018 16:44

carsonbot removed the Status: Needs Work label Jun 11, 2018

fabpot approved these changes Jun 11, 2018

View reviewed changes

carsonbot added the Status: Reviewed label Jun 11, 2018

fabpot merged commit 13523ad into symfony:master Jun 11, 2018

nicolas-grekas deleted the cache-stats branch June 11, 2018 13:38

This was referenced Jun 25, 2018

[Cache] Prevent stampede at warmup using flock() #27604

Merged

Make cache stampede protection optional, configurable, and more flexible #27729

Closed

nicolas-grekas mentioned this pull request Jul 1, 2018

[Cache] doc CacheInterface::get() symfony/symfony-docs#9986

Closed

nicolas-grekas modified the milestones: next, 4.2 Nov 1, 2018

This was referenced Nov 3, 2018

Release v4.2.0-BETA1 #29072

Closed

Release v4.2.0-BETA1 #29073

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Cache] Add stampede protection via probabilistic early expiration #27009

[Cache] Add stampede protection via probabilistic early expiration #27009

nicolas-grekas commented Apr 22, 2018 •

edited

Loading

javiereguiluz Apr 23, 2018

javiereguiluz Apr 23, 2018

nicolas-grekas Apr 23, 2018 •

edited

Loading

javiereguiluz Apr 23, 2018

javiereguiluz Apr 23, 2018

javiereguiluz Apr 23, 2018

nicolas-grekas commented Apr 23, 2018 •

edited

Loading

nicolas-grekas commented Apr 24, 2018 •

edited

Loading

Tobion Apr 24, 2018

nicolas-grekas Apr 24, 2018

Tobion Apr 24, 2018

nicolas-grekas Apr 24, 2018 •

edited

Loading

nicolas-grekas commented May 21, 2018

nicolas-grekas commented May 28, 2018 •

edited

Loading

Nyholm commented May 28, 2018

nicolas-grekas commented May 28, 2018 •

edited

Loading

nicolas-grekas commented May 31, 2018

nicolas-grekas commented Jun 6, 2018

Nyholm left a comment

Nyholm Jun 8, 2018

nicolas-grekas Jun 9, 2018

Nyholm Jun 8, 2018

nicolas-grekas Jun 9, 2018

fabpot left a comment

fabpot Jun 10, 2018

fabpot Jun 10, 2018

fabpot Jun 10, 2018

fabpot Jun 10, 2018

nicolas-grekas commented Jun 10, 2018

fabpot Jun 10, 2018

nicolas-grekas Jun 10, 2018

fabpot commented Jun 11, 2018

		}
		// For compacity, expiry and creation duration are packed in the key of a array, using magic numbers as separators

[Cache] Add stampede protection via probabilistic early expiration #27009

[Cache] Add stampede protection via probabilistic early expiration #27009

Conversation

nicolas-grekas commented Apr 22, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicolas-grekas Apr 23, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicolas-grekas commented Apr 23, 2018 • edited Loading

nicolas-grekas commented Apr 24, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicolas-grekas Apr 24, 2018 • edited Loading

Choose a reason for hiding this comment

nicolas-grekas commented May 21, 2018

nicolas-grekas commented May 28, 2018 • edited Loading

Nyholm commented May 28, 2018

nicolas-grekas commented May 28, 2018 • edited Loading

nicolas-grekas commented May 31, 2018

nicolas-grekas commented Jun 6, 2018

Nyholm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabpot left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicolas-grekas commented Jun 10, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabpot commented Jun 11, 2018

nicolas-grekas commented Apr 22, 2018 •

edited

Loading

nicolas-grekas Apr 23, 2018 •

edited

Loading

nicolas-grekas commented Apr 23, 2018 •

edited

Loading

nicolas-grekas commented Apr 24, 2018 •

edited

Loading

nicolas-grekas Apr 24, 2018 •

edited

Loading

nicolas-grekas commented May 28, 2018 •

edited

Loading

nicolas-grekas commented May 28, 2018 •

edited

Loading