TagAwareAdapter over non-binary memcached connections corrupts memcache #27416

palex-fpt · 2018-05-30T01:34:21Z

Q	A
Branch?	3.4
Bug fix?	yes
New feature?	no
BC breaks?	no
Deprecations?	no
Tests pass?	no
Fixed tickets	#27405
License	MIT
Doc PR

TagAwareAdapter uses non-ascii symbols in key names. It breaks memcached connections in non-binary mode.

nicolas-grekas · 2018-05-30T04:24:28Z

This approach doesn't work, eg. a simple space is also forbidden in keys when using the non binary protocol.

Instead of changing the prefix for all backends (which has the downside of invalidating all existing pools btw), we should consider encoding keys, only in the memcached adapter.

nicolas-grekas · 2018-05-30T06:35:45Z

src/Symfony/Component/Cache/Traits/MemcachedTrait.php

+     *
+     * @param $key
+     */
+    private static function encodeKey($key)


I would suggest to inline the function and remove the method.

nicolas-grekas · 2018-05-30T06:36:38Z

src/Symfony/Component/Cache/Traits/MemcachedTrait.php

@@ -49,6 +50,7 @@ private function init(\Memcached $client, $namespace, $defaultLifetime)
            }
            $this->maxIdLength -= strlen($client->getOption(\Memcached::OPT_PREFIX_KEY));
            $this->client = $client;
+            $this->isTextProtocol = !$client->getOption(\Memcached::OPT_BINARY_PROTOCOL);


What do you think of always encoding the key, binary or not? That'd make the stored data independent from the actual protocol.

It adds some overhead. But it should not be high. There're two possible problems:

client loses control over cache keys (mcrouter uses key names for app-level routing)

non-symfony and/or non-php clients would have difficulties to share cache.

I don't share memcache with non-symfony apps, so I`m ok with it.

I think that's ok: the overhead should be negligible vs the network round-trips, and url encoding preserves most of the chars in the keys, making them still handleable.
Note that keys should be decoded on retrieval.

nicolas-grekas · 2018-05-30T06:58:30Z

src/Symfony/Component/Cache/Traits/MemcachedTrait.php

@@ -198,6 +200,12 @@ protected function doSave(array $values, $lifetime)
            $lifetime += time();
        }

+        if ($this->isTextProtocol) {
+            $encoded_values = array();
+            array_walk($values, function ($value, $key) use (&$encoded_values) { $encoded_values[rawurlencode($key)] = $value; });


I generally don't like array_walk(), too much references there. Should be done with a foreach instead.
The temp var should use camel case coding name also.

nicolas-grekas · 2018-05-30T07:01:09Z

All methods should encode/decode on read/write. Some are missing for now.
LGTM otherwise thanks.

nicolas-grekas · 2018-05-30T09:28:19Z

(but tests fail)

stof · 2018-05-30T10:08:27Z

src/Symfony/Component/Cache/Tests/Adapter/MemcachedAdapterTest.php

@@ -191,4 +192,29 @@ public function provideDsnWithOptions()
            array(\Memcached::OPT_SOCKET_RECV_SIZE => 1, \Memcached::OPT_SOCKET_SEND_SIZE => 2, \Memcached::OPT_RETRY_TIMEOUT => 8),
        );
    }
+
+    public function testNonBinaryMemcachedMode()


Instead of adding a few new tests for that, covering only a small part of the API, I think we should instead have a separate test case overriding createCachePool, to run the full adapter tests in non-binary mode

MemcachedAdapterTest tests MemcachedAdapter::createConnection only. Should I add separate test classes to test MemcachedAdapter get, set, etc.. methods in text and binary modes?

It looks like I should add key containing \0 and spaces into \Cache\IntegrationTests\SimpleCacheTest::validKeys

This way it would be tested against all cache implementations.

Or rely on the namespace to do so?

I added 'si\0mp le' key to \Cache\IntegrationTests\SimpleCacheTest::validKeys and ran tests local. All cache tests succeeded.

stof · 2018-05-30T10:10:16Z

src/Symfony/Component/Cache/Traits/MemcachedTrait.php

+            $encodedValues[rawurlencode($key)] = $value;
+        }
+
+        return $this->checkResultCode($this->getClient()->setMulti($encodedValues, $lifetime));


this is not decoding keys in the returned value. Can it have keys in it ?

\Memcached::setMulti returns bool. Did I missed something else?

nicolas-grekas · 2018-06-04T20:18:19Z

src/Symfony/Component/Cache/Tests/Simple/MemcachedCacheTextModeTest.php

+    {
+        $client = $defaultLifetime ? AbstractAdapter::createConnection('memcached://'.getenv('MEMCACHED_HOST'), array('binary_protocol' => false)) : self::$client;
+
+        return new MemcachedCache($client, str_replace('\\', '.', __CLASS__), $defaultLifetime);


Since the 2nd arg here is the namespace and is part of the key passed to memcached, we could add \0 and spaces to test the encoding?

base test \Symfony\Component\Cache\Tests\Simple\CacheTestCase::validKeys adds \0 to tested keys

…d connection in ascii mode

nicolas-grekas · 2018-06-08T22:45:36Z

Thank you @palex-fpt.

…upts memcache (Aleksey Prilipko) This PR was merged into the 3.4 branch. Discussion ---------- TagAwareAdapter over non-binary memcached connections corrupts memcache | Q | A | ------------- | --- | Branch? | 3.4 | Bug fix? | yes | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | no | Fixed tickets | #27405 | License | MIT | Doc PR | TagAwareAdapter uses non-ascii symbols in key names. It breaks memcached connections in non-binary mode. Commits ------- 67d4e6d bug #27405 [Cache] TagAwareAdapter should not corrupt memcached connection in ascii mode

carsonbot added Status: Needs Review Bug labels May 30, 2018

palex-fpt force-pushed the 27405 branch from 33880e2 to 7e41ff6 Compare May 30, 2018 06:17

nicolas-grekas reviewed May 30, 2018

View reviewed changes

nicolas-grekas added this to the 3.4 milestone May 30, 2018

palex-fpt force-pushed the 27405 branch from 380ee83 to 7261d74 Compare May 30, 2018 07:52

carsonbot added Status: Reviewed and removed Status: Needs Review labels May 30, 2018

nicolas-grekas approved these changes May 30, 2018

View reviewed changes

stof reviewed May 30, 2018

View reviewed changes

palex-fpt force-pushed the 27405 branch 2 times, most recently from 6ce07dd to 96c6cf9 Compare May 31, 2018 06:06

nicolas-grekas reviewed Jun 4, 2018

View reviewed changes

bug symfony#27405 [Cache] TagAwareAdapter should not corrupt memcache…

67d4e6d

…d connection in ascii mode

palex-fpt force-pushed the 27405 branch from 96c6cf9 to 67d4e6d Compare June 5, 2018 07:08

nicolas-grekas merged commit 67d4e6d into symfony:3.4 Jun 8, 2018

palex-fpt deleted the 27405 branch June 9, 2018 00:03

This was referenced Jun 25, 2018

Release v3.4.12 #27704

Merged

Release v4.0.12 #27705

Merged

Release v4.1.1 #27706

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TagAwareAdapter over non-binary memcached connections corrupts memcache #27416

TagAwareAdapter over non-binary memcached connections corrupts memcache #27416

palex-fpt commented May 30, 2018

nicolas-grekas commented May 30, 2018

nicolas-grekas May 30, 2018

nicolas-grekas May 30, 2018

palex-fpt May 30, 2018 •

edited

Loading

nicolas-grekas May 30, 2018

nicolas-grekas May 30, 2018 •

edited

Loading

nicolas-grekas commented May 30, 2018

nicolas-grekas commented May 30, 2018

stof May 30, 2018

palex-fpt May 30, 2018

palex-fpt May 31, 2018

nicolas-grekas May 31, 2018

palex-fpt May 31, 2018

stof May 30, 2018

palex-fpt May 30, 2018

nicolas-grekas Jun 4, 2018

palex-fpt Jun 5, 2018 •

edited

Loading

nicolas-grekas commented Jun 8, 2018

TagAwareAdapter over non-binary memcached connections corrupts memcache #27416

TagAwareAdapter over non-binary memcached connections corrupts memcache #27416

Conversation

palex-fpt commented May 30, 2018

nicolas-grekas commented May 30, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

palex-fpt May 30, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicolas-grekas May 30, 2018 • edited Loading

Choose a reason for hiding this comment

nicolas-grekas commented May 30, 2018

nicolas-grekas commented May 30, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

palex-fpt Jun 5, 2018 • edited Loading

Choose a reason for hiding this comment

nicolas-grekas commented Jun 8, 2018

palex-fpt May 30, 2018 •

edited

Loading

nicolas-grekas May 30, 2018 •

edited

Loading

palex-fpt Jun 5, 2018 •

edited

Loading