Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Cache] Add support for a tag lifetime in the RedisTagAwareAdapter #47509

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

Spea
Copy link
Contributor

@Spea Spea commented Sep 7, 2022

Q A
Branch? 4.4
Bug fix? yes
New feature? maybe (see below)
Deprecations? no
Discussion #45507
License MIT

We use the RedisTagAwareAdapter cache to tag cached auth tokens for a specific user. These cached tokens have a rather short lifetime of 5 minutes.
By tagging those token caches, we can invalidate all token caches if the user (e.g.) is disabled/deleted. However, since the tag itself never expires (and some users don't come back for months), we see a constant increase of non-expiring keys in our redis database, which also increases the memory usage. Changing the eviction policy to allkeys-lru or allkeys-lfu might fix this, but these policies are not allowed to be used with the RedisTagAwareAdapter and thus are not an option.

To fix this, I added a new parameter to the RedisTagAwareAdapter where you can specify a potential tag lifetime. This parameter can have the following values:

  • null (default) -> same behaviour as before, the tags will not expire
  • true -> use the lifetime from the item itself
  • numeric -> explicitly define the tag lifetime

From our point of view this is more of a bug fix and I think it would make sense to backport this parameter to the oldest supported version (in this case 4.4). But I also can see that this might be more of a new feature, so if you think it makes more sense to only add it to the 6.2 branch, I'll adjust the base branch (and code) accordingly.

Based on how we agree on this, I'll update the changelog and add a proper documentation PR.

@carsonbot
Copy link

It looks like you unchecked the "Allow edits from maintainer" box. That is fine, but please note that if you have multiple commits, you'll need to squash your commits into one before this can be merged. Or, you can check the "Allow edits from maintainers" box and the maintainer can squash for you.

Cheers!

Carsonbot

@Spea Spea force-pushed the expire-redis-tags branch from 5d68852 to 94ec4be Compare September 7, 2022 11:48
@nicolas-grekas
Copy link
Member

nicolas-grekas commented Sep 12, 2022

Unless I missed something, this would be very unsafe. If a tag expires before the items that are bound to it, said items won't be deleted when the same tag will be invalidated later on.

You're saying that tags remain in the storage while you deleted the users? How does that work? Reading the code, deleteItems calls sRem when deleting items, and Redis doesn't store empty sets. So I don't see how this can happen yet.

@Spea
Copy link
Contributor Author

Spea commented Sep 13, 2022

Unless I missed something, this would be very unsafe. If a tag expires before the items that are bound to it, said items won't be deleted when the same tag will be invalidated later on.

@nicolas-grekas I added a fail-safe mechanism to the code for this, so that the tag for an item can not have a lower lifetime than the item itself. See https://github.com/rebuy-de/symfony/blob/expire-redis-tags/src/Symfony/Component/Cache/Adapter/RedisTagAwareAdapter.php#L341-L344

You're saying that tags remain in the storage while you deleted the users? How does that work? Reading the code, deleteItems calls sRem when deleting items, and Redis doesn't store empty sets. So I don't see how this can happen yet.

Sorry for not being more clear on this. Le me try to explain how our current workflow looks like:

  • User logs in
  • We fetch the user from our external auth service based on the generated oauth token
  • The result is then cached along with a tag. The cache item itself has a lifetime of 5 minutes

Now there are two possibilities where the cache (and/or tag) is explicitly cleared.

  • The user logs out -> cache item will be cleared
  • The user is deleted/disabled -> cache item will be cleared based on the tag

But if none of these two scenarios happen (e.g. because the user just left the website and is never coming back), the cache item will just expire after 5 minutes without the tag ever being deleted/expiring. And thats why we want to be able to define a lifetime for those tags.

@nicolas-grekas
Copy link
Member

Thanks that for the details.
The issue with adding an expiry to tag items is that Redis will flag them as volatile, which means they could be evicted while their corresponding items aren't.

Here is another idea: we could make the class implement PrunableInterface.
You would be responsible for calling prune() from time to time.
The implementation should be done in Lua ideally.

Would that make sense to you? Up to give it a try?

@Spea
Copy link
Contributor Author

Spea commented Sep 13, 2022

That sounds like a good alternative. I'll give it a try. Should I create a new PR or work on the new solution in this one?

@nicolas-grekas
Copy link
Member

A new PR would be fine, to keep this one for history.

@nicolas-grekas
Copy link
Member

nicolas-grekas commented Sep 14, 2022

Looking forward to your PR on the topic. Closing meanwhile, thanks for raising this point.

@Spea Spea deleted the expire-redis-tags branch September 14, 2022 08:07
nicolas-grekas added a commit that referenced this pull request Oct 11, 2022
…angling symlinks (nicolas-grekas)

This PR was merged into the 6.2 branch.

Discussion
----------

[Cache] Make FilesystemTagAwareAdapter::prune() remove dangling symlinks

| Q             | A
| ------------- | ---
| Branch?       | 6.2
| Bug fix?      | no
| New feature?  | no
| Deprecations? | no
| Tickets       | -
| License       | MIT
| Doc PR        | -

As spotted in #47509 (comment)
This could be considered as a bugfix, but I suggest to be conservative and consider this as a minor improvement.

Commits
-------

cc333ba [Cache] Make FilesystemTagAwareAdapter::prune() remove dangling symlinks
symfony-splitter pushed a commit to symfony/cache that referenced this pull request Oct 11, 2022
…angling symlinks (nicolas-grekas)

This PR was merged into the 6.2 branch.

Discussion
----------

[Cache] Make FilesystemTagAwareAdapter::prune() remove dangling symlinks

| Q             | A
| ------------- | ---
| Branch?       | 6.2
| Bug fix?      | no
| New feature?  | no
| Deprecations? | no
| Tickets       | -
| License       | MIT
| Doc PR        | -

As spotted in symfony/symfony#47509 (comment)
This could be considered as a bugfix, but I suggest to be conservative and consider this as a minor improvement.

Commits
-------

cc333baae6 [Cache] Make FilesystemTagAwareAdapter::prune() remove dangling symlinks
@tourze
Copy link
Contributor

tourze commented Oct 28, 2023

The cache tag's expire time should depends on the longest expire time key.
How about execute the lua script like this when sadd in \Symfony\Component\Cache\Adapter\RedisTagAwareAdapter::doSave:

local current_expiry = redis.call('TTL', KEYS[1])
if current_expiry > 100 then
   return 0
else
   redis.call('EXPIRE', KEYS[1], 100)
   return 1
end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants