-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
[19.0][MIG] session_db #3413
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
sbidoul
wants to merge
37
commits into
OCA:19.0
Choose a base branch
from
acsone:19.0-mig-session_db-sbi
base: 19.0
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+915
−0
Open
[19.0][MIG] session_db #3413
Changes from all commits
Commits
Show all changes
37 commits
Select commit
Hold shift + click to select a range
400a7f1
session_db : store sessions in a database rather than in filestore
nseinlet 464dbee
[MIG] session_db to 16.0
sbidoul bc95bb9
session_db: use SESSION_LIFETIME constant
sbidoul f486a8f
session_db: do not update write_date on get
sbidoul 2d70126
session_db: declare maintainer
sbidoul 91b343b
session_db: explain why such as module is useful
sbidoul a84f9cc
[UPD] Update session_db.pot
e8db6d2
[UPD] README.rst
OCA-git-bot 47db0ff
[ADD] icon.png
OCA-git-bot ef852d0
session_db 16.0.1.0.1
OCA-git-bot 80565b3
session_db: improve resiliency to database errors
sbidoul 3c1f208
session_db 16.0.1.0.2
OCA-git-bot e78d481
session_db: gevent and thread support
sbidoul 0a30969
session_db: cosmetics
sbidoul f3ac596
session_db: improve cursor release
sbidoul ccf35b4
session_db 16.0.1.0.3
OCA-git-bot 76704b5
session_db: reconnect if needed
sbidoul aa0fe95
session_db: add a few tests
sbidoul df61ef9
session_db: refactor retry handling
sbidoul c911513
session_db: fix tests for v16 compatibility
sbidoul 3b1d2cc
session_db 16.0.1.0.4
OCA-git-bot 8fe6968
Fixes the issue "PGSessionStore.vacuum() got an unexpected keyword ar…
b6c0eba
session_db 16.0.1.0.5
OCA-git-bot d0897a4
[UPD] README.rst
OCA-git-bot 6c0c47a
[IMP] session_db: pre-commit auto fixes
sbidoul 834d006
[MIG] session_db: from 16.0 to 17.0
sbidoul 46e127b
[IMP] session_db: remove roadmap
sbidoul 8110200
[UPD] Update session_db.pot
ba64d5e
[BOT] post-merge updates
OCA-git-bot 5f62d03
Added translation using Weblate (Italian)
mymage ea890ca
[IMP] session_db: pre-commit auto fixes
sbidoul 387f4d0
[MIG] session_db: migration to 18.0 from 17.0
sbidoul 805c1e6
[UPD] Update session_db.pot
364039c
[BOT] post-merge updates
OCA-git-bot de3ec33
Update translation files
weblate 5146d58
[FIX] session_db: Fix creation of postgres_uri in tests
lal-solute b53d0a0
[MIG] session_db from 18 to 19
sbidoul File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| .. image:: https://odoo-community.org/readme-banner-image | ||
| :target: https://odoo-community.org/get-involved?utm_source=readme | ||
| :alt: Odoo Community Association | ||
|
|
||
| ==================== | ||
| Store sessions in DB | ||
| ==================== | ||
|
|
||
| .. | ||
| !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | ||
| !! This file is generated by oca-gen-addon-readme !! | ||
| !! changes will be overwritten. !! | ||
| !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | ||
| !! source digest: sha256:1f019db79d78ab20a204e51d1750ed8b2e7c22dd3d585569f97579c339bc34c7 | ||
| !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | ||
|
|
||
| .. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png | ||
| :target: https://odoo-community.org/page/development-status | ||
| :alt: Beta | ||
| .. |badge2| image:: https://img.shields.io/badge/license-LGPL--3-blue.png | ||
| :target: http://www.gnu.org/licenses/lgpl-3.0-standalone.html | ||
| :alt: License: LGPL-3 | ||
| .. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fserver--tools-lightgray.png?logo=github | ||
| :target: https://github.com/OCA/server-tools/tree/19.0/session_db | ||
| :alt: OCA/server-tools | ||
| .. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png | ||
| :target: https://translation.odoo-community.org/projects/server-tools-19-0/server-tools-19-0-session_db | ||
| :alt: Translate me on Weblate | ||
| .. |badge5| image:: https://img.shields.io/badge/runboat-Try%20me-875A7B.png | ||
| :target: https://runboat.odoo-community.org/builds?repo=OCA/server-tools&target_branch=19.0 | ||
| :alt: Try me on Runboat | ||
|
|
||
| |badge1| |badge2| |badge3| |badge4| |badge5| | ||
|
|
||
| Store sessions in a database instead of the filesystem. This simplifies | ||
| the configuration of horizontally scalable deployments, by avoiding the | ||
| need for a distributed filesystem to store the Odoo sessions. | ||
|
|
||
| **Table of contents** | ||
|
|
||
| .. contents:: | ||
| :local: | ||
|
|
||
| Usage | ||
| ===== | ||
|
|
||
| Set this module in the server wide modules. | ||
|
|
||
| Set a ``SESSION_DB_URI`` environment variable as a full postgresql | ||
| connection string, like ``postgres://user:passwd@server/db`` or ``db``. | ||
|
|
||
| It is recommended to use a dedicated database for this module, and | ||
| possibly a dedicated postgres user for additional security. | ||
|
|
||
| Bug Tracker | ||
| =========== | ||
|
|
||
| Bugs are tracked on `GitHub Issues <https://github.com/OCA/server-tools/issues>`_. | ||
| In case of trouble, please check there if your issue has already been reported. | ||
| If you spotted it first, help us to smash it by providing a detailed and welcomed | ||
| `feedback <https://github.com/OCA/server-tools/issues/new?body=module:%20session_db%0Aversion:%2019.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_. | ||
|
|
||
| Do not contact contributors directly about support or help with technical issues. | ||
|
|
||
| Credits | ||
| ======= | ||
|
|
||
| Authors | ||
| ------- | ||
|
|
||
| * Odoo SA | ||
| * ACSONE SA/NV | ||
|
|
||
| Maintainers | ||
| ----------- | ||
|
|
||
| This module is maintained by the OCA. | ||
|
|
||
| .. image:: https://odoo-community.org/logo.png | ||
| :alt: Odoo Community Association | ||
| :target: https://odoo-community.org | ||
|
|
||
| OCA, or the Odoo Community Association, is a nonprofit organization whose | ||
| mission is to support the collaborative development of Odoo features and | ||
| promote its widespread use. | ||
|
|
||
| .. |maintainer-sbidoul| image:: https://github.com/sbidoul.png?size=40px | ||
| :target: https://github.com/sbidoul | ||
| :alt: sbidoul | ||
|
|
||
| Current `maintainer <https://odoo-community.org/page/maintainer-role>`__: | ||
|
|
||
| |maintainer-sbidoul| | ||
|
|
||
| This module is part of the `OCA/server-tools <https://github.com/OCA/server-tools/tree/19.0/session_db>`_ project on GitHub. | ||
|
|
||
| You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| from . import pg_session_store |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| { | ||
| "name": "Store sessions in DB", | ||
| "version": "19.0.1.0.0", | ||
| "author": "Odoo SA,ACSONE SA/NV,Odoo Community Association (OCA)", | ||
| "license": "LGPL-3", | ||
| "website": "https://github.com/OCA/server-tools", | ||
| "maintainers": ["sbidoul"], | ||
| } |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # Translation of Odoo Server. | ||
| # This file contains the translation of the following modules: | ||
| # | ||
| msgid "" | ||
| msgstr "" | ||
| "Project-Id-Version: Odoo Server 18.0\n" | ||
| "Report-Msgid-Bugs-To: \n" | ||
| "Last-Translator: \n" | ||
| "Language-Team: \n" | ||
| "MIME-Version: 1.0\n" | ||
| "Content-Type: text/plain; charset=UTF-8\n" | ||
| "Content-Transfer-Encoding: \n" | ||
| "Plural-Forms: \n" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,216 @@ | ||
| # Copyright (c) Odoo SA 2017 | ||
| # @author Nicolas Seinlet | ||
| # Copyright (c) ACSONE SA 2022 | ||
| # @author Stéphane Bidoul | ||
| import functools | ||
| import json | ||
| import logging | ||
| import os | ||
|
|
||
| import psycopg2 | ||
|
|
||
| import odoo | ||
| from odoo import http | ||
| from odoo.tools._vendor import sessions | ||
|
|
||
| _logger = logging.getLogger(__name__) | ||
|
|
||
| lock = None | ||
| if odoo.evented: | ||
| import gevent.lock | ||
|
|
||
| lock = gevent.lock.RLock() | ||
| elif odoo.tools.config["workers"] == 0: | ||
| import threading | ||
|
|
||
| lock = threading.RLock() | ||
|
|
||
|
|
||
| def with_lock(func): | ||
| def wrapper(*args, **kwargs): | ||
| try: | ||
| if lock is not None: | ||
| lock.acquire() | ||
| return func(*args, **kwargs) | ||
| finally: | ||
| if lock is not None: | ||
| lock.release() | ||
|
|
||
| return wrapper | ||
|
|
||
|
|
||
| def with_cursor(func): | ||
| def wrapper(self, *args, **kwargs): | ||
| tries = 0 | ||
| while True: | ||
| tries += 1 | ||
| try: | ||
| self._ensure_connection() | ||
| return func(self, *args, **kwargs) | ||
| except (psycopg2.InterfaceError, psycopg2.OperationalError): | ||
| self._close_connection() | ||
| if tries > 4: | ||
| _logger.warning( | ||
| "session_db operation try %s/5 failed, aborting", tries | ||
| ) | ||
| raise | ||
| _logger.info("session_db operation try %s/5 failed, retrying", tries) | ||
|
|
||
| return wrapper | ||
|
|
||
|
|
||
| class PGSessionStore(sessions.SessionStore): | ||
| def __init__(self, uri, session_class=None): | ||
| super().__init__(session_class) | ||
| self._uri = uri | ||
| self._cr = None | ||
| self._open_connection() | ||
| self._setup_db() | ||
|
|
||
| def __del__(self): | ||
| self._close_connection() | ||
|
|
||
| @with_lock | ||
| def _ensure_connection(self): | ||
| if self._cr is None: | ||
| self._open_connection() | ||
|
|
||
| @with_lock | ||
| def _open_connection(self): | ||
| self._close_connection() | ||
| cnx = odoo.sql_db.db_connect(self._uri, allow_uri=True) | ||
| self._cr = cnx.cursor() | ||
| self._cr._cnx.autocommit = True | ||
|
|
||
| @with_lock | ||
| def _close_connection(self): | ||
| """Return cursor to the pool.""" | ||
| if self._cr is not None: | ||
| try: | ||
| self._cr.close() | ||
| except Exception: # pylint: disable=except-pass | ||
| pass | ||
| self._cr = None | ||
|
|
||
| @with_lock | ||
| @with_cursor | ||
| def _setup_db(self): | ||
| self._cr.execute( | ||
| """ | ||
| CREATE TABLE IF NOT EXISTS http_sessions ( | ||
| sid varchar PRIMARY KEY, | ||
| write_date timestamp without time zone NOT NULL, | ||
| payload text NOT NULL | ||
| ) | ||
| """ | ||
| ) | ||
|
|
||
| @with_lock | ||
| @with_cursor | ||
| def save(self, session): | ||
| payload = json.dumps(dict(session)) | ||
| self._cr.execute( | ||
| """ | ||
| INSERT INTO http_sessions(sid, write_date, payload) | ||
| VALUES (%(sid)s, now() at time zone 'UTC', %(payload)s) | ||
| ON CONFLICT (sid) | ||
| DO UPDATE SET payload = %(payload)s, | ||
| write_date = now() at time zone 'UTC' | ||
| """, | ||
| dict(sid=session.sid, payload=payload), | ||
| ) | ||
|
|
||
| @with_lock | ||
| @with_cursor | ||
| def delete(self, session): | ||
| self._cr.execute("DELETE FROM http_sessions WHERE sid=%s", (session.sid,)) | ||
|
|
||
| @with_lock | ||
| @with_cursor | ||
| def get(self, sid): | ||
| self._cr.execute("SELECT payload FROM http_sessions WHERE sid=%s", (sid,)) | ||
| try: | ||
| data = json.loads(self._cr.fetchone()[0]) | ||
| except Exception: | ||
| return self.new() | ||
|
|
||
| return self.session_class(data, sid, False) | ||
|
|
||
| # Odoo's FileSystemSessionStore has a few additional methods that are independent | ||
| # of the actual storage backend. We reuse them here. | ||
| rotate = http.FilesystemSessionStore.rotate | ||
| generate_key = http.FilesystemSessionStore.generate_key | ||
| is_valid_key = http.FilesystemSessionStore.is_valid_key | ||
| delete_old_sessions = http.FilesystemSessionStore.delete_old_sessions | ||
|
|
||
| @with_lock | ||
| @with_cursor | ||
| def vacuum(self, max_lifetime=http.SESSION_LIFETIME): | ||
| self._cr.execute( | ||
| "DELETE FROM http_sessions " | ||
| "WHERE now() at time zone 'UTC' - write_date > %s", | ||
| (f"{max_lifetime} seconds",), | ||
| ) | ||
|
|
||
| @with_lock | ||
| @with_cursor | ||
| def get_missing_session_identifiers(self, identifiers: list[str]) -> set[str]: | ||
| """ | ||
| :param identifiers: session identifiers whose file existence must be checked | ||
| identifiers are a part session sid (first 42 chars) | ||
| :type identifiers: iterable | ||
| :return: the identifiers which are not present on the filesystem | ||
| :rtype: set | ||
|
|
||
| Note 1: | ||
| Working with identifiers 42 characters long means that | ||
| we don't have to work with the entire sid session, | ||
| while maintaining sufficient entropy to avoid collisions. | ||
| See details in ``generate_key``. | ||
|
|
||
| Note 2: | ||
| Scans the session store for inactive (GC'd) sessions. | ||
| Performance is acceptable for an infrequent background job. | ||
| """ | ||
| missing_identifiers = set() | ||
| for identifier in identifiers: | ||
| self._cr.execute( | ||
| "SELECT sid FROM http_sessions WHERE sid LIKE %s||'%%' LIMIT 1", | ||
| (identifier,), | ||
| ) | ||
| if self._cr.rowcount == 0: | ||
| missing_identifiers.add(identifier) | ||
| return missing_identifiers | ||
|
|
||
| @with_lock | ||
| @with_cursor | ||
| def delete_from_identifiers(self, identifiers: list[str]) -> None: | ||
| for identifier in identifiers: | ||
| if not http._session_identifier_re.match(identifier): | ||
| raise ValueError( | ||
| "Identifier format incorrect, " | ||
| "did you pass in a string instead of a list?" | ||
| ) | ||
| self._cr.execute( | ||
| "DELETE FROM http_sessions WHERE sid LIKE %s||'%%'", (identifier,) | ||
| ) | ||
|
|
||
|
|
||
| _original_session_store = http.root.__class__.session_store | ||
|
|
||
|
|
||
| @functools.cached_property | ||
| def session_store(self): | ||
| session_db_uri = os.environ.get("SESSION_DB_URI") | ||
| if session_db_uri: | ||
| _logger.debug("HTTP sessions stored in: db") | ||
| return PGSessionStore(session_db_uri, session_class=http.Session) | ||
| return _original_session_store.__get__(self, self.__class__) | ||
|
|
||
|
|
||
| # Monkey patch of standard methods | ||
| _logger.debug("Monkey patching session store") | ||
| http.root.__class__.session_store = session_store | ||
| http.root.__class__.session_store.__set_name__(http.root.__class__, "session_store") | ||
| # Reset the lazy property cache | ||
| vars(http.root).pop("session_store", None) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| [build-system] | ||
| requires = ["whool"] | ||
| build-backend = "whool.buildapi" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| Store sessions in a database instead of the filesystem. This simplifies | ||
| the configuration of horizontally scalable deployments, by avoiding the | ||
| need for a distributed filesystem to store the Odoo sessions. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| Set this module in the server wide modules. | ||
|
|
||
| Set a `SESSION_DB_URI` environment variable as a full postgresql | ||
| connection string, like `postgres://user:passwd@server/db` or `db`. | ||
|
|
||
| It is recommended to use a dedicated database for this module, and | ||
| possibly a dedicated postgres user for additional security. |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would not be correct if
identifiercan contain a%but I don't think that is possible?