Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat(encryption): Add integration tests for occ commands and fix them#54316

Merged
susnux merged 9 commits intomasterfrom
feat/add-encryption-integration-tests
Sep 10, 2025
Merged

feat(encryption): Add integration tests for occ commands and fix them#54316
susnux merged 9 commits intomasterfrom
feat/add-encryption-integration-tests

Conversation

@come-nc
Copy link
Contributor

@come-nc come-nc commented Aug 7, 2025 •
edited
Loading

  • Resolves: #

Summary

TODO

  • Add intergations tests for encryption
  • Test encrypt-all
  • Test decrypt-all
  • Test basic file operations
  • Fix decrypt-all
  • Fix "Inconsistent data, File unshared, but owner not found. Should not happen"

Checklist

@come-nc come-nc added this to the Nextcloud 32 milestone Aug 7, 2025
@come-nc come-nc self-assigned this Aug 7, 2025
@come-nc come-nc added the 2. developing Work in progress label Aug 7, 2025
@come-nc come-nc force-pushed the feat/add-encryption-integration-tests branch from fc10598 to 9621c6d Compare August 7, 2025 15:03
@come-nc
Copy link
Contributor Author

come-nc commented Aug 7, 2025

Tests are working, but code is not.

decrypt-all does not work, encrypted version is not set to 0.

When doing only encrypt-all, the cleanup in the end run in "Inconsistent data, File unshared, but owner not found. Should not happen":

{
  "reqId": "yUsdYhahv4ySuATDznYS",
  "level": 3,
  "time": "2025-08-07T14:38:57+00:00",
  "remoteAddr": "",
  "user": "--",
  "app": "no app in context",
  "method": "",
  "url": "--",
  "message": "Inconsistent data, File unshared, but owner not found. Should not happen",
  "userAgent": "--",
  "version": "32.0.0.2",
  "exception": {
    "Exception": "Exception",
    "Message": "Inconsistent data, File unshared, but owner not found. Should not happen",
    "Code": 0,
    "Trace": [
      {
        "file": "/nextcloud/lib/private/Encryption/EncryptionEventListener.php",
        "line": 52,
        "function": "getUpdate",
        "class": "OC\\Encryption\\EncryptionEventListener",
        "type": "->"
      },
      {
        "file": "/nextcloud/lib/private/EventDispatcher/ServiceEventListener.php",
        "line": 57,
        "function": "handle",
        "class": "OC\\Encryption\\EncryptionEventListener",
        "type": "->"
      },
      {
        "file": "/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php",
        "line": 220,
        "function": "__invoke",
        "class": "OC\\EventDispatcher\\ServiceEventListener",
        "type": "->"
      },
      {
        "file": "/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php",
        "line": 56,
        "function": "callListeners",
        "class": "Symfony\\Component\\EventDispatcher\\EventDispatcher",
        "type": "->"
      },
      {
        "file": "/nextcloud/lib/private/EventDispatcher/EventDispatcher.php",
        "line": 67,
        "function": "dispatch",
        "class": "Symfony\\Component\\EventDispatcher\\EventDispatcher",
        "type": "->"
      },
      {
        "file": "/nextcloud/lib/private/EventDispatcher/EventDispatcher.php",
        "line": 79,
        "function": "dispatch",
        "class": "OC\\EventDispatcher\\EventDispatcher",
        "type": "->"
      },
      {
        "file": "/nextcloud/lib/private/Files/Node/HookConnector.php",
        "line": 169,
        "function": "dispatchTyped",
        "class": "OC\\EventDispatcher\\EventDispatcher",
        "type": "->"
      },
      {
        "file": "/nextcloud/lib/private/legacy/OC_Hook.php",
        "line": 85,
        "function": "postRename",
        "class": "OC\\Files\\Node\\HookConnector",
        "type": "->"
      },
      {
        "file": "/nextcloud/lib/private/Files/View.php",
        "line": 836,
        "function": "emit",
        "class": "OC_Hook",
        "type": "::"
      },
      {
        "file": "/nextcloud/apps/encryption/lib/Crypto/EncryptAll.php",
        "line": 257,
        "function": "rename",
        "class": "OC\\Files\\View",
        "type": "->"
      },
      {
        "file": "/nextcloud/apps/encryption/lib/Crypto/EncryptAll.php",
        "line": 218,
        "function": "encryptFile",
        "class": "OCA\\Encryption\\Crypto\\EncryptAll",
        "type": "->",
        "args": [
          "*** sensitive parameters replaced ***"
        ]
      },
      {
        "file": "/nextcloud/apps/encryption/lib/Crypto/EncryptAll.php",
        "line": 183,
        "function": "encryptUsersFiles",
        "class": "OCA\\Encryption\\Crypto\\EncryptAll",
        "type": "->",
        "args": [
          "*** sensitive parameters replaced ***"
        ]
      },
      {
        "file": "/nextcloud/apps/encryption/lib/Crypto/EncryptAll.php",
        "line": 157,
        "function": "encryptAllUserFilesWithMasterKey",
        "class": "OCA\\Encryption\\Crypto\\EncryptAll",
        "type": "->",
        "args": [
          "*** sensitive parameters replaced ***"
        ]
      },
      {
        "file": "/nextcloud/apps/encryption/lib/Crypto/EncryptAll.php",
        "line": 107,
        "function": "encryptAllUsersFiles",
        "class": "OCA\\Encryption\\Crypto\\EncryptAll",
        "type": "->",
        "args": [
          "*** sensitive parameters replaced ***"
        ]
      },
      {
        "file": "/nextcloud/apps/encryption/lib/Crypto/Encryption.php",
        "line": 468,
        "function": "encryptAll",
        "class": "OCA\\Encryption\\Crypto\\EncryptAll",
        "type": "->",
        "args": [
          "*** sensitive parameters replaced ***"
        ]
      },
      {
        "file": "/nextcloud/core/Command/Encryption/EncryptAll.php",
        "line": 92,
        "function": "encryptAll",
        "class": "OCA\\Encryption\\Crypto\\Encryption",
        "type": "->",
        "args": [
          "*** sensitive parameters replaced ***"
        ]
      },
      {
        "file": "/nextcloud/3rdparty/symfony/console/Command/Command.php",
        "line": 326,
        "function": "execute",
        "class": "OC\\Core\\Command\\Encryption\\EncryptAll",
        "type": "->"
      },
      {
        "file": "/nextcloud/3rdparty/symfony/console/Application.php",
        "line": 1078,
        "function": "run",
        "class": "Symfony\\Component\\Console\\Command\\Command",
        "type": "->"
      },
      {
        "file": "/nextcloud/3rdparty/symfony/console/Application.php",
        "line": 324,
        "function": "doRunCommand",
        "class": "Symfony\\Component\\Console\\Application",
        "type": "->"
      },
      {
        "file": "/nextcloud/3rdparty/symfony/console/Application.php",
        "line": 175,
        "function": "doRun",
        "class": "Symfony\\Component\\Console\\Application",
        "type": "->"
      },
      {
        "file": "/nextcloud/lib/private/Console/Application.php",
        "line": 187,
        "function": "run",
        "class": "Symfony\\Component\\Console\\Application",
        "type": "->"
      },
      {
        "file": "/nextcloud/console.php",
        "line": 90,
        "function": "run",
        "class": "OC\\Console\\Application",
        "type": "->"
      }
    ],
    "File": "/nextcloud/lib/private/Encryption/EncryptionEventListener.php",
    "Line": 76,
    "message": "Inconsistent data, File unshared, but owner not found. Should not happen",
    "exception": {},
    "CustomMessage": "Inconsistent data, File unshared, but owner not found. Should not happen"
  }
}

@come-nc
Copy link
Contributor Author

come-nc commented Aug 7, 2025

Notes regarding the uid thing:

  • Encryption event listener passes a uid to the OC\Encryption\Update class
  • It gets passed down to the encryption module
  • It’s used to decrypt the file key, to then reencrypt for all users with access to the file
  • When master key is used, this is useless I think, as it’s the same key for all users.
  • When events are triggered by occ or background job, there is no uid in session and this is broken
  • For user keys, it’s not possible anyway for occ and background jobs to get the user password, so I’m not sure what can be done
  • Current code is also broken if several event happens in the same process for different owners

I think we should remove or ignore this uid parameter, and just use the user in session for user key, and bail out if there is none.

@come-nc come-nc force-pushed the feat/add-encryption-integration-tests branch from d7684b8 to 6d149de Compare August 25, 2025 13:41
@come-nc
Copy link
Contributor Author

come-nc commented Aug 25, 2025

For decrypt-all, the root of the issue is from #48651
Since that PR, when copying an encrypted file to a non-encrypted version (because encryption have been disabled), the information from the filecache is copied over to the copy at the end, so the encrypted version is set at the value of the source file instead of being 0.
It took me a while to spot it because encrypted is first correctly set by the wrapper at 0. But View does this extra Updater call that overrides it.

I’m not sure I understand the original PR explanation, because the encryption wrapper calls $this->updateEncryptedVersion in copyBetweenStorage, and for folders it recurses, so it should correctly call it on all subfiles even several folders below.

I do not know how to cleanly fix that, since the storage wrapper does not get called after the copyUpdate happens.
Thoughts @icewind1991 and @artonge ?

@come-nc come-nc force-pushed the feat/add-encryption-integration-tests branch from d8f97a9 to 9f613a7 Compare August 25, 2025 15:59
@come-nc come-nc force-pushed the feat/add-encryption-integration-tests branch from 9f613a7 to 09647bf Compare September 2, 2025 11:48
@come-nc
Copy link
Contributor Author

come-nc commented Sep 2, 2025

So after investigating with Louis we decided to simply fix copyFromCache for the specific case of encryption being disabled and set encrypted to 0 for the target file.
This fixes decrypt-all command.
I will fix the rest of the tests and then ask for reviews, and in a follow-up PR I’ll see if I can change encrypt-all to preserve fileids.

@come-nc come-nc force-pushed the feat/add-encryption-integration-tests branch from 69bd471 to e23dfe9 Compare September 4, 2025 15:05
@come-nc come-nc changed the title feat(encryption): Add integration tests for occ commands feat(encryption): Add integration tests for occ commands and fix them Sep 4, 2025
@come-nc come-nc marked this pull request as ready for review September 4, 2025 16:06
@come-nc come-nc requested a review from a team as a code owner September 4, 2025 16:06
@come-nc come-nc requested review from Altahrim, nfebe and sorbaugh and removed request for a team September 4, 2025 16:06
@come-nc come-nc added 3. to review Waiting for reviews and removed 2. developing Work in progress labels Sep 4, 2025
@come-nc come-nc requested review from artonge and icewind1991 and removed request for sorbaugh September 4, 2025 16:06
artonge
artonge reviewed Sep 4, 2025
View reviewed changes
@come-nc
feat(encryption): Add integration tests for occ commands
55ad42a 
Add tests for encrypt-all and decrypt-all.

Signed-off-by: Côme Chilliet <[email protected]>
@come-nc
fix(encryption): Do not depend upon user in session unless really nec…
b74c753 
…essary

Should fix a bunch of stuff when encryption listener is triggered by events from occ commands or background jobs

Signed-off-by: Côme Chilliet <[email protected]>
@come-nc
chore(tests): Adapt encryption tests to code changes
7ac0856 
Signed-off-by: Côme Chilliet <[email protected]>
come-nc and others added 5 commits September 9, 2025 11:46
@come-nc
fix(encryption): Correctly set encrypted to 0 when copying
12532bb 
If encryption got disabled, copying should set encrypted to 0 for the
 new unencrypted copy. For instance when using encryption:decrypt-all

Signed-off-by: Côme Chilliet <[email protected]>
@come-nc
fix(encryption): Take encryption enabled status into account
14d6945 
shouldEncrypt now returns false for all paths if encryption is disabled.

Signed-off-by: Côme Chilliet <[email protected]>
@come-nc
fix(tests): Fix type issues and other problems with encryption tests
bc5e29f 
Signed-off-by: Côme Chilliet <[email protected]>
@come-nc @artonge
chore: Improve wording of logged error in apps/encryption/lib/KeyMana…
bfcb269 
…ger.php

Co-authored-by: Louis <[email protected]>
Signed-off-by: Côme Chilliet <[email protected]>
@come-nc
chore(encryption): Remove unused attribute $uid in KeyManager::getFil…
8330f14 
…eKey

It’s a private API in the application, no need to keep an unused
 attribute.

Signed-off-by: Côme Chilliet <[email protected]>
@come-nc come-nc force-pushed the feat/add-encryption-integration-tests branch from 5eb2661 to 8330f14 Compare September 9, 2025 09:46
@nextcloud-bot nextcloud-bot mentioned this pull request Sep 10, 2025
Merged
@susnux susnux merged commit 6cdf098 into master Sep 10, 2025
198 checks passed
@susnux susnux deleted the feat/add-encryption-integration-tests branch September 10, 2025 16:08
@come-nc
Copy link
Contributor Author

come-nc commented Sep 11, 2025

/backport to stable32

@backportbot backportbot bot mentioned this pull request Sep 11, 2025
Merged
@come-nc
Copy link
Contributor Author

come-nc commented Sep 11, 2025

/backport to stable31

@backportbot backportbot bot mentioned this pull request Sep 11, 2025
Closed
2 tasks
@come-nc
Copy link
Contributor Author

come-nc commented Sep 11, 2025

/backport to stable30

@backportbot backportbot bot mentioned this pull request Sep 11, 2025
Closed
2 tasks
@skjnldsv skjnldsv modified the milestones: Nextcloud 32, Nextcloud 33 Sep 28, 2025
@nextcloud-bot nextcloud-bot mentioned this pull request Jan 6, 2026
Merged
@joshtrichards joshtrichards mentioned this pull request Jan 6, 2026
Open
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@susnux susnux susnux approved these changes

@artonge artonge artonge approved these changes

@Altahrim Altahrim Awaiting requested review from Altahrim Altahrim is a code owner automatically assigned from nextcloud/server-backend

@nfebe nfebe Awaiting requested review from nfebe nfebe is a code owner automatically assigned from nextcloud/server-backend

@icewind1991 icewind1991 Awaiting requested review from icewind1991

@nickvergessen nickvergessen Awaiting requested review from nickvergessen

Assignees

@come-nc come-nc

Labels

3. to review Waiting for reviews feature: encryption (server-side)

Projects

None yet

Milestone

Nextcloud 32

Development

Successfully merging this pull request may close these issues.

4 participants

@come-nc @susnux @artonge @skjnldsv