fix: pruning does not get rid of physical files #35905
Conversation
|
Looks like this PR is not ready to merge, because of the following issues:
Please fix the issues and try again If you have any trouble, please check the PR guidelines |
|
Kody Review CompleteGreat news! π Keep up the excellent work! π Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
Codecov ReportAll modified and coverable lines are covered by tests β
Additional details and impacted files@@ Coverage Diff @@
## develop #35905 +/- ##
===========================================
- Coverage 64.65% 64.63% -0.02%
===========================================
Files 3244 3245 +1
Lines 95398 95400 +2
Branches 17853 17857 +4
===========================================
- Hits 61677 61664 -13
- Misses 30826 30838 +12
- Partials 2895 2898 +3
Flags with carried forward coverage won't be shown. Click here to find out more. π New features to boost your workflow:
|
|
Code Review Completed! π₯The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
Code Review Completed! π₯The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
Code Review Completed! π₯The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
Code Review Completed! π₯The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
Code Review Completed! π₯The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
Kody Review CompleteGreat news! π Keep up the excellent work! π Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
Code Review Completed! π₯The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
| @@ -139,7 +139,7 @@ export class Store { | |||
| // Copy file data | |||
| await store.write(rs, copyId, (err) => { | |||
| if (err) { | |||
| void this.removeById(copyId); | |||
| void this.deleteById(copyId); | |||
There was a problem hiding this comment.
void store.deleteById(copyId);The cleanup on copy failure incorrectly targets the source store instead of the destination store, potentially leaving partially created files in the target store.
This issue appears in multiple locations:
- apps/meteor/server/ufs/ufs-store.ts: Lines 142-142
Please ensure cleanup on copy failure targets the destination store by callingstore.deleteById(copyId)instead ofthis.deleteById(copyId).
Talk to Kody by mentioning @kody
Was this suggestion helpful? React with π or π to help Kody learn from this interaction.
ββ
| @@ -52,7 +52,7 @@ export async function ufsComplete(fileId: string, storeName: string, options?: { | |||
| // Clean upload if error occurs | |||
| rs.on('error', (err) => { | |||
| console.error(err); | |||
| void store.removeById(fileId, { session: options?.session }); | |||
| void store.deleteById(fileId, { session: options?.session }); | |||
There was a problem hiding this comment.
try {
await store.deleteById(fileId, { session: options?.session });
} catch (deleteErr) {
console.error('Failed to delete file:', deleteErr);
}The deleteById operation lacks error handling, which could lead to incomplete cleanup in case of failures.
This issue appears in multiple locations:
- apps/meteor/server/ufs/ufs-methods.ts: Lines 55-55
- apps/meteor/server/ufs/ufs-methods.ts: Lines 78-78
Please add error handling using try-catch blocks to ensure proper cleanup and logging of failures during thedeleteByIdoperation.
Talk to Kody by mentioning @kody
Was this suggestion helpful? React with π or π to help Kody learn from this interaction.
ββ
Code Review Completed! π₯The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
Code Review Completed! π₯The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
β¦-messages tests
Kody Review CompleteGreat news! π Keep up the excellent work! π Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
Code Review Completed! π₯The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
1 similar comment
Code Review Completed! π₯The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
β¦toast handling and settings management
Kody Review CompleteGreat news! π Keep up the excellent work! π Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
Code Review Completed! π₯The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
Code Review Completed! π₯The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
Code Review Completed! π₯The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
| // FIXME: This will fail in the afterAll step when trying to delete the channel | ||
| test.fail('delete channel with dangling files', async ({ settings }) => { | ||
| await settings.set('FileUpload_FileSystemPath', FILE_SYSTEM_PATHS.CHANGED); | ||
| await sendFileMessage(poHomeChannel, FILE_NAMES.FILE_3); | ||
| await settings.set('FileUpload_FileSystemPath', FILE_SYSTEM_PATHS.TEMPORARY); | ||
| }); | ||
|
|
||
| // FIXME: This will fail because the chat.delete API will return 400 (ENOENT) | ||
| test.fail('delete message with dangling files', async ({ settings }) => { | ||
| await settings.set('FileUpload_FileSystemPath', FILE_SYSTEM_PATHS.CHANGED); | ||
| await sendFileMessage(poHomeChannel, FILE_NAMES.FILE_3); | ||
| await settings.set('FileUpload_FileSystemPath', FILE_SYSTEM_PATHS.TEMPORARY); | ||
| await poHomeChannel.page.getByLabel('Close').click(); | ||
| await poHomeChannel.content.openLastMessageMenu(); | ||
| await poHomeChannel.page.getByRole('menuitem', { name: 'Delete' }).click(); | ||
| const modal = poHomeChannel.page.getByLabel('Are you sure?'); | ||
| await modal.getByRole('button', { name: 'Yes, delete' }).click(); | ||
| await expect(modal).not.toBeVisible(); | ||
| await expect(poHomeChannel.content.lastUserMessage).not.toBeVisible(); | ||
| }); | ||
| }); |
There was a problem hiding this comment.
If we delete the message when fs.unlink fails for any reason, then we lose track of that file and won't be able to delete it properly.
Ideally, the message gets deleted/hidden, but the reference to the file remains in the database so pruning is still attempted later.
I forced ENOENT in this test suite by changing FileUpload_FileSystemPath as just an easy way to reproduce. There are many possible errors:
Kody Review CompleteGreat news! π Keep up the excellent work! π Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
Code Review Completed! π₯The code review was successfully completed based on your current configurations. Kody Guide: Usage and ConfigurationInteracting with Kody
Current Kody ConfigurationReview OptionsThe following review options are enabled or disabled:
β |
Proposed changes (including videos or screenshots)
Issue(s)
https://rocketchat.atlassian.net/browse/SUP-745
Steps to test or reproduce
Further comments
Pull Request Description:
This pull request addresses an issue in the Rocket.Chat repository where pruning does not effectively remove physical files. The changes are made in the
apps/meteor/server/ufs/ufs-local.tsfile, specifically within theLocalStoreclass. The update enhances the file deletion process by incorporating improved error handling and type checking. While these improvements are significant, there are still opportunities to enhance security and maintainability further. The changes are proposed from theSUP-745-fix-pruning-ufs-localbranch to thedevelopbranch.