UFAL/The process output is not displayed because of S3 direct download #971
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WalkthroughA new method was added to BitstreamBuilder to create bitstreams without associating them with a bundle. BitstreamRestController was updated to only redirect to S3 presigned URLs for bitstreams in the "ORIGINAL" bundle. An integration test was introduced to verify that non-"ORIGINAL" bundle bitstreams are not redirected. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant BitstreamRestController
participant S3Service
Client->>BitstreamRestController: GET /bitstreams/{id}/content
alt Bitstream in "ORIGINAL" bundle and S3 direct download enabled
BitstreamRestController->>S3Service: getPresignedUrl(bitstream)
S3Service-->>BitstreamRestController: presigned URL
BitstreamRestController-->>Client: HTTP 302 Redirect to presigned URL
else Bitstream not in "ORIGINAL" bundle or S3 direct download disabled
BitstreamRestController-->>Client: Serve bitstream content directly (HTTP 200)
end
Suggested reviewers
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms (3)
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
dspace-server-webapp/src/test/java/org/dspace/app/rest/BitstreamRestControllerIT.java (1)
1418-1469: Great test coverage for the non-S3 direct download case.This test effectively validates that bitstreams outside the "ORIGINAL" bundle are served directly instead of being redirected to S3. The test sets up the environment correctly, creates a bitstream without a bundle, and verifies the expected response.
A small suggestion to make the test even more robust would be to also verify that no call to the S3DirectDownloadService was made:
+ // Verify that the S3 direct download service was never called + Mockito.verify(s3DirectDownloadService, Mockito.never()) + .generatePresignedUrl(anyString(), anyString(), anyInt(), anyString());
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Cache: Disabled due to data retention organization setting
Knowledge Base: Disabled due to data retention organization setting
📒 Files selected for processing (3)
dspace-api/src/test/java/org/dspace/builder/BitstreamBuilder.java(2 hunks)dspace-server-webapp/src/main/java/org/dspace/app/rest/BitstreamRestController.java(1 hunks)dspace-server-webapp/src/test/java/org/dspace/app/rest/BitstreamRestControllerIT.java(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (3)
- GitHub Check: dspace-dependencies / docker-build (linux/amd64, ubuntu-latest, true)
- GitHub Check: Run Integration Tests
- GitHub Check: Run Unit Tests
🔇 Additional comments (3)
dspace-server-webapp/src/main/java/org/dspace/app/rest/BitstreamRestController.java (1)
186-201: LGTM: Proper handling of S3 redirects for Original bundle only.The addition of this check to only redirect bitstreams in the "ORIGINAL" bundle to S3 presigned URLs is well-implemented. The comments explain the purpose clearly, and the streams API usage is clean and efficient.
dspace-api/src/test/java/org/dspace/builder/BitstreamBuilder.java (2)
50-57: Good addition of support for creating standalone bitstreams.This new method will help test scenarios where bitstreams aren't in a specific bundle, which is essential for the S3 download behavior change.
90-101: Implementation looks correct.The private create method properly handles the case when a bundle is not required, calling the appropriate bitstreamService method.
There was a problem hiding this comment.
Pull Request Overview
This PR addresses an issue where process output is not displayed because of the S3 direct download configuration. Key changes include:
- Adjusting the S3 download mechanism to apply only to bitstreams stored in the "ORIGINAL" bundle.
- Adding tests to verify that bitstreams not in the "ORIGINAL" bundle are correctly served without S3 redirection.
- Enhancing the BitstreamBuilder API with an overloaded create method to support conditional bundle attachment.
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| dspace-server-webapp/src/test/java/org/dspace/app/rest/BitstreamRestControllerIT.java | Added a test case for non-"ORIGINAL" bitstreams to validate direct download behavior. |
| dspace-server-webapp/src/main/java/org/dspace/app/rest/BitstreamRestController.java | Updated logic to restrict S3 direct download to bitstreams in the "ORIGINAL" bundle. |
| dspace-api/src/test/java/org/dspace/builder/BitstreamBuilder.java | Introduced an overloaded createBitstream method to optionally attach a bundle. |
Comments suppressed due to low confidence (1)
dspace-api/src/test/java/org/dspace/builder/BitstreamBuilder.java:50
- [nitpick] The parameter name 'hasBundle' may be ambiguous regarding its intent. Consider renaming it to 'attachBundle' or adding a clarifying comment to enhance readability.
public static BitstreamBuilder createBitstream(Context context, Item item, InputStream is, boolean hasBundle)
vidiecan
requested changes
May 28, 2025
dspace-server-webapp/src/main/java/org/dspace/app/rest/BitstreamRestController.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Pull Request Overview
This PR addresses the issue where the process output is not displayed due to S3 direct download by limiting S3 presigned URLs only to files stored in the ORIGINAL bundle. Key changes include:
- Updating BitstreamRestController to conditionally redirect to an S3 download URL only when the file belongs to the ORIGINAL bundle.
- Adding a new test in BitstreamRestControllerIT to ensure that files outside the ORIGINAL bundle are served directly.
- Introducing an overloaded createBitstream method in BitstreamBuilder to support creating bitstreams with or without an attached bundle.
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| dspace-server-webapp/src/test/java/org/dspace/app/rest/BitstreamRestControllerIT.java | Added a new test for direct download behavior when file is not in the ORIGINAL bundle. |
| dspace-server-webapp/src/main/java/org/dspace/app/rest/BitstreamRestController.java | Updated S3 direct download logic with a check for the ORIGINAL bundle. |
| dspace-api/src/test/java/org/dspace/builder/BitstreamBuilder.java | Added overloaded methods to optionally flag if a bitstream comes with a bundle. |
Comments suppressed due to low confidence (2)
dspace-server-webapp/src/test/java/org/dspace/app/rest/BitstreamRestControllerIT.java:1420
- [nitpick] Consider renaming the test method 'testNotS3DirectDownloadWhenSpecificFile' to a more descriptive name such as 'testDirectDownloadBypassForNonOriginalBundle' to clearly reflect that the test verifies files outside the ORIGINAL bundle are not redirected.
@Test
public void testNotS3DirectDownloadWhenSpecificFile() throws Exception {
dspace-api/src/test/java/org/dspace/builder/BitstreamBuilder.java:50
- [nitpick] Consider renaming the boolean parameter 'hasBundle' to a more descriptive term (e.g., 'attachBundle') to clarify the intent and improve readability of the method signature.
public static BitstreamBuilder createBitstream(Context context, Item item, InputStream is, boolean hasBundle)
vidiecan
approved these changes
Jun 4, 2025
milanmajchrak
added a commit
that referenced
this pull request
Jun 16, 2025
* UFAL/DOI - Added type of resource to data cite (#975) * UFAL/The process output is not displayed because of S3 direct download (#971) * Check if type is html (#983) * Ufal dtq sync062025 (#985) * we should identify as clarin-dspace * update email templates to use dspace.shortname * match v5 submission * get rid of lr.help.phone * Add option to configure oai sample identifier * NRP deposit * UFAL/Improve file preview generating (#972) --------- Co-authored-by: milanmajchrak <[email protected]> * Fix the file preview integration test (#989) * The hasPreview method has been changed, but the IT wasn't updated correctly --------- Co-authored-by: Paurikova2 <[email protected]> Co-authored-by: Ondřej Košarko <[email protected]>
milanmajchrak
added a commit
that referenced
this pull request
Jul 10, 2025
* UFAL/DOI - Added type of resource to data cite (#975) * UFAL/The process output is not displayed because of S3 direct download (#971) * The S3 direct download is provided only for the files located in the ORIGINAL bundle * Use constant for the ORIGINAL string value * Check if type is html (#983) * check if type is html * added test for html mime type * used static string for text/html, added check * Ufal dtq sync062025 (#985) * we should identify as clarin-dspace Fix test (cherry picked from commit 6cdf2d1) * update email templates to use dspace.shortname dspace.name can be a long string not fit for Email subjects nor signatures (cherry picked from commit 98d60dd) * match v5 submission (cherry picked from commit 4a2b65f) * get rid of lr.help.phone Phone is now conditional in the templates. Use `mail.message.helpdesk.telephone` if you want it. The change in the *.java files is to preserve the params counts. The relevant templates are getting the phone directly from config (cherry picked from commit cba5695) * Add option to configure oai sample identifier some validators use this value, should be a real id in prod deployments (cherry picked from commit 912f13f) * NRP deposit license (cherry picked from commit ba23878) * Fix ufal#1219 Get rid of setting the jsse.enableSNIExtension property which causes issues with handle minting (cherry picked from commit 7d03173) * UFAL/Improve file preview generating (#972) * get name and size from metadata and header of file, avoid input stream using * remove temp file, checkstyle, do not load full file * add { } after if * added check for max preview file * used ZipFile and TarArchived for filepreview generating * added removed lines * used 7z for zip and tar files * removed 7z and used zip and tar entry * improved file previrew generating speed, used string builder, xml builder, authorization only if is required * checkstyle, return boolean from haspreview and previrews from getPreview, replaced return with continue * fix problem with hibernate session * fix .tar.gz generating * skip fully entry for tar * added indexes for speed up queries * added license header * named constant by upper case * inicialized fileInfo, refactorization of code based on copilot review --------- Co-authored-by: milanmajchrak <[email protected]> * Fix the file preview integration test (#989) * The hasPreview method has been changed, but the IT wasn't updated correctly * Use the correct checkbox for the input field - use repeatable (#991) * UFAL/EU Sponsor openaire id should not be required (#1001) * EU Sponsor openaire id should not be required * Not required also in the czech submission forms * Logging error message while emailing users (#1000) * Logging error message --------- Co-authored-by: Matus Kasak <[email protected]> Co-authored-by: milanmajchrak <[email protected]> * UFAL/Teaching and clariah submissions does not have clarin-license (#1005) * UFAL/Fix logging in LogoImportController (#1003) * fix logging * used formatter for msg * UFAL/Update the resource policy rights when changing submitter (#1002) * removed res policies for submitter and created newones when item is shared * avoid magic number, use constant * set submitter in existing res policies * removed not used shared link * UFAL/Added date to title when creating new version (#984) * added date to versioned item title * used more modern approach for getting current time * renamed test * used var for reusing * UFAL/Item handle info in email after download request (#1006) * Added item handle to email * Exception when item not found * Checked grammar * Handled multiple items found by bitstream * Using PID instead of handle --------- Co-authored-by: Matus Kasak <[email protected]> --------- Co-authored-by: Paurikova2 <[email protected]> Co-authored-by: Ondřej Košarko <[email protected]> Co-authored-by: Kasinhou <[email protected]> Co-authored-by: Matus Kasak <[email protected]> Co-authored-by: jurinecko <[email protected]>
kosarko
added a commit
to ufal/clarin-dspace
that referenced
this pull request
Jul 14, 2025
dataquest-dev/dtq-dev sync Syncing dtq-dev ~lindat-2025.07.16198085191. This contains the following changes: UFAL/DOI - Added type of resource to data cite (dataquest-dev#975) Check if type is html (dataquest-dev#983) UFAL/Improve file preview generating (dataquest-dev#972) UFAL/Fix logging in LogoImportController (dataquest-dev#1003) UFAL/Update the resource policy rights when changing submitter (dataquest-dev#1002) UFAL/Added date to title when creating new version (dataquest-dev#984) UFAL/The process output is not displayed because of S3 direct download (dataquest-dev#971) Fix the file preview integration test (dataquest-dev#989) Use the correct checkbox for the input field - use repeatable (dataquest-dev#991) UFAL/EU Sponsor openaire id should not be required (dataquest-dev#1001) Logging error message while emailing users (dataquest-dev#1000) UFAL/Item handle info in email after download request (dataquest-dev#1006) Ufal dtq sync062025 (dataquest-dev#985) Merge commit '33d330a' into HEAD UFAL/Teaching and clariah submissions does not have clarin-license (dataquest-dev#1005)
milanmajchrak
added a commit
that referenced
this pull request
Jul 24, 2025
* UFAL/DOI - Added type of resource to data cite (#975) * UFAL/The process output is not displayed because of S3 direct download (#971) * The S3 direct download is provided only for the files located in the ORIGINAL bundle * Use constant for the ORIGINAL string value * Check if type is html (#983) * check if type is html * added test for html mime type * used static string for text/html, added check * Ufal dtq sync062025 (#985) * we should identify as clarin-dspace Fix test (cherry picked from commit 6cdf2d1) * update email templates to use dspace.shortname dspace.name can be a long string not fit for Email subjects nor signatures (cherry picked from commit 98d60dd) * match v5 submission (cherry picked from commit 4a2b65f) * get rid of lr.help.phone Phone is now conditional in the templates. Use `mail.message.helpdesk.telephone` if you want it. The change in the *.java files is to preserve the params counts. The relevant templates are getting the phone directly from config (cherry picked from commit cba5695) * Add option to configure oai sample identifier some validators use this value, should be a real id in prod deployments (cherry picked from commit 912f13f) * NRP deposit license (cherry picked from commit ba23878) * Fix ufal#1219 Get rid of setting the jsse.enableSNIExtension property which causes issues with handle minting (cherry picked from commit 7d03173) * UFAL/Improve file preview generating (#972) * get name and size from metadata and header of file, avoid input stream using * remove temp file, checkstyle, do not load full file * add { } after if * added check for max preview file * used ZipFile and TarArchived for filepreview generating * added removed lines * used 7z for zip and tar files * removed 7z and used zip and tar entry * improved file previrew generating speed, used string builder, xml builder, authorization only if is required * checkstyle, return boolean from haspreview and previrews from getPreview, replaced return with continue * fix problem with hibernate session * fix .tar.gz generating * skip fully entry for tar * added indexes for speed up queries * added license header * named constant by upper case * inicialized fileInfo, refactorization of code based on copilot review --------- Co-authored-by: milanmajchrak <[email protected]> * Fix the file preview integration test (#989) * The hasPreview method has been changed, but the IT wasn't updated correctly * Use the correct checkbox for the input field - use repeatable (#991) * UFAL/EU Sponsor openaire id should not be required (#1001) * EU Sponsor openaire id should not be required * Not required also in the czech submission forms * Logging error message while emailing users (#1000) * Logging error message --------- Co-authored-by: Matus Kasak <[email protected]> Co-authored-by: milanmajchrak <[email protected]> * UFAL/Teaching and clariah submissions does not have clarin-license (#1005) * UFAL/Fix logging in LogoImportController (#1003) * fix logging * used formatter for msg * UFAL/Update the resource policy rights when changing submitter (#1002) * removed res policies for submitter and created newones when item is shared * avoid magic number, use constant * set submitter in existing res policies * removed not used shared link * UFAL/Added date to title when creating new version (#984) * added date to versioned item title * used more modern approach for getting current time * renamed test * used var for reusing * UFAL/Item handle info in email after download request (#1006) * Added item handle to email * Exception when item not found * Checked grammar * Handled multiple items found by bitstream * Using PID instead of handle --------- Co-authored-by: Matus Kasak <[email protected]> * UFAL/Incorrect password hash funct used during migration (#999) * password in request is already hashed, used different password hash funct * renamed password param in eperson endpoint * [devOps] labelling reviewing process * [devOps] labelling reviewing process * UFAL/New version keeps the old identifier * UFAL/Send email to editor after submitting item (#1016) Co-authored-by: Matus Kasak <[email protected]> * UFAL/Local file size is 0 for file with no zero size (#1017) * update item metadata after the bitstream size has changed * issue 1241: ItemFilesMetadataRepair script implementation (DSpace#1243) (#1021) * issue 1241: ItemFilesMetadataRepair script implementation * extend script to be applicabble for all items, and for items with files metadata that have missing bitstreams (files) * implement dry-run option * option description fix * Improve error message * Use "0" instead of "" + 0 * Improve error message (cherry picked from commit 706f6f6) Co-authored-by: kuchtiak-ufal <[email protected]> * UFAL/Refbox upgrade (#1015) * Created integration test * Created an endpoint for complete ref box information like in the v5 * Added integration tests for formatting authors * Removed double semicolon * Fetch the metadata value following the current locale * Updated firstMetadataValue because it did return empty string instead of null * Use DEFAULT_LANGUAGE instead of current locale * UFAL/Added doc - issue link (#1023) --------- Co-authored-by: Paurikova2 <[email protected]> Co-authored-by: Ondřej Košarko <[email protected]> Co-authored-by: Kasinhou <[email protected]> Co-authored-by: Matus Kasak <[email protected]> Co-authored-by: jurinecko <[email protected]> Co-authored-by: jm <jm@maz> Co-authored-by: kuchtiak-ufal <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem description
Summary by CodeRabbit