Incompatiblity of anonymous PUT request to public bucket with S3 Java SDK version 2.30 #21005
Description
When I start minio via Docker locally with a public bucket and I try to put a file there using the latest version of the S3 Java SDK and anonymous credentials, I am getting this exception:
software.amazon.awssdk.services.s3.model.S3Exception: Missing fields in request. (Service: S3, Status Code: 400, Request ID: ...
Expected Behavior
The file should be added.
Current Behavior
I am getting the above exception.
Steps to Reproduce (for bugs)
docker run -p 9000:9000 -p9001:9001 --restart=unless-stopped --name minio -e MINIO_DEFAULT_BUCKETS=my-bucket:public bitnami/minio@sha256:ca31735c5a8e6a85de77c63bea09d8780a788ca9bbf7f1d730610c5da84a5796
S3AsyncClient asyncClient = S3AsyncClient.builder()
.endpointOverride(URI.create("http://localhost:9000"))
.forcePathStyle(true)
.credentialsProvider(AnonymousCredentialsProvider.create())
.build();
final var input = new ByteArrayInputStream("file content".getBytes());
PutObjectRequest putObjectRequest = PutObjectRequest.builder()
.bucket("my-bucket")
.key("/test/filename")
.metadata(Map.of())
.contentType("text/plain")
.build();
asyncClient.putObject(putObjectRequest, AsyncRequestBody.fromInputStream(
input,
(long) input.available(),
Executors.newSingleThreadExecutor())
).get();On v2.30.31 of the Java SDK, I am getting this stack trace:
Exception in thread "main" java.util.concurrent.ExecutionException: software.amazon.awssdk.services.s3.model.S3Exception: Missing fields in request. (Service: S3, Status Code: 400, Request ID: 18298B10D2319065, Extended Request ID: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8) (SDK Attempt Count: 1)
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
...
Caused by: software.amazon.awssdk.services.s3.model.S3Exception: Missing fields in request. (Service: S3, Status Code: 400, Request ID: 18298B10D2319065, Extended Request ID: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8) (SDK Attempt Count: 1)
at software.amazon.awssdk.services.s3.model.S3Exception$BuilderImpl.build(S3Exception.java:113)
at software.amazon.awssdk.services.s3.model.S3Exception$BuilderImpl.build(S3Exception.java:61)
at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.retryPolicyDisallowedRetryException(RetryableStageHelper.java:168)
at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeAttemptExecute(AsyncRetryableStage.java:135)
Caused by: software.amazon.awssdk.services.s3.model.S3Exception: Missing fields in request. (Service: S3, Status Code: 400, Request ID: 18298B10D2319065, Extended Request ID: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8) (SDK Attempt Count: 1)
at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:152)
at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.lambda$attemptExecute$1(AsyncRetryableStage.java:123)
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2179)
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$execute$0(MakeAsyncHttpRequestStage.java:110)
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2179)
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.completeResponseFuture(MakeAsyncHttpRequestStage.java:253)
at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$executeHttpRequest$3(MakeAsyncHttpRequestStage.java:167)
at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934)
at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911)
at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
It works in version 2.29.52; there was a change to how checksums are handled in v2.30.X of the SDK.
In version 2.30.X, it works if I either specify some credentials, or calculate the checksum explicitly:
.credentialsProvider(() -> AwsBasicCredentials.create("minioadmin", "minioadmin"))or
Crc32Checksum crc32Checksum = new Crc32Checksum();
crc32Checksum.update(data, 0, data.length);
String checksum = Base64.getEncoder().encodeToString(crc32Checksum.getChecksumBytes());
// ...
PutObjectRequest putObjectRequest = PutObjectRequest.builder()
// ...
.checksumAlgorithm(ChecksumAlgorithm.CRC32)
.checksumCRC32(checksum)
// ...Context
Trying to use minio for testing before deploying code to run against S3.
Your Environment
This exact docker image:
bitnami/minio@sha256:ca31735c5a8e6a85de77c63bea09d8780a788ca9bbf7f1d730610c5da84a5796
Running on WSL:
Linux 5.15.167.4-microsoft-standard-WSL2 #1 SMP Tue Nov 5 00:21:55 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux