Closed
Description
Loving this tool so far! I recently ran into an issue where it seems that performing a Put or a Get operation against my local S3 bucket is collapsing XML-like structures. This in turn causes MD5 digest validation errors.
I have an S3 bucket...
aws --endpoint-url=http://localhost:4572 s3api create-bucket --bucket my-bucket
I start with a file called test.xml:
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl"?>
<submission>
<field1>fwefwefewfew</field1>
<field2>fwefwefwe</field2>
<field3>fefwefwe</field3>
</submission>
I then put this object into S3 using the following command:
aws --endpoint-url=http://localhost:4572 s3api put-object --bucket my-bucket --key data/test1 --body test.xml
I'll grab the file back using the following command:
aws --endpoint-url=http://localhost:4572 s3api get-object --bucket my-bucket --key data/test1 test-back.xml
The file now looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl"?>
<submission><field1>fwefwefewfew</field1><field2>fwefwefwe</field2><field3>fefwefwe</field3></submission>
I found this while troubleshooting the following exception stack:
com.amazonaws.SdkClientException: Unable to verify integrity of data download. Client calculated content hash didn't match hash calculated by Amazon S3. The data may be corrupt.
at com.amazonaws.services.s3.internal.DigestValidationInputStream.validateMD5Digest(DigestValidationInputStream.java:79)
at com.amazonaws.services.s3.internal.DigestValidationInputStream.read(DigestValidationInputStream.java:61)
at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:72)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at com.amazonaws.util.IOUtils.toByteArray(IOUtils.java:44)
at com.amazonaws.util.IOUtils.toString(IOUtils.java:58)
at com.amazonaws.services.s3.AmazonS3Client.getObjectAsString(AmazonS3Client.java:1485)
I was able to work around this bug when I collapsed the XML structure myself prior to adding the file to S3. This is not ideal though.
Any ideas on why this is happening? I am using localstack version 0.8.8, installed with pip.
Thanks for any help!