box: fix truncated files being uploaded successfully when the source ends early - #9785
Merged
Merged
Conversation
…ends early The single-shot upload path sent the source straight to Box as a multipart body with no Content-Length, so a source that supplied fewer bytes than its declared size produced a short request that Box accepted and stored, and the upload was reported as a success. Count the bytes actually read and fail the upload if they do not match the declared size. The multipart path already reads each chunk with io.ReadFull and so already fails in this case.
ncw
approved these changes
Aug 21, 2026
ncw
left a comment
Member
There was a problem hiding this comment.
Thank you, nice fix.
I checked the integration tests
=== RUN TestIntegration/FsMkdir/FsPutShortEOF/Simple
fstests.go:2596: Put returned an error (acceptable): expected 10240 bytes in input, but got 5120: unexpected EOF
👍
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Continues the "source ends early" sweep (5d05754 sia, a2baa97 pikpak, e1bf940 filelu,
7357fb8 internetarchive, 884b28c azurefiles, 18fa445 compress).
Problem
(*Object).upload— the single-shot path taken when the size is at or belowupload_cutoff(50 MiB by default) — passes the source straight to Box as the multipartbody:
No
ContentLengthis set, so the request is chunked and a source that stops short simplyproduces a shorter body. Box accepts it, returns one entry, and the upload is reported as a
success with a truncated file stored.
uploadMultipartis not affected: it fills each chunk withio.ReadFull, which returnsio.ErrUnexpectedEOFwhen the source ends early.Fix
Wrap the body in
readers.NewCountingReaderand compare the bytes actually read against thedeclared size, matching the sia fix.
sizeis already computed inUpdateand passed touploadMultipart; it is now passed touploadas well.Verification
go build ./...,go test ./backend/box/...andgolangci-lint run ./backend/box/...(0 issues) all pass.
RCLONE_CONFIG=/notfound go test ./...is unchanged from master:cmd/gitannexandcmd/serve/s3fail identically before and after this commit on mymachine (no
git-annexorminiobinary locally). No Box remote here, so the backendintegration tests have not been run against a real remote.
Possibly the same, not yet checked
While looking for this I noticed four more backends that stream the source with no
ContentLengthand no byte-count check, so they may have the same problem — flagging ratherthan claiming, as I have not confirmed any of them:
yandex(yandex.go:1135),gofile(gofile.go:1450),filefabric(
filefabric.go:1276),googlephotos(googlephotos.go:1198).Happy to follow up on those if useful.