yandex: fix truncated files being uploaded successfully when the source ends early - #9786
Merged
Merged
Conversation
…ce ends early Update already wrapped the source in a counting reader but never looked at the count, so a source that supplied fewer bytes than its declared size was uploaded as a chunked request, accepted by the server and reported as a success with a truncated file stored. Compare the bytes actually read against the declared size.
ncw
approved these changes
Aug 21, 2026
ncw
left a comment
Member
There was a problem hiding this comment.
Nice fix - thank you.
The FsPutShortEOF integration test against a real Yandex remote and the fix behaves exactly as intended: the upload now fails with "expected 10240 bytes in input, but got 5120: unexpected EOF" instead of succeeding
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Continues the "source ends early" sweep (5d05754 sia, a2baa97 pikpak, e1bf940 filelu,
7357fb8 internetarchive, 884b28c azurefiles, 18fa445 compress).
Problem
(*Object).Updatealready wraps the source in a counting reader:but
BytesRead()is never called anywhere in the package, so the count is discarded.The upload itself is a
PUTwithBody: in1and noContentLength, so it is chunked: asource that stops short just sends a shorter body, the server accepts it, and
Updatecarries on to set the modtime and returns success with a truncated file stored.
Fix
Compare the bytes actually read against the declared size after the upload, using the
counter that is already there. No new imports.
Verification
go build ./...,go test ./backend/yandex/...andgolangci-lint run ./backend/yandex/...(0 issues) all pass.
RCLONE_CONFIG=/notfound go test ./...is unchanged from master:cmd/gitannexandcmd/serve/s3fail identically before and after this commit on mymachine (no
git-annexorminiobinary locally). No Yandex remote here, so the backendintegration tests have not been run against a real remote.
Companion to #9785 (box), same class.