Thanks to visit codestin.com
Credit goes to github.com

Skip to content

SF#147 resuming a multipart upload only skips up to 1000 existing files/parts  #346

@mdomsch

Description

@mdomsch

(From https://sourceforge.net/p/s3tools/bugs/147/)
Milestone: Enhancement_request
Status: open
Owner: nobody
Labels: None
Priority: 5
Updated: 3 hours ago
Created: 3 hours ago
Creator: anierman
Private: No

A multipart upload failed after several thousand 15MB files/parts had been uploaded.

After restarting the upload with put and --upload-id only the first 1000 files/parts were skipped due to md5/size matches. Re-uploading began at file/part 1001.

The issue is that list_multipart in S3.py only returns 1000 records at most, see:
http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListParts.html

To skip all of the appropriate matches--past 1000--IsTruncated could be detected in the response and appropriate output retrieved in a loop (making use of part-number​-marker).

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions