extract common pre-push and push uploading code#1071
Conversation
Adds "ref" parameter extracted from git pre-push hook into the upload request. This way servers with branch access permissions can enforce them before large objects are transfered. Issue: #845
In push hook refs comes from user and points to local branches. Transporting them to server doesn't make sense.
Don't add empty "ref" parameter to the lfs-request object when its value is empty.
Instead of passing around additional string parameter with destination ref, we pass special struct called UploadMetadata. For now UploadMetadata contains only ref field but in the future more metadata like file name (or path) can be added.
Move "ref" property of batch request into "meta" object, this way all meta data have a common place to live instead of being spread all around the batch request.
Since we also want to have metadata included in fetch request the UploadMetadata struct is renamed to TransferMetadata. Patch that adds metadata to fetch request will follow.
Include meta.ref property in download request object, same as it is in the upload request. In case of download meta.ref contain name of currently check out local branch and can be only a suggestion to server implementation.
Normalize the value of meta.ref parameter of batch request object. Previously ref parameter format differs between "download" and "upload" objects. The "upload" always was using the fully qualified branch name (eg. refs/heads/master), when the "download" used short name (eg. master). Right now meta.ref will have consistent values. When "refs/heads/" prefix will be appended when it is not present. Special "HEAD" reference will be replaced with empty string, which will result in meta.ref parameter not being included in the request object.
|
Feeling a lot better about this cli := newClient()
cli.RemoteName = lfs.Config.CurrentRemote
cli.DryRun = prePushDryRun
scanOpt := lfs.NewScanRefsOptions()
scanOpt.ScanMode = lfs.ScanLeftToRemoteMode
scanOpt.RemoteName = cli.RemoteName
pointers, err := lfs.ScanRefs(left, right, scanOpt)
if err != nil {
Panic(err, "Error scanning for Git LFS files")
}
cli.Upload(pointers) |
|
Hopefully this wasn't a bad idea (if it is, I'll just merge b535690 into #1040), but I merged #969 in here, fixing merge conflicts. It's obvious where we're not filling out the transfer meta data for |
|
Here's a compare view if you just want to review the push/pre-push changes to #1040. |
|
|
||
| return RemoteBranchForLocalBranch(r.Name) | ||
| } | ||
|
|
There was a problem hiding this comment.
This returns the remote tracking branch name for a local ref, returning the ref name as a fallback. Now, we may want to support the local:remote syntax, as seen in commands like git push heroku heroku:master. I think that'd be parsed in the command itself and passed to the initialization of the *Ref, which would be preferred over r.Name.
Thoughts?
There was a problem hiding this comment.
I think this is fine for auto selection which is what the function is designed to do, but yeah if we want to support full syntax this may be overridden; the problem is I'm not sure we get the remote-side name in the pre-push hook?
I'm somewhat skeptical of the actual usefulness of this ref in the api anyway except as a convenience hint for early rejection. It's so easy to game it's of no real use security wise.
|
|
||
| uploadedOids := lfs.NewStringSet() | ||
| cli := newClient() | ||
| cli.RemoteName = lfs.Config.CurrentRemote |
There was a problem hiding this comment.
Hmm, feels weird to have to copy the remote name when everything else uses Config directly, feels like this might be fragile - either forgetting to assign it, or other code which changes lfs.Config.CurrentRemote thinking it applies everywhere (some tests do this already I think). I think we should keep a single source of truth.
|
Are you intending to wrap downloading in clientContext too? Although there's somewhat less for it to do than in the upload case. If not, or even if 'not right away' then maybe rename clientContext to uploadContext for now. |
I have some issues with this code. But at least now pre-push and push both call shared functions to upload objects.
From #1040 (comment):
This PR is a start towards shared functions for pre-push and push commands. While it seems to work, there are a number of things I'd like to change:
lfs.Configurationlike this. I wanted something more than aStringSetto share between multiple ref uploads in the same command.EDIT: Fixed this in a later commit ^^^
Instead of promoting to the lfs package, what if we started on a high level client like we discussed in #779? I am a little concerned that my yak shave trying to get #969 merged is going on too long though :)
I don't think the commands should be assembling upload queue objects or anything. I think they should just be responsible for turning the CLI or STDIN arguments into code like this: