Add support for reference clone#1007
Conversation
9fc129c to
2088b84
Compare
|
Thanks for the PR. I'd love to get this into v1.2 (#844). I don't have a lot of experience with git references though, so it may take a little bit to review and test this properly though. |
There was a problem hiding this comment.
I think this should be wrapped in a utility function (same sequence used in fetch too, DRY)
|
This is looking good; I'd like to do a bit more testing with it too but I'd support this for 1.2. It's similar to the 'shared object store' idea I've tinkered with before, but I like that this uses an existing git principle. I'm also glad to see that hard links are now supported on Windows via os.Link in go 1.5; that wasn't the case in earlier Go versions and I've had write my own Windows implementation before. Benefitting from that does depend on NTFS partitions but your fallbacks cope with that. 👍 |
2088b84 to
46a24b8
Compare
There was a problem hiding this comment.
I would make this safer by doing 2 things:
- Validate using
os.SameFileto make suresrcanddstare not the same - Instead of calling
CopyFileContentswithdstas the direct output, instead copy to a temp file and thenos.Renameon success. This will be more robust under unusual cases like forced termination.
There was a problem hiding this comment.
@sinbad Thanks for your feedback. About the temporary file location: do you think it would be better to use os.TempDir() or store the tempfile in the destination directory? Latter is probably better from performance point of view because os.TempDir() might be in a different file system.
|
Other than the safety suggestions this works great for me. It actually behaves better than core git, which can potentially break if the referenced repo is garbage collected on shared objects (can't happen here because the local repo LFS object is either a hard link or a copy). Nice work! |
There was a problem hiding this comment.
I tend to prefer smaller guard clauses. What about:
func IfNoLocalObjExistsLinkOrCopyFromReferenceMedia(oid string, size int64) error {
if ObjectExistsOfSize(oid, size) {
return nil
}
// rest of the function
}Also, I'm not thrilled with the name IfNoLocalObjExistsLinkOrCopyFromReferenceMedia. What about just LinkOrCopyFromReference ?
|
First, thanks again for posting this, and for your patience while I take my sweet time reviewing it. I have two bigger pieces of feedback: According to gitrepository-layout, The second, is that I'm not sure we need to link/copy those files from alternates to the current repository. It could result in a lot of duplicate files. It seems like I'd say once the safety issue is resolved, and |
When it's a hardlink it won't actually be a duplicate, just a reference counted file object shared in both places. The only time this wouldn't work & fall back on a copy is if the reference is on a separate filesystem, or on Windows if you're using FAT instead of NTFS (should be pretty unlikely now). I actually like the way it's done here better than the way git does it with |
|
I think the safety aspect is less important here than in Git. Git needs every object locally, otherwise basic operations don't work. The only thing Git LFS absolutely must have in the local |
|
Personally I think that both of you have valid points here and maybe the behaviour should be configurable. E.g. one typical use case would be that the reference repository would be located on a network share. Now, if one is cloning a personal working repository to a laptop, copying the content to the local disk would be naturally preferable. On the other hand on a CI server it could safe lots of space if one could avoid the data duplication and simply refer to the network share. |
That's a great use case for the current behavior. I'm cool with shipping this now, and adding my suggestion later possibly 👍 |
- check that copy source and destination are not the same path - atomic file copy operation by using tempfile and move
46a24b8 to
bfb28d2
Compare
| } | ||
| defer func() { | ||
| tmp.Close() | ||
| _ = os.Remove(tmp.Name()) |
There was a problem hiding this comment.
Not worth changing unless you have to make another change, but I tend to omit the _ = if I'm not going to do anything with it. Just some microfeedback :)
- Close instead of Sync since we are already done with the file - Style: Omit return value for os.Remove()
|
Thanks for all the tweaks! I still think there are more improvements that can be made, and will add them to the roadmap if you or someone else decides to take it on. |
|
Thank you once more both of you for your valuable reviews! I'm happy to get it merged and I hope I will have the opportunity to work on the improvements later on too. |
Fixes #459 and possibly helps for #515 and #766.