build: omit DWARF tables by default#1937
Conversation
| )) | ||
| } | ||
| if !*BuildDwarf || *BuildAll { | ||
| LdFlags = append(LdFlags, "-s", "-w") |
There was a problem hiding this comment.
Does it really make sense to only use -s, -w for BuildAll? From the outside, it's probably hard to comprehend why builds for a single platform are suddenly not stripped.
IMO the current default should be to always strip, regardless of BuildAll, unless BuildDwarf is given. And while at it, I'd probably rename BuildDwarf to something like BuildDebug.
If that turns out to be inconvenient during development, invert the logic and to not strip by default unless a BuildRelease is given, independently of BuildAll being given or not.
There was a problem hiding this comment.
Ah, this is an artifact of when I was playing around with the command before I pushed the PR. At this stage, the flag was --no-dwarf, so the absence of it would include the DWARF tables by default. I wanted --all to be the equivalent of --all --no-dwarf without having to type that every time.
Since the flag is now --dwarf, I went ahead and removed that implicit behavior in 5f18308. Thanks for pointing this out!
|
Does this reduce stack trace visibility at all, for when chasing down panic bugs reported from the community that we may not be able to reproduce? |
I agree. Not worth it to have a binary about 500k smaller. This looks fine to me, but I, like @sinbad, would like to see how stack traces compare on this build. |
|
@sinbad @technoweenie great question. I did test out some example stack traces when playing around with this locally, I just forgot to include them in the original pull-request. Here they are, with and without the DWARF tables: With DWARF tables
~/g/git-lfs (no-dwarf-tables) $ ./script/bootstrap --dwarf && ./bin/git-lfs logs boomtown
commands/mancontent_gen.go
Using go1.8rc3
Welcome to Boomtown
Without DWARF tables
~/g/git-lfs (no-dwarf-tables) $ ./script/bootstrap && ./bin/git-lfs logs boomtown
commands/mancontent_gen.go
Using go1.8rc3
Welcome to Boomtown
And a diff between the two: --- /Users/ttaylorr/Desktop/with-dwarf 2017-02-14 09:29:41.000000000 -0700
+++ /Users/ttaylorr/Desktop/without-dwarf 2017-02-14 09:29:47.000000000 -0700
@@ -1,6 +1,6 @@
Welcome to Boomtown
-Errors logged to /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/.git/lfs/objects/logs/20170214T092505.376310742.log
+Errors logged to /Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/.git/lfs/objects/logs/20170214T092551.184256199.log
Use `git lfs logs last` to view the log.
~/g/git-lfs (no-dwarf-tables) $ ./bin/git-lfs logs last
git-lfs/1.5.0 (GitHub; darwin amd64; go 1.8rc3; git 5f183083)Since they're the same, I think that this pull-request is ready to go. |
sinbad
left a comment
There was a problem hiding this comment.
Just one fix needed that isn't a bug now but might be depending on future use. 👍
| if len(LdFlag) > 0 { | ||
| args = append(args, "-ldflags", LdFlag) | ||
| if len(LdFlags) > 0 { | ||
| args = append(args, "-ldflags", strings.Join(LdFlags, " ")) |
There was a problem hiding this comment.
The manual joining into 1 arg could possibly have some quoting issues in the future, I suggest instead:
args = append(args, "-ldflags")
args = append(args, LdFlags...)
There was a problem hiding this comment.
I don't think this is quite how the -ldflags flag works in practice. The -ldflags expects a string rather than introducing a set of arguments. Applying this change yields an empty ldflags set, and instead tries to pass the contents of LdFlags as packages for go build to build.
~/g/git-lfs (no-dwarf-tables!) $ ./script/bootstrap && gzip < bin/git-lfs | wc -c
Using go1.8rc3
can't load package: package github.com/git-lfs/git-lfs/config.GitCommit=5f183083: cannot find package "github.com/git-lfs/git-lfs/config.GitCommit=5f183083" in any of:
/usr/local/Cellar/go/1.8rc3/libexec/src/github.com/git-lfs/git-lfs/config.GitCommit=5f183083 (from $GOROOT)
/Users/ttaylorr/go/src/github.com/git-lfs/git-lfs/config.GitCommit=5f183083 (from $GOPATH)
can't load package: package -s: cannot find package "-s" in any of:
/usr/local/Cellar/go/1.8rc3/libexec/src/-s (from $GOROOT)
/Users/ttaylorr/go/src/-s (from $GOPATH)
can't load package: package -w: cannot find package "-w" in any of:
/usr/local/Cellar/go/1.8rc3/libexec/src/-w (from $GOROOT)
/Users/ttaylorr/go/src/-w (from $GOPATH)
can't load package: package -o: cannot find package "-o" in any of:
/usr/local/Cellar/go/1.8rc3/libexec/src/-o (from $GOROOT)
/Users/ttaylorr/go/src/-o (from $GOPATH)
can't load package: package bin/git-lfs: cannot find package "bin/git-lfs" in any of:
/usr/local/Cellar/go/1.8rc3/libexec/src/bin/git-lfs (from $GOROOT)
/Users/ttaylorr/go/src/bin/git-lfs (from $GOPATH)
2017/02/14 10:02:12 exit status 1
exit status 1
There was a problem hiding this comment.
Ahh, you're right, my bad - forgot that -ldflags was its own nested beast. I guess we just deal with any quoting issues manually if/when they occur then.
|
@ttaylorr thanks for doing this. FWIW my wish is really only to keep Git for Windows' installer as lean as possible, therefore I have no qualm about including an unpacked Would you have a built |
|
@dscho my pleasure. Here are both AMD64 and 386 builds with and without the DWARF tables. I'd build the installer for you myself, but I blew away my Windows VM! git-lfs-windows-386-1.5.0-dwarf.zip |
|
Thanks, @ttaylorr. I tested this by re-generating the (LZMA-compressing) Git for Windows installer with the current binaries, then adding no-DWARF BTW is it intentional that you bundled v1.5.0 versions instead of v1.5.5 ones? When I compare the v1.5.5 release to the ones you provided, I see 3,200,602 (v1.5.5) vs 7,005,184 (v1.5.0, no-DWARF) vs 10,521,088 (v1.5.0, DWARF). Those are rather dramatic differences there... Do you have any idea why? |
🎉
Our |
|
But why are there so big differences in the sizes between the versions? |
Sorry, I'm not sure what the size difference is caused by. Here are the numbers that I get:
|
|
@ttaylorr I was referring to the file sizes of the Also, I uploaded git-lfs v1.5.6 to our Pacman repository (which is MSYS2's equivalent to Homebrew, kinda). But that release does not have your size improvement yet. Any idea which git-lfs version will have them? |
The The next release (v2.0.0) will include this change. |
This pull-request adds the
--dwarfflag toscript/bootstrap, and excludes DWARF tables from the compiled binaries by default.Here are some ✨ metrics comparing the sizes of our binaries before/after:
Building a single (compressed) binary with DWARF tables:
Building a single (compressed) binary without DWARF tables:
This shaves 22% (761265 bytes -> .76 MB) off of the compressed binaries. I ran some tests against all of the platforms/architectures we build for, and the average size reduction of the compressed releases was
With an average of a -25.63% reduction in compressed size.
A couple of notes:
script/bootstrap, since that's the way we'll build it for release, I think we should be as close to that as possible in development/test environments. I don't usedlvoften with LFS, but if you need to have the debugger symbols, you can always re-runscript/bootstrapwith the--dwarfflag.upx[1] which seems to yield an average compression of about 15-25%. Though this would further reduce the size, it would increase the startup time by anywhere from 15-160msec. A while ago, @technoweenie stated his concern in Add support for Git LFS git-for-windows/build-extra#110 (comment):Before merging, I'd like to revisit this point to see if it's still the case with the
processfilter being more prominent and startup cost being less of a concern. Personally, I'd avoid further compression withupxor similar since even the minimum of 15ms extra to launch a command seems like too much to me. Additionally, for users still rocking the legacy single-process clean/smudge filters, the overhead would be much more noticeable during large checkouts./cc @git-lfs/core @dscho