-
Notifications
You must be signed in to change notification settings - Fork 61
Reduce tiny allocations & reduce trivia size #487
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reduce tiny allocations & reduce trivia size #487
Conversation
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## master #487 +/- ##
=======================================
Coverage 83.48% 83.48%
=======================================
Files 11 11
Lines 8484 8491 +7
=======================================
+ Hits 7083 7089 +6
- Misses 1401 1402 +1
Continue to review full report in Codecov by Sentry.
|
|
So only for error messages? Sounds good. Although I'd rather have it be a small array optimization instead. So it can grow on the heap if needed. |
|
the TriviaToken change is a difference (previously the fields would always be empty) - this might break things in other packages. But I think we can rather do this now than after 1.0.0
the stackbuffer starts on the stack and only grows on the heap if needed (which it didn't in my tests) |
vushu
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems like a worthwhile optimization.
|
problem right now: std.sumtype isn't supported by the gdc-latest version - updating to gdc 12 should fix that though |
|
DCD BUILD FAILED Build statistics: ------ libdparse statistics ------
statistics (-before, +after)
-library size=3434196 libdparse.a
+library size=3510756 libdparse.a
rough build time=21s
------ DCD statistics ------
statistics (-before, +after)
client size=1055768 bin/dcd-client
-server size=3006064 bin/dcd-server
-rough build time=98s
+server size=3018608 bin/dcd-server
+rough build time=100s
-DCD run_tests.sh Elapsed (wall clock) time (h:mm:ss or m:ss): 0:05.94
-DCD run_tests.sh Maximum resident set size (kbytes): 15220
+DCD run_tests.sh Elapsed (wall clock) time (h:mm:ss or m:ss): 0:06.01
+DCD run_tests.sh Maximum resident set size (kbytes): 10528
short requests: (215x)
- min request time = 0.010ms
- 10th percentile = 0.136ms
- median time = 0.517ms
- 90th percentile = 0.929ms
- max request time = 2.024ms
+ min request time = 0.011ms
+ 10th percentile = 0.122ms
+ median time = 0.507ms
+ 90th percentile = 0.931ms
+ max request time = 2.030ms
top 5 GC sources in server:
-bytes allocated, allocations, type, function, file:line
- 7554112 81399 void[] std.array.Appender!(DSymbol*[]).Appender.ensureAddable.__lambda9 /opt/hostedtoolcache/dc/dmd-2.102.2/x64/dmd2/linux/bin64/../../src/phobos/std/array.d:3577
- 7536640 460 void[] std.array.Appender!(TokenStructure!(ubyte, "import dparse.lexer:TokenTriviaFields; mixin TokenTriviaFields;")[]).Appender.ensureAddable.__lambda9 /opt/hostedtoolcache/dc/dmd-2.102.2/x64/dmd2/linux/bin64/../../src/phobos/std/array.d:3577
- 2277376 278 void[] std.array.Appender!(char[][]).Appender.ensureAddable.__lambda9 /opt/hostedtoolcache/dc/dmd-2.102.2/x64/dmd2/linux/bin64/../../src/phobos/std/array.d:3577
- 1830176 57193 std.array.Appender!(dsymbol.symbol.DSymbol*[]).Appender.Data std.array.Appender!(DSymbol*[]).Appender.this /opt/hostedtoolcache/dc/dmd-2.102.2/x64/dmd2/linux/bin64/../../src/phobos/std/array.d:3452
- 1414144 599 void[] std.array.Appender!(const(TokenStructure!(ubyte, "import dparse.lexer:TokenTriviaFields; mixin TokenTriviaFields;"))[]).Appender.ensureAddable.__lambda9 /opt/hostedtoolcache/dc/dmd-2.102.2/x64/dmd2/linux/bin64/../../src/phobos/std/array.d:3577Full build output |
We use malloc and GC.addRange
5e0547b to
68c2cac
Compare
|
gonna just merge and fix whatever is happening on DCD's side, since I can't reproduce any issues here |
We don't save much on RAM usage / allocated bytes, but many many GC calls, especially over longer periods of DCD usage:
removed allocations running run_tests.sh with dlang-community/DCD#733 as benchmark:
it looks like it never actually exceeds the stack size, so actually no additional allocations at all from this (in exchange for about +64 bytes for
new Parser)massif indicates that
Expressionis the majority of DCD's memory, if we want to reduce memory usage of DCD:although this could just be a data interpretation issue and it means all AST nodes in general take up the majority of DCD's heap memory.