Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Call detekt reflectively #2282

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
arturbosch merged 3 commits into from
Mar 5, 2020
Merged

Call detekt reflectively #2282

arturbosch merged 3 commits into from
Mar 5, 2020

Conversation

@arturbosch
Copy link
Member

@arturbosch arturbosch commented Jan 19, 2020 •
edited
Loading

Continuation of #2200 with hopefully most of the classloader leaks fixed ...
Metaspace OOM issues may still occur as every detekt run loads ~50mb of classes when using --parallel.

Following screenshot shows that the gradle daemon can unload detekt classes though:
2020-01-19-222256_704x365_scrot

Insights from previous PR:

  • hard coding the detekt version is 3x faster than our current Gradle Plugin
    • downside is that we can't dynamically set the detekt classpath (changing versions etc)
    • we can't use our plugin mechanism!
      • we may convert the extra plugin classpath to --plugins jars... parameters however when trying this I got unexpected crashes ...
  • using the project.ant method like the gradle builtin quality tools fork a jvm
    • an anttask can be run without forking the JVM, but again Metaspace
    • performance was nearly the same as now with worse logging anttask: detekt output on every line
  • Calling detekt reflectively revealed some classloader memory leaks
    • still not sure about the performance though the Metaspace issues are still present
    • I hope that with this way we can save ~200ms JVM starting overhead?
  • Did not try out the worker api though reading it's docu does not show how a javaexec could be speed up
    • To my knownledge the worker api brings performance benefits for easier tasks than starting a new process?

@arturbosch arturbosch mentioned this pull request Jan 27, 2020
Closed
@arturbosch arturbosch force-pushed the reflective-gradle-plugin branch 2 times, most recently from 501f0f1 to 8900caf Compare January 27, 2020 20:52
@codecov-io
Copy link

codecov-io commented Jan 27, 2020 •
edited
Loading

Codecov Report

Merging #2282 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##             master    #2282   +/-   ##
=========================================
  Coverage     82.82%   82.82%           
  Complexity     2138     2138           
=========================================
  Files           352      352           
  Lines          6080     6080           
  Branches       1109     1109           
=========================================
  Hits           5036     5036           
  Misses          475      475           
  Partials        569      569

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 07a83d3...b0a1d18. Read the comment docs.

@arturbosch arturbosch mentioned this pull request Jan 28, 2020
Merged
@arturbosch
Copy link
Member Author

arturbosch commented Jan 28, 2020 •
edited
Loading

I've finally managed to find a way to call detekt reflectively without an OOM error -> caching the class loader. This is actually huge as the gradle daemon now has the class loader cached and any subsequential build profits from it.
We save the startup jvm process cost (~200ms) plus loading all ~40mb classes :).
Need to further profile the performance.

Edit: Using reflection could be the way for us to keep our plugin mechanism and migrate to a kotlin compiler plugin architecture.

BraisGabin
BraisGabin reviewed Jan 29, 2020
View reviewed changes
Copy link
Member

@BraisGabin BraisGabin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did some fast measures and the improvement is huge!

@arturbosch arturbosch added this to the 1.7.0 milestone Feb 26, 2020
@arturbosch arturbosch force-pushed the reflective-gradle-plugin branch from 8900caf to b0a1d18 Compare March 2, 2020 23:39
@arturbosch arturbosch marked this pull request as ready for review March 2, 2020 23:53
@arturbosch arturbosch merged commit 4ca1f58 into master Mar 5, 2020
@arturbosch arturbosch deleted the reflective-gradle-plugin branch March 5, 2020 11:10
@arturbosch
Copy link
Member Author

arturbosch commented Mar 5, 2020

Links to #2035

@arturbosch arturbosch mentioned this pull request Mar 5, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@BraisGabin BraisGabin BraisGabin left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

1.7.0

Development

Successfully merging this pull request may close these issues.

4 participants

@arturbosch @codecov-io @BraisGabin