Introduce tooling api module #2861

arturbosch · 2020-07-12T10:14:02Z

Sorry for the huge PR upfront. Please review this commit by commit.

I want to make this right without a breaking change and an API which will work for the gradle, sonar, maven, bazel, intellij and cli integrations of the core module.

Why an extra tooling api?

All mentioned integrations use core directly via the horrible ProcessingSettings class.
cli is/(was) more core than core
Introduce tooling-api #2860

Notable additional changes in this PR:

Running a specified single rule is supported by the tooling api. Cli's SingleRunner implementation was removed.
KtFileModifier is now just an extension and has no special treatment in the core classes.
ProcessingSettings is still public but only used internally in core. We can better refactor/remove this class in the future. Edit: I've refactored the class.
core decides when to fail the build via a MaxIssuePolicy sealed class.
UnstableApi annotation now has a reason property.
createRunner test helper which uses NullPrintStream by default.
formatting module does not depend on core anymore!

Usages:

cli: in this PR
gradle-plugin: TODO
intellij-plugin: Use tooling-api to call detekt detekt-intellij-plugin#94
sonar-detekt: Use detekt's tooling api sonar-detekt#125
compiler-plugin: Use tooling api to call detekt detekt-compiler-plugin#1

codecov · 2020-07-12T10:22:33Z

Codecov Report

Merging #2861 into master will decrease coverage by 0.49%.
The diff coverage is 80.51%.

@@             Coverage Diff              @@
##             master    #2861      +/-   ##
============================================
- Coverage     80.78%   80.28%   -0.50%     
- Complexity     2346     2417      +71     
============================================
  Files           388      421      +33     
  Lines          7050     7359     +309     
  Branches       1288     1346      +58     
============================================
+ Hits           5695     5908     +213     
- Misses          709      756      +47     
- Partials        646      695      +49

Impacted Files	Coverage Δ	Complexity Δ
...n/io/gitlab/arturbosch/detekt/api/ConsoleReport.kt	`100.00% <ø> (ø)`	`5.00 <0.00> (ø)`
.../kotlin/io/gitlab/arturbosch/detekt/cli/CliArgs.kt	`100.00% <ø> (ø)`	`3.00 <0.00> (-5.00)`
.../io/gitlab/arturbosch/detekt/cli/Configurations.kt	`0.00% <0.00%> (-84.62%)`	`0.00 <0.00> (ø)`
...in/kotlin/io/gitlab/arturbosch/detekt/core/Junk.kt	`60.00% <ø> (+16.25%)`	`0.00 <0.00> (ø)`
...io/gitlab/arturbosch/detekt/core/KtFileModifier.kt	`0.00% <0.00%> (ø)`	`0.00 <0.00> (ø)`
...arturbosch/detekt/core/baseline/BaselineHandler.kt	`72.22% <ø> (ø)`	`15.00 <0.00> (ø)`
...kt/core/config/DefaultPropertiesConfigValidator.kt	`100.00% <ø> (ø)`	`1.00 <0.00> (ø)`
...itlab/arturbosch/detekt/core/extensions/Loading.kt	`66.66% <ø> (+66.66%)`	`0.00 <0.00> (ø)`
...tlab/arturbosch/detekt/core/reporting/Colorizer.kt	`76.92% <0.00%> (ø)`	`0.00 <0.00> (ø)`
...otlin/io/gitlab/arturbosch/detekt/cli/CliRunner.kt	`7.14% <7.14%> (ø)`	`1.00 <1.00> (?)`
... and 129 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 265b217...ffcde93. Read the comment docs.

BraisGabin

I can't continue with the review right now. I leave the current comments and I'll try to finish the review this nigh or tomorrow.

I really like this refecator. Great job!

detekt-cli/build.gradle.kts

detekt-core/build.gradle.kts

detekt-core/src/main/kotlin/io/gitlab/arturbosch/detekt/core/config/DefaultConfig.kt

detekt-core/src/main/kotlin/io/gitlab/arturbosch/detekt/core/tooling/DefaultAnalysisResult.kt

detekt-tooling/src/main/kotlin/io/github/detekt/tooling/api/AnalysisResult.kt

detekt-core/src/main/kotlin/io/gitlab/arturbosch/detekt/core/tooling/AnalysisFacade.kt

detekt-core/src/main/kotlin/io/gitlab/arturbosch/detekt/core/tooling/Lifecycle.kt

detekt-cli/src/main/kotlin/io/gitlab/arturbosch/detekt/cli/runners/ConfigExporter.kt

detekt-core/src/main/kotlin/io/gitlab/arturbosch/detekt/core/KtTreeCompiler.kt

arturbosch · 2020-07-12T19:32:00Z

I'm not sure why jacoco fails on detekt-core with Java8 but I can reproduce this locally...

arturbosch · 2020-07-12T19:45:15Z

I'm not sure why jacoco fails on detekt-core with Java8 but I can reproduce this locally...

This is a really strange one!
The stacktrace on the console is:

But taking a look at the gradle test report I get:

org.gradle.api.internal.tasks.testing.TestSuiteExecutionException: Could not complete execution for Gradle Test Executor 15.
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:63)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:133)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: org/junit/platform/commons/util/BlacklistedExceptions
	at org.junit.platform.launcher.core.DefaultLauncher.discoverEngineRoot(DefaultLauncher.java:187)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:168)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:132)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
	... 25 more
Caused by: java.lang.ClassNotFoundException: org.junit.platform.commons.util.BlacklistedExceptions
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 32 more

So Gradle can't find org.junit.platform.commons.util.BlacklistedExceptions at runtime.

Now I stumbled upon this issue junit-team/junit-framework#2322 ....

BraisGabin

Nice one. Huge, but really good. ProcessingSettings is kind of a god class but I think that we can think about it later.

BraisGabin · 2020-07-13T17:27:19Z

detekt-core/src/main/kotlin/io/gitlab/arturbosch/detekt/core/reporting/Colorizer.kt

 }

-private val isColoredOutputSupported: Boolean = !IS_WINDOWS
+private val isColoredOutputSupported: Boolean = !SystemInfo.isWindows


detekt-tooling/src/main/kotlin/io/github/detekt/tooling/api/Detekt.kt

Add tooling support for MaxIssuePolicy.

… failure logic

Removes special treatment inside DetektFacade

…lass

…moving and reworking existing test cases

arturbosch · 2020-07-14T14:35:44Z

I've fixed the build/classloader issue.
The PR is ready.
I will try to increase the code coverage now.

Edit: the code coverage changes on rules modules are buggy in this PR and should not decrease on master.

BraisGabin · 2020-07-14T16:32:12Z

Should we deprecate Main.buildRunner() from :detekt-cli?

arturbosch · 2020-07-14T18:13:32Z

Should we deprecate Main.buildRunner() from :detekt-cli?

Already done :)

detekt/detekt-cli/src/main/kotlin/io/gitlab/arturbosch/detekt/cli/Main.kt

Lines 39 to 47 in 7314928

    
           @NotApiButProbablyUsedByUsers 
        
           @Deprecated( 
        
               "Don't build a runner yourself.", 
        
               ReplaceWith( 
        
                   "DetektCli.load().run(args, outputPrinter, errorPrinter)", 
        
                   "io.github.detekt.tooling.api.DetektCli" 
        
               ) 
        
           ) 
        
           fun buildRunner(

BraisGabin · 2020-07-14T18:53:22Z

Should we deprecate Main.buildRunner() from :detekt-cli?

Already done :)

detekt/detekt-cli/src/main/kotlin/io/gitlab/arturbosch/detekt/cli/Main.kt

Lines 39 to 47 in 7314928

@NotApiButProbablyUsedByUsers

@Deprecated(

"Don't build a runner yourself.",

ReplaceWith(

"DetektCli.load().run(args, outputPrinter, errorPrinter)",

"io.github.detekt.tooling.api.DetektCli"

)

)

fun buildRunner(

ups! my bad. I had this as an annotation in my note book from when I did the code review and I forgot to comment it at that moment and then I didn't recheck.

BraisGabin self-requested a review July 12, 2020 11:50

BraisGabin reviewed Jul 12, 2020

View reviewed changes

BraisGabin mentioned this pull request Jul 12, 2020

Exclusion patterns #2851

Closed

BraisGabin approved these changes Jul 13, 2020

View reviewed changes

arturbosch added 18 commits July 13, 2020 22:36

Prototype a configuration dsl

6dce98b

Implement a service returning the default configuration

8c56459

Implement a service returning the runtime detekt version

3c26983

Add logging and issues specific configuration to ProcessingSpec

4c2ac4e

Define interfaces to load detekt instances during runtime

234b9f0

Use new DetektProvider in cli Runner for detekt analysis

90ff614

Rename IssuesSpec to RulesSpec

f53853d

Support to run chosen single or multiple rules via spec api

89f54e8

Move logic to check for build error to core

1f64f62

Add tooling support for MaxIssuePolicy.

Make Runner config validation testcases independent of possible build…

a457cc4

… failure logic

Simplify test runner creation

e98b124

Make KtFileModifier a FileProcessListener

e9c719d

Removes special treatment inside DetektFacade

Allow to add a reason UnstableApi annotation

6e3a94f

Remove unnecessary DetektFacade.create factory functions

87807a5

Implement remaining DetektFacade functions based on a new Lifecycle c…

200480c

…lass

Fix using excludes

5d45c64

Fix using given BindingContext instead of creating one

d429fcd

Rename to AnalysisFacade and Analyzer for core classes

9c3ecd5

arturbosch force-pushed the tooling-api branch from edd8fd1 to c34dc6f Compare July 14, 2020 12:40

arturbosch added 4 commits July 14, 2020 14:56

Support running a single from core module

3c864d9

Unwrap the unexpected error before throwing it

f96eedb

Move LicenceHeader test to rules module where it is declared

62beb58

Finalize migration to loading Config from Spec instead of CliArgs by …

23fae31

…moving and reworking existing test cases

arturbosch added 14 commits July 14, 2020 14:56

Move configUris outside the constructor

ff87983

Allow extensions to be disabled via their ids in ExtensionSpec

dc84186

Remove unused constructs

7b860a9

Deprecate convenient print method only usable for PrintStreams

3337fb9

Remove obsolete ExitCode enum

414a176

Make core a runtimeOnly dependency

56bb129

Make all detekt-core dependencies implementation-dependencies

b71e379

Make DefaultConfig object internal

0b4c815

Make AnalysisResult.exitCode an extension function

7314928

Remove BindingTrace entry point

4337b86

Make using a parsing strategy explicit

7cc5d3b

Use yml to be consistent with other naming

9b3378e

Remove unnecessary factory method for KtTreeCompiler

88a8c55

Make Throwable branch the else branch

2ab16da

arturbosch force-pushed the tooling-api branch from c34dc6f to 2ab16da Compare July 14, 2020 12:57

arturbosch marked this pull request as ready for review July 14, 2020 12:57

Close only self constructed URLClassloaders

95b1ef5

arturbosch added this to the 1.11.0 milestone Jul 14, 2020

arturbosch added 3 commits July 14, 2020 17:13

Test exit codes from AnalysisResult

9fb4ef1

Test either plugin paths or a classloader is used

cc7c103

Test loading a DetektCli instance

ffcde93

arturbosch merged commit 5a0fe62 into master Jul 15, 2020

arturbosch deleted the tooling-api branch July 15, 2020 17:55

arturbosch modified the milestones: 1.11.0, 1.10.1 Aug 2, 2020

chao2zhang mentioned this pull request Nov 2, 2020

Throw error instead of logging as error in analysis phase #3193

Merged

Uh oh!

Introduce tooling api module #2861

Introduce tooling api module #2861

Uh oh!

Conversation

arturbosch commented Jul 12, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Jul 12, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

BraisGabin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

arturbosch commented Jul 12, 2020

Uh oh!

arturbosch commented Jul 12, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BraisGabin left a comment

Choose a reason for hiding this comment

Uh oh!

BraisGabin Jul 13, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

arturbosch commented Jul 14, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BraisGabin commented Jul 14, 2020

Uh oh!

arturbosch commented Jul 14, 2020

Uh oh!

BraisGabin commented Jul 14, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

arturbosch commented Jul 12, 2020 •

edited

Loading

codecov bot commented Jul 12, 2020 •

edited

Loading

arturbosch commented Jul 12, 2020 •

edited

Loading

arturbosch commented Jul 14, 2020 •

edited

Loading