This is huge!! 🎉

I haven't tested this PR yet, but have you considered the issues I mentioned at #2035 (comment)? There was also some input from @marschwar and @BraisGabin, and from what I saw there were still some open questions.

Reducing build time in CI could be done by only running Gradle plugin tests on a single platform and by migrating some functional tests to unit tests. It's also possible the build cache is not working correctly here - in theory if there are changes not affecting Gradle plugin then those tests should not need to be run since results are stored in build cache. Investigating that should address the slow CI builds.

Also when comparing performance issues we should compare invocations of detektMain task on multi-module projects, not detekt task since it doesn't use type resolution which more rules will require over time. It will be faster with this PR, but I'd expect the difference in % terms to be less because detekt is quite slow when doing type resolution as we know.

This PR will also prevent testing snapshot releases of detekt with the Gradle plugin.

The Gradle plugin has issues but I think it's better to tackle those issues one by one rather than take this approach unless some of these downsides are addressed.

Reducing build time in CI could be done by only running Gradle plugin tests on a single platform and by migrating some functional tests to unit tests. It's also possible the build cache is not working correctly here - in theory if there are changes not affecting Gradle plugin then those tests should not need to be run since results are stored in build cache. Investigating that should address the slow CI builds.

I think that this PR is not about our times in CI. It's about the performance of the gradle task for any user.

Also when comparing performance issues we should compare invocations of detektMain task on multi-module projects, not detekt task since it doesn't use type resolution which more rules will require over time. It will be faster with this PR, but I'd expect the difference in % terms to be less because detekt is quite slow when doing type resolution as we know.

I'll measure it and upload the data in the issue.

I think that this PR is not about our times in CI.

Agreed, was just responding to a point made in #2035 (comment)

I haven't tested this PR yet, but have you considered the issues I mentioned at #2035 (comment)? There was also some input from @marschwar and @BraisGabin, and from what I saw there were still some open questions.

I've replied on that issue. in short: Gradle ships with kotlin-compiler-embeddable-1.3.50-patched-for-gradle-6.0.1.jar and the compiler plugin runs as a gradle+kotlinc plugin with compileOnly dependency on kotlin-compiler-embeddable.

Reducing build time in CI could be done by only running Gradle plugin tests on a single platform and by migrating some functional tests to unit tests. It's also possible the build cache is not working correctly here - in theory if there are changes not affecting Gradle plugin then those tests should not need to be run since results are stored in build cache. Investigating that should address the slow CI builds.

Right. Maybe it is the combination with the included build. Do you remember why we used a composite build for the gradle plugin?

Also when comparing performance issues we should compare invocations of detektMain task on multi-module projects, not detekt task since it doesn't use type resolution which more rules will require over time. It will be faster with this PR, but I'd expect the difference in % terms to be less because detekt is quite slow when doing type resolution as we know.

Most users do not use the type resolution features and this PR would speed up detekt for them.
For using type resolution a compiler plugin is the way to go in the future to be as fast as the kotlin compiler.

This PR will also prevent testing snapshot releases of detekt with the Gradle plugin.

What if we still allow custom cli versions but print a warning that this way is deprecated and should only be used to test snapshots?

The Gradle plugin has issues but I think it's better to tackle those issues one by one rather than take this approach unless some of these downsides are addressed.

Gradle ships with kotlin-compiler-embeddable-1.3.50-patched-for-gradle-6.0.1.jar

It's a different JAR but unless it relocates classes isn't there chance of conflict? I'm no expert though :/

Maybe it is the combination with the included build.

Could be, but as far as I can see the included build is independent of the main build, so changes in modules like detekt-api shouldn't trigger a rebuild/retest of the plugin. But looking at this now, build scan reports "Dependencies were resolved during project configuration" flagging :detekt-formatting:implementation, and gradle/gradle#5669 (comment) seems to indicate that in this case it might affect the composite build. I'll look at fixing this.

Do you remember why we used a composite build for the gradle plugin?

Yes, it was because when testing the Gradle plugin in the detekt project itself every change in the Gradle plugin required uploading the artifact to a local Maven repo before consuming it in detekt itself. That was annoying when testing changes to the plugin. An alternative to this might be using the --include-build command line flag when making changes like this and not using Gradle plugin as an included build by default, though not sure if there'll be chicken & egg issues later. More at #924

BTW I checked the build cache usage on some build scans and the Gradle plugin tests are cached properly in the build cache and restored on future CI runs, as long as the dependencies don't cause a rebuild. No low-hanging fruit here.

What if we still allow custom cli versions but print a warning that this way is deprecated and should only be used to test snapshots?

To make dev easier we want to specify the artifact, not just the version, so passing project(':detekt-cli') is still possible. Makes it much easier to work on changes that affect CLI and Gradle plugin at the same time.

Gradle ships with kotlin-compiler-embeddable-1.3.50-patched-for-gradle-6.0.1.jar

It's a different JAR but unless it relocates classes isn't there chance of conflict? I'm no expert though :/

It should be fine. I don't think they would change the API somehow but rather fix something or implement a workaround until the kotlin team fixes it holistically.

Maybe it is the combination with the included build.

Could be, but as far as I can see the included build is independent of the main build, so changes in modules like detekt-api shouldn't trigger a rebuild/retest of the plugin. But looking at this now, build scan reports "Dependencies were resolved during project configuration" flagging :detekt-formatting:implementation, and gradle/gradle#5669 (comment) seems to indicate that in this case it might affect the composite build. I'll look at fixing this.

Do you remember why we used a composite build for the gradle plugin?

Yes, it was because when testing the Gradle plugin in the detekt project itself every change in the Gradle plugin required uploading the artifact to a local Maven repo before consuming it in detekt itself. That was annoying when testing changes to the plugin. An alternative to this might be using the --include-build command line flag when making changes like this and not using Gradle plugin as an included build by default, though not sure if there'll be chicken & egg issues later. More at #924

BTW I checked the build cache usage on some build scans and the Gradle plugin tests are cached properly in the build cache and restored on future CI runs, as long as the dependencies don't cause a rebuild. No low-hanging fruit here.

What if we still allow custom cli versions but print a warning that this way is deprecated and should only be used to test snapshots?

To make dev easier we want to specify the artifact, not just the version, so passing project(':detekt-cli') is still possible. Makes it much easier to work on changes that affect CLI and Gradle plugin at the same time.

+1, I will try this now. The performance benefit should remain but overriding via detekt "....:cli:..." should be still possible.

Lets try releasing a RC-1 then and test on our projects.

Edit: Basically this means we are getting rid of the included build setup.