Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add dependency override for shapeless #1453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 14, 2024
Merged

Add dependency override for shapeless #1453

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8f0ce3f
Add dependency override for shapeless
jeremyrsmith Nov 6, 2024
8f24638
Different version for 2.13
jeremyrsmith Nov 6, 2024
13b25af
Move override to common settings
jeremyrsmith Nov 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Add dependency override for shapeless 
Resolves a compiletime/runtime mismatch when Spark is in user classpath
  • Loading branch information
@jeremyrsmith
jeremyrsmith committed Nov 6, 2024
commit 8f0ce3f72b54a29de91608db7878c27605974ba7
1 change: 1 addition & 0 deletions build.sbt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,7 @@ val `polynote-kernel` = project.settings(
"com.github.javaparser" % "javaparser-symbol-solver-core" % versions.javaparser,
"org.scalamock" %% "scalamock" % "4.4.0" % "test"
),
dependencyOverrides += "com.chuusai" %% "shapeless" % "2.3.2",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to pick a specific version based on what matches Spark?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I think so. I wish there were a way to tell sbt, "use the version that you get from Spark, not a later version" but there isn't as far as I know.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there no like "provided" dependency? compileOnly or whatever?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is provided... but I don't see how that would solve this issue?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, by "compiletime/runtime" I mean cell compile time and cell runtime. The issue is that the shapeless in polynote-kernel-assembly (which needs to be there in the absence of Spark) gets used during compile time for some reason, and its macros are what get executed. But during runtime, Spark's shapeless gets used, which doesn't have the code that the macro referred to.

distFiles := Seq(assembly.value) ++ (Compile / dependencyClasspath).value.collect {
case jar if jar.data.name.matches(".*scala-(library|reflect|compiler|collection-compat|xml).*") => jar.data
},
Expand Down
Loading