[SPARK-59476][SQL] Throw NULL_DATA_SOURCE_OPTION for null JDBC url instead of NullPointerException - #58766
Open
dejankrak-db wants to merge 1 commit into
Open
[SPARK-59476][SQL] Throw NULL_DATA_SOURCE_OPTION for null JDBC url instead of NullPointerException#58766dejankrak-db wants to merge 1 commit into
dejankrak-db wants to merge 1 commit into
Conversation
…stead of NullPointerException A JDBC read with a null `url` option satisfied the presence check (`parameters.isDefinedAt(JDBC_URL)`) but produced a null value, so `DriverManager.getDriver(null)` threw a raw NullPointerException that surfaced as INTERNAL_ERROR. Guard the null url in JDBCOptions and throw the user-facing NULL_DATA_SOURCE_OPTION error, mirroring the existing null-value guard in `asProperties`. Co-authored-by: Isaac <[email protected]>
HyukjinKwon
approved these changes
Sep 13, 2026
uros-b
approved these changes
Sep 13, 2026
uros-b
left a comment
Member
There was a problem hiding this comment.
Thank you @dejankrak-db and @HyukjinKwon!
uros-b
reviewed
Sep 13, 2026
| // `isDefinedAt` above only checks the key is present; the value can still be `null`. Guard it | ||
| // here so a null url surfaces as a user-facing error instead of a raw NullPointerException from | ||
| // `DriverManager.getDriver(url)` below. Mirrors the null-value guard in `asProperties`. | ||
| if (url == null) { |
Member
There was a problem hiding this comment.
@dejankrak-db Could you just please check whether this is reachable:
The new url == null check never runs. asProperties is initialized first (SPARK-48601) and already throws NULL_DATA_SOURCE_OPTION for every null option, including url. The new test should pass on master without the JDBCOptions change.
uros-b
requested changes
Sep 13, 2026
uros-b
left a comment
Member
There was a problem hiding this comment.
Requesting changes just to prevent accidental merge until we verify the behaviour. Please see comments above.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
A JDBC read with a null
urloption satisfies the presence checkrequire(parameters.isDefinedAt(JDBC_URL), ...)inJDBCOptions(the key ispresent), but the value itself is
null. That null then flows intoDriverManager.getDriver(url), which throws a rawNullPointerExceptionsurfacing to the user as an opaque
INTERNAL_ERROR.This PR guards the null value right after it is read in
JDBCOptionsand throwsthe user-facing
NULL_DATA_SOURCE_OPTIONerror instead, mirroring the existingnull-value guard already present in
JDBCOptions.asProperties.Why are the changes needed?
A null option value is a user input error, not an internal Spark failure. It
should be reported with the actionable
NULL_DATA_SOURCE_OPTIONerror class(pointing at the
urloption) rather than aNullPointerException/INTERNAL_ERROR, which is confusing and looks like a Spark bug.Does this PR introduce any user-facing change?
Yes. Constructing a JDBC relation with a null
urloption now fails with theNULL_DATA_SOURCE_OPTIONerror (aSparkIllegalArgumentException) instead of aNullPointerExceptionsurfaced asINTERNAL_ERROR. No change for valid inputs.How was this patch tested?
Added a unit test in
JdbcUtilsSuitethat constructsJDBCOptionswith a nullurland asserts theNULL_DATA_SOURCE_OPTIONerror is raised with theexpected
optionparameter.Was this patch authored or co-authored using generative AI tooling?
Generated-by: Isaac
This pull request and its description were written by Isaac.