Codestin Search App

urosstan-db · 2026-09-12T17:54:13Z

What changes were proposed in this pull request?

Report invalid JDBC date and timestamp partition bounds with SparkIllegalArgumentException, condition INVALID_JDBC_PARTITION_BOUND, and SQLSTATE 42616.
Include the invalid option (lowerBound or upperBound), supplied value, and expected data type in the error message.
Use 42616 ("Invalid options specified"), following Avro and Protobuf option-validation errors.
Draft pending an Apache Spark JIRA.

Why are the changes needed?

Invalid temporal bounds currently throw a plain IllegalArgumentException without an error condition or SQLSTATE.
The existing message, such as Cannot parse the bound value {bound} as timestamp, does not identify which JDBC option to correct.

Does this PR introduce any user-facing change?

Yes. Clearer error messages for invalid JDBC date and timestamp bounds when partitioned scan is requested. Validation logic is unchanged.

How was this patch tested?

Added JDBCRelationSuite coverage for invalid lower and upper bounds across DATE, TIMESTAMP, and TIMESTAMP_NTZ, including placeholders, empty strings, and invalid calendar dates. Positive cases check the generated partition predicates.

Updated the existing JDBCSuite timezone-bound regression test to check the structured error.

build/sbt "sql/testOnly org.apache.spark.sql.execution.datasources.jdbc.JDBCRelationSuite" \
  "core/testOnly org.apache.spark.SparkThrowableSuite" \
  "sql/testOnly org.apache.spark.sql.jdbc.JDBCSuite -- -z columnPartition"

Both affected test expectations now use 42616; these suites have not been rerun for the SQLSTATE-only update.
```
build/sbt "catalyst/scalastyle" "sql/scalastyle" "sql/Test/scalastyle"
```

Was this patch authored or co-authored using generative AI tooling?

Generated-by: OpenAI Codex CLI 0.153.4

urosstan-db · 2026-09-12T18:11:16Z

+import org.apache.spark.sql.catalyst.analysis.caseSensitiveResolution
+import org.apache.spark.sql.types.{DateType, StructType, TimestampNTZType, TimestampType}
+
+class JDBCRelationSuite extends SparkFunSuite {


We can convert all tests to integration tests as well, but I think current approach (one integration test, full matrix with unit tests) is a good trade-off between test cost and coverage.

urosstan-db · 2026-09-12T18:36:25Z

+    "message" : [
+      "Cannot parse the value <value> for JDBC option <option> as <dataType>."
+    ],
+    "sqlState" : "42616"


Using 42616 ("Invalid options specified") here. Summary of the SQLSTATE investigation:

The closest existing precedents are Avro Boolean option parsing and Protobuf Boolean/integer option parsing. Both report STDS_INVALID_OPTION_VALUE.WITH_MESSAGE, whose SQLSTATE is 42616. The state data source also uses this family for invalid option values.

Other formats are mixed: CSV quote/escape length and lineSep validation, and CSV/JSON charset validation, use 22023; Parquet/ORC unavailable-codec errors use 56038. There is no single SQLSTATE used for all data-source option errors.

JDBC has many checks, but its most comparable validations still have no SQLSTATE:

Validation Existing error SQLSTATE

Invalid integer options (numPartitions, queryTimeout, fetchsize, batchsize) Direct .toInt / NumberFormatException None

batchsize < 1, missing required partition options, or lower bound greater than upper bound require / IllegalArgumentException None

Invalid isolationLevel _LEGACY_ERROR_TEMP_2081 None

Write numPartitions <= 0 _LEGACY_ERROR_TEMP_2087 None

Conflicting, missing, or empty dbtable / query _LEGACY_ERROR_TEMP_2078 through 2080 None

Null option value NULL_DATA_SOURCE_OPTION 22024

hint unsupported by the dialect HINT_UNSUPPORTED_FOR_JDBC_DIALECT 42822

Sources: JDBCOptions, bound ordering, write partition count, and legacy error definitions. The legacy conditions omit SQLSTATE; using SparkIllegalArgumentException does not supply one implicitly.

Schema-valued JDBC options reuse parser/schema errors: createTableColumnTypes has syntax errors (42601), duplicate columns (42711), and unsupported types (0A000), as covered in JDBCWriteSuite. The nearby JdbcDialects.getBuiltInDialect API uses 22023 for unknown names, but that validates an API argument, not a data-source option.

So this choice follows the Avro/Protobuf option-conversion precedent, not an established JDBC convention. 22023 ("invalid parameter value") is the more portable standard category; 42616 is DB2-origin and nonstandard, but already established in Spark for this kind of option validation. The JDBC-specific condition preserves the option name, rejected value, and expected type.

urosstan-db · 2026-09-13T15:04:12Z

Code review — Claude Opus (xhigh reasoning)

Verdict: 0 blocking, 0 non-blocking, 0 nits. A clean, well-scoped, backward-compatible change with good test coverage.

What the change does

Invalid JDBC lowerBound/upperBound bounds on a date/timestamp partition column previously threw a plain IllegalArgumentException ("Cannot parse the bound value X as timestamp") — no error condition, no SQLSTATE, and no indication of which option was wrong. This PR reports them through a structured condition INVALID_JDBC_PARTITION_BOUND (SQLSTATE 42616, "Invalid options specified"): Cannot parse the value <value> for JDBC option <option> as <dataType>. A new factory QueryCompilationErrors.invalidJdbcPartitionBoundError builds it, and JDBCRelation.toInternalBoundValue now receives the option name so the failure identifies which bound failed.

Verification (static)

SparkIllegalArgumentException extends IllegalArgumentException, so the exception-type change stays source/binary-compatible for existing catch/intercept[IllegalArgumentException] callers; only the message and the added condition/SQLSTATE change.
SQLSTATE 42616 is defined in error-states.json and is the code the Avro/Protobuf option-validation errors use; the new condition is placed in correct alphabetical order.
The factory's toDSOption/toSQLConfVal/toSQLType each double-quote their argument, matching the checkError parameters in both suites; JDBC_LOWER_BOUND/JDBC_UPPER_BOUND equal "lowerBound"/"upperBound".
The error is raised inside a by-name getOrElse, so only on parse failure; the success return is unchanged.
Scope is intentionally limited to DATE/TIMESTAMP/TIMESTAMP_NTZ; numeric partition columns keep their existing value.toLong (NumberFormatException) path, unchanged.

This is a static review — the two affected suites were not executed here (the description notes they weren't rerun after the SQLSTATE-only update), but the code and test expectations agree on 42616 and the message parameters.

Notes (non-blocking)

Title is still [WIP] and the description says the PR is pending an Apache Spark JIRA — worth resolving both before merge.

_{Automated review generated by Claude Opus at xhigh reasoning effort.}

urosstan-db

Codex review

No findings. 0 blocking, 0 non-blocking, 0 nits.

No changes recommended; the existing review requests are addressed.

Verification

Reviewed all five changed files, the temporal parsers, error formatting, exception inheritance, and JDBC V1/V2 callers. Both read paths use the same partition routine, and the new exception remains an IllegalArgumentException. The tests cover both bound names for all three temporal types and preserve valid predicate expectations. The efficiency scanner found no issues.

Tests were not run during this review.

urosstan-db · 2026-09-13T15:17:46Z

  },
+  "INVALID_JDBC_PARTITION_BOUND" : {
+    "message" : [
+      "Cannot parse the value <value> for JDBC option <option> as <dataType>."


Suggested change

"Cannot parse the value <value> for JDBC option <option> as <dataType>."

"Cannot parse the value <value> for JDBC data source option <option> as <dataType>."

urosstan-db added 3 commits September 12, 2026 16:54

[SQL] Report structured errors for invalid JDBC partition bounds

c6a5815

[SQL] Update timestamp bound regression test for structured error

641422c

[SQL] Use style-compatible quoting in JDBC bound tests

75349a9

urosstan-db commented Sep 12, 2026

View reviewed changes

Comment thread common/utils/src/main/resources/error/error-conditions.json Outdated

urosstan-db commented Sep 12, 2026

View reviewed changes

Comment thread sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRelation.scala Outdated

urosstan-db commented Sep 12, 2026

View reviewed changes

urosstan-db added 2 commits September 12, 2026 18:16

[SQL] Clarify JDBC bound option parameter names

fcf4293

[SQL] Use invalid-options SQLSTATE for JDBC partition bounds

abe545f

urosstan-db commented Sep 12, 2026

View reviewed changes