fix(bigquery): retry cancel job and routine creation on transient HTTP 5xx errors#13416
Conversation
There was a problem hiding this comment.
Code Review
This pull request wraps the callable in BigQueryRetryHelper.runWithRetries with a translating callable that catches IOException and throws a BigQueryException. The feedback suggests refactoring the anonymous Callable class into a lambda expression to improve readability and conciseness.
6e90263 to
3234b7c
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request updates BigQueryRetryHelper.java to wrap the execution of the callable in a translatingCallable. This wrapper catches any IOException thrown during the retry execution and translates it into a BigQueryException. There are no review comments provided, and the changes appear to be correct and complete.
3234b7c to
7f7a7d7
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request updates BigQueryRetryHelper to wrap callables in a translating callable that catches HttpResponseException and rethrows it as a BigQueryException. The reviewer noted that catching only HttpResponseException is too restrictive and suggested catching IOException instead to properly handle and retry other transient network issues.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request updates BigQueryRetryHelper.java to wrap the execution of the callable in a translating callable. This translating callable catches HttpResponseException and wraps it in a BigQueryException before rethrowing. There are no review comments, and I have no feedback to provide.
7f7a7d7 to
f9b53b0
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request wraps the default BigQuery exception handler in BigQueryRetryHelper to explicitly retry on HttpResponseException for server error status codes (500, 502, 503, and 504). A new unit test is added to verify retries on 503 errors. The feedback suggests delegating createNextAttempt to the default algorithm instead of returning null to ensure robust timing logic delegation.
| if (algorithm == BigQueryBaseService.DEFAULT_BIGQUERY_EXCEPTION_HANDLER) { | ||
| algorithm = wrapDefaultAlgorithm(algorithm); | ||
| } |
There was a problem hiding this comment.
Can these changes be made on the specific RPCs that we want to add it to? I'm worried that this would retry on some non-query RPCs (e.g. createJob or something) which we may not want to retry.
There was a problem hiding this comment.
Thanks for the feedback. I've updated the code to apply the retries for:
1. Read Metadata RPCs
bigQueryRpc.getDatasetSkipExceptionTranslation(viagetDataset(...))bigQueryRpc.getTableSkipExceptionTranslation(viagetTable(...))bigQueryRpc.getJobSkipExceptionTranslation(viagetJob(...))bigQueryRpc.getIamPolicySkipExceptionTranslation(viagetIamPolicy(...))
2. List RPCs
bigQueryRpc.listDatasetsSkipExceptionTranslation(vialistDatasets(...))bigQueryRpc.listTablesSkipExceptionTranslation(vialistTables(...))bigQueryRpc.listJobsSkipExceptionTranslation(vialistJobs(...))bigQueryRpc.listModelsSkipExceptionTranslation(vialistModels(...))bigQueryRpc.listRoutinesSkipExceptionTranslation(vialistRoutines(...))bigQueryRpc.listTableDataSkipExceptionTranslation(vialistTableData(...))bigQueryRpc.getQueryResultsSkipExceptionTranslation(viagetQueryResults(...))
3. Job Actions
bigQueryRpc.cancelSkipExceptionTranslation(viacancel(...)— resolvesb/467066417)
4. IAM & Permissions
bigQueryRpc.testIamPermissionsSkipExceptionTranslation(viatestIamPermissions(...))
5. Routine Creation (Specifically added for UDF Test)
bigQueryRpc.createSkipExceptionTranslation(Routines only) (viacreateRoutine(...)— resolvesb/467066596)
fb05963 to
f195928
Compare
…P 5xx errors (#13416) Retries are applied to: ### 1. Read Metadata RPCs * **`bigQueryRpc.getDatasetSkipExceptionTranslation`** (via `getDataset(...)`) * **`bigQueryRpc.getTableSkipExceptionTranslation`** (via `getTable(...)`) * **`bigQueryRpc.getJobSkipExceptionTranslation`** (via `getJob(...)`) * **`bigQueryRpc.getIamPolicySkipExceptionTranslation`** (via `getIamPolicy(...)`) ### 2. List RPCs * **`bigQueryRpc.listDatasetsSkipExceptionTranslation`** (via `listDatasets(...)`) * **`bigQueryRpc.listTablesSkipExceptionTranslation`** (via `listTables(...)`) * **`bigQueryRpc.listJobsSkipExceptionTranslation`** (via `listJobs(...)`) * **`bigQueryRpc.listModelsSkipExceptionTranslation`** (via `listModels(...)`) * **`bigQueryRpc.listRoutinesSkipExceptionTranslation`** (via `listRoutines(...)`) * **`bigQueryRpc.listTableDataSkipExceptionTranslation`** (via `listTableData(...)`) * **`bigQueryRpc.getQueryResultsSkipExceptionTranslation`** (via `getQueryResults(...)`) ### 3. Job Actions * **`bigQueryRpc.cancelSkipExceptionTranslation`** (via `cancel(...)` — resolves `b/467066417`) ### 4. IAM & Permissions * **`bigQueryRpc.testIamPermissionsSkipExceptionTranslation`** (via `testIamPermissions(...)`) ### 5. Routine Creation (Specifically added for UDF Test) * **`bigQueryRpc.createSkipExceptionTranslation` (Routines only)** (via `createRoutine(...)` — resolves `b/467066596`) b/467066417, b/467066596
🤖 I have created a release *beep* *boop* --- <details><summary>1.88.0</summary> ## [1.88.0](v1.87.1...v1.88.0) (2026-07-06) ### Features * **auth:** Add support for Regional Access Boundaries ([#13499](#13499)) ([b721e43](b721e43)) * automate libraries-bom tag and release creation on release publish ([#13655](#13655)) ([7a7e2ff](7a7e2ff)) * **bigquery-jdbc:** add `EnableProjectDiscovery` connection property for metadata methods ([#13344](#13344)) ([ab9669a](ab9669a)) * **bigquery-jdbc:** migrate statement execution thread tracking to connection-scoped executor ([#13454](#13454)) ([341e51a](341e51a)) * **bigquery-jdbc:** respect standard JVM trustStore properties by default ([#13435](#13435)) ([7af3224](7af3224)) * **bigquery-jdbc:** support connection-scoped executor isolation and dynamic queue warnings ([#13413](#13413)) ([79e26b8](79e26b8)) * **bigquery:** add internal listProjects API to core client ([#13429](#13429)) ([3580407](3580407)) * **bigtable:** route point read rows to shim ([#13542](#13542)) ([2c7e5f5](2c7e5f5)) * **deps:** add jspecify to shared dependencies ([#13412](#13412)) ([582053d](582053d)) * enable self-signed JWTs by default in ServiceOptions ([#13338](#13338)) ([110d2b7](110d2b7)) * **gapic-generator:** Add NullMarked annotation to generated classes ([#13517](#13517)) ([825dadd](825dadd)) * **google/cloud/agentregistry/v1:** onboard a new library ([#13509](#13509)) ([6728816](6728816)) * scale up connection worker pool based on latency ([#13384](#13384)) ([eb379b3](eb379b3)) * **spanner:** add settings for gRCP keep-alive ([#13643](#13643)) ([9d78307](9d78307)) * **spanner:** auth login support for Spanner Omni endpoints ([#13470](#13470)) ([55930b4](55930b4)) * **storage:** add checksum validation in the json read channel ([#13270](#13270)) ([872d7b7](872d7b7)) * **storage:** add checksum validation on json read paths ([#13269](#13269)) ([396b042](396b042)) * **storage:** add full object checksum validation for bidi flow ([#13266](#13266)) ([9113d80](9113d80)) * **storage:** add full object checksum validation for grpc flow ([#13265](#13265)) ([a5ba606](a5ba606)) * **storage:** Adding CumulativeHasher wrapper class for Full object … ([#13239](#13239)) ([bd40324](bd40324)) * **storage:** adding disabling option for bidi reads ([#13506](#13506)) ([591cae0](591cae0)) * **storage:** log additional bytes received from GCS in read path ([#13427](#13427)) ([9492aa2](9492aa2)) * **storage:** update compose sample to support deleteSourceObjects option ([#13493](#13493)) ([50a8658](50a8658)) * use self-signed JWTs in Spanner MutableCredentials ([#13381](#13381)) ([29543a2](29543a2)) ### Bug Fixes * Add logging-logback to gapic-libraries-bom ([#13663](#13663)) ([512e370](512e370)) * **auth:** Fix UserCredentials serialization clientSecret leak ([#13465](#13465)) ([2d9d01c](2d9d01c)) * **auth:** handle missing APPDATA on Windows gracefully ([#13471](#13471)) ([77e84a9](77e84a9)), closes [#12565](#12565) * **bigquery-jdbc:** add proper version to BigQueryDriver ([#13294](#13294)) ([9fd84fc](9fd84fc)) * **bigquery-jdbc:** avoid rollback on statement close in manual commit mode ([#13503](#13503)) ([79bbee1](79bbee1)) * **bigquery-jdbc:** correct `FilterTablesOnDefaultDataset` fallback logic ([#13625](#13625)) ([f7eb23d](f7eb23d)) * **bigquery-jdbc:** ensure largeResults are handled in PreparedStatement ([#13496](#13496)) ([d3e7a19](d3e7a19)) * **bigquery-jdbc:** ensure test uses unique dataset & cleans up ([#13587](#13587)) ([a6bb362](a6bb362)) * **bigquery-jdbc:** explicitly set location when available for temporary dataset ([#13508](#13508)) ([40cd0a7](40cd0a7)) * **bigquery-jdbc:** propagate connection proxy settings to auth library ([#13539](#13539)) ([87dda5d](87dda5d)) * **bigquery-jdbc:** shade org.slf4j ([#13547](#13547)) ([7ef1312](7ef1312)) * **bigquery-jdbc:** update format validation tests to create tables for the test ([#13588](#13588)) ([4bb3a61](4bb3a61)) * **bigquery-jdbc:** update Maven Central artifact link in the README ([#13563](#13563)) ([8707a65](8707a65)) * **bigquery:** retry cancel job and routine creation on transient HTTP 5xx errors ([#13416](#13416)) ([35eb041](35eb041)) * **bigquery:** route JOB_CREATION_REQUIRED through fast query path ([#13437](#13437)) ([64cde7f](64cde7f)) * **bigquery:** Update fast query path to allow jobTimeoutMs in the request ([#13502](#13502)) ([1a6f4d5](1a6f4d5)) * **bigtable:** honour session_load override when server-returned session_load is 0 ([#13629](#13629)) ([b56f9b8](b56f9b8)) * **bqjdbc:** pass rowsInPage with TableResult ([#13238](#13238)) ([203a91e](203a91e)) * **ci:** build dependencies before convergence check ([#13638](#13638)) ([9de3209](9de3209)) * **ci:** ignore SNAPSHOT suffix in flatten-plugin check ([#13639](#13639)) ([bb65627](bb65627)) * **ci:** run root install in dependency convergence check ([#13666](#13666)) ([86d35e4](86d35e4)) * **ci:** skip existing-versions-check for gapic-generator-java-root ([#13590](#13590)) ([a1f3b8a](a1f3b8a)) * **datastore:** Configure TraceExporter with Datastore credentials in E2E tests ([#13627](#13627)) ([d281a62](d281a62)) * **datastore:** disable built-in OpenTelemetry SDK when metrics export is disabled ([#13543](#13543)) ([53b7142](53b7142)) * **datastore:** reduce flakiness of E2E tracing tests ([#13645](#13645)) ([8afc0ab](8afc0ab)) * **deps:** update dependency com.google.apis:google-api-services-bigquery to v2-rev20260612-2.0.0 ([#13570](#13570)) ([ca5ff79](ca5ff79)) * **deps:** update dependency com.google.apis:google-api-services-dns to v1-rev20260616-2.0.0 ([#12796](#12796)) ([0dd60ca](0dd60ca)) * **deps:** update dependency com.google.cloud:google-cloud-pubsub-bom to v1.151.0 ([#12097](#12097)) ([d169006](d169006)) * **deps:** update dependency com.google.cloud:libraries-bom to v26.84.0 ([#12663](#12663)) ([95c7774](95c7774)) * **deps:** update first-party storage dependencies to v1-rev20260524-2.0.0 ([#13406](#13406)) ([bdeedb8](bdeedb8)) * exclude java-vertexai from existing versions check ([#13632](#13632)) ([edaa4e3](edaa4e3)) * fallback on VPC ([#13567](#13567)) ([e8c428d](e8c428d)) * **java-cloud-bom:** execute pre-install pass in GHA release notes generator ([#13608](#13608)) ([c365c10](c365c10)) * **release:** check remote tags using git ls-remote to prevent already exists error ([#13574](#13574)) ([6cf0567](6cf0567)) * **spanner:** fail fast when inline-begin DML returns no transaction id ([#13536](#13536)) ([94315ce](94315ce)) * **spanner:** pin inline read-write transactions to default endpoint ([#13562](#13562)) ([6908dee](6908dee)) * **spanner:** remove explicit tink version to resolve dependency convergence conflict ([#13602](#13602)) ([863c26e](863c26e)) * **spanner:** update dynamic channel pool default configuration ([#13646](#13646)) ([37b77c3](37b77c3)) ### Dependencies * Add org.bouncycastle:bcprov-jdk18on to java-shared-dependencies ([#13531](#13531)) ([e3d2082](e3d2082)) * **auth:** Update commons-codec to v1.18.0 ([#13598](#13598)) ([fd89957](fd89957)) * Update gapic-showcase to v0.41.0 ([#13582](#13582)) ([09faaac](09faaac)) * Upgrade google-http-client to v2.1.1 ([#13578](#13578)) ([6abef19](6abef19)) ### Documentation * add instructions for manual code generation checks in CI ([#13596](#13596)) ([f6162e2](f6162e2)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). --------- Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Retries are applied to:
1. Read Metadata RPCs
bigQueryRpc.getDatasetSkipExceptionTranslation(viagetDataset(...))bigQueryRpc.getTableSkipExceptionTranslation(viagetTable(...))bigQueryRpc.getJobSkipExceptionTranslation(viagetJob(...))bigQueryRpc.getIamPolicySkipExceptionTranslation(viagetIamPolicy(...))2. List RPCs
bigQueryRpc.listDatasetsSkipExceptionTranslation(vialistDatasets(...))bigQueryRpc.listTablesSkipExceptionTranslation(vialistTables(...))bigQueryRpc.listJobsSkipExceptionTranslation(vialistJobs(...))bigQueryRpc.listModelsSkipExceptionTranslation(vialistModels(...))bigQueryRpc.listRoutinesSkipExceptionTranslation(vialistRoutines(...))bigQueryRpc.listTableDataSkipExceptionTranslation(vialistTableData(...))bigQueryRpc.getQueryResultsSkipExceptionTranslation(viagetQueryResults(...))3. Job Actions
bigQueryRpc.cancelSkipExceptionTranslation(viacancel(...)— resolvesb/467066417)4. IAM & Permissions
bigQueryRpc.testIamPermissionsSkipExceptionTranslation(viatestIamPermissions(...))5. Routine Creation (Specifically added for UDF Test)
bigQueryRpc.createSkipExceptionTranslation(Routines only) (viacreateRoutine(...)— resolvesb/467066596)b/467066417, b/467066596