feat(storage): Adding CumulativeHasher wrapper class for Full object …#13239
Merged
Conversation
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces the CumulativeHasher class, which wraps a Hasher to track and validate cumulative CRC32C checksums for full object reads. It also updates the visibility of existing checksum exception constructors in Hasher.java. The review feedback highlights a logic gap where direct calls to hash methods were not being accumulated, potentially leading to incorrect cumulative totals. Additionally, suggestions were made to mark the new exception class as final and to use Locale.US in string formatting for consistency with existing patterns in the codebase.
Comment on lines
+45
to
+88
| @Override | ||
| public Crc32cLengthKnown hash(ByteBuffer b) { | ||
| return delegate.hash(b); | ||
| } | ||
|
|
||
| @Override | ||
| public Crc32cLengthKnown hash(ByteString byteString) { | ||
| return delegate.hash(byteString); | ||
| } | ||
|
|
||
| @Override | ||
| public void validate(Crc32cValue<?> expected, Supplier<ByteBuffer> b) throws ChecksumMismatchException { | ||
| ByteBuffer byteBuffer = b.get(); | ||
| Crc32cLengthKnown actual = delegate.hash(byteBuffer); | ||
| if (actual != null) { | ||
| if (expected != null && !actual.eqValue(expected)) { | ||
| throw new ChecksumMismatchException(expected, actual); | ||
| } | ||
| accumulate(actual); | ||
| } | ||
| } | ||
|
|
||
| @Override | ||
| public void validate(Crc32cValue<?> expected, ByteString byteString) throws ChecksumMismatchException { | ||
| Crc32cLengthKnown actual = delegate.hash(byteString); | ||
| if (actual != null) { | ||
| if (expected != null && !actual.eqValue(expected)) { | ||
| throw new ChecksumMismatchException(expected, actual); | ||
| } | ||
| accumulate(actual); | ||
| } | ||
| } | ||
|
|
||
| @Override | ||
| public void validateUnchecked(Crc32cValue<?> expected, ByteString byteString) | ||
| throws UncheckedChecksumMismatchException { | ||
| Crc32cLengthKnown actual = delegate.hash(byteString); | ||
| if (actual != null) { | ||
| if (expected != null && !actual.eqValue(expected)) { | ||
| throw new UncheckedChecksumMismatchException(expected, actual); | ||
| } | ||
| accumulate(actual); | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
The CumulativeHasher should accumulate checksums for all data processed, including calls to the hash methods. Currently, hash(ByteBuffer) and hash(ByteString) only delegate without updating cumulativeHash. Updating these methods to accumulate and then having the validate methods leverage them ensures consistency and reduces duplication.
@Override
public Crc32cLengthKnown hash(ByteBuffer b) {
Crc32cLengthKnown actual = delegate.hash(b);
if (actual != null) {
accumulate(actual);
}
return actual;
}
@Override
public Crc32cLengthKnown hash(ByteString byteString) {
Crc32cLengthKnown actual = delegate.hash(byteString);
if (actual != null) {
accumulate(actual);
}
return actual;
}
@Override
public void validate(Crc32cValue<?> expected, Supplier<ByteBuffer> b) throws ChecksumMismatchException {
Crc32cLengthKnown actual = hash(b.get());
if (actual != null && expected != null && !actual.eqValue(expected)) {
throw new ChecksumMismatchException(expected, actual);
}
}
@Override
public void validate(Crc32cValue<?> expected, ByteString byteString) throws ChecksumMismatchException {
Crc32cLengthKnown actual = hash(byteString);
if (actual != null && expected != null && !actual.eqValue(expected)) {
throw new ChecksumMismatchException(expected, actual);
}
}
@Override
public void validateUnchecked(Crc32cValue<?> expected, ByteString byteString)
throws UncheckedChecksumMismatchException {
Crc32cLengthKnown actual = hash(byteString);
if (actual != null && expected != null && !actual.eqValue(expected)) {
throw new UncheckedChecksumMismatchException(expected, actual);
}
}
nidhiii-27
approved these changes
Jun 15, 2026
nidhiii-27
left a comment
Contributor
There was a problem hiding this comment.
As discussed offline, there would be some modifications in the constructor in a different PR.
ShreyasSinha
requested changes
Jun 16, 2026
ShreyasSinha
approved these changes
Jun 16, 2026
This was referenced Jul 1, 2026
lqiu96
pushed a commit
that referenced
this pull request
Jul 7, 2026
🤖 I have created a release *beep* *boop* --- <details><summary>1.88.0</summary> ## [1.88.0](v1.87.1...v1.88.0) (2026-07-06) ### Features * **auth:** Add support for Regional Access Boundaries ([#13499](#13499)) ([b721e43](b721e43)) * automate libraries-bom tag and release creation on release publish ([#13655](#13655)) ([7a7e2ff](7a7e2ff)) * **bigquery-jdbc:** add `EnableProjectDiscovery` connection property for metadata methods ([#13344](#13344)) ([ab9669a](ab9669a)) * **bigquery-jdbc:** migrate statement execution thread tracking to connection-scoped executor ([#13454](#13454)) ([341e51a](341e51a)) * **bigquery-jdbc:** respect standard JVM trustStore properties by default ([#13435](#13435)) ([7af3224](7af3224)) * **bigquery-jdbc:** support connection-scoped executor isolation and dynamic queue warnings ([#13413](#13413)) ([79e26b8](79e26b8)) * **bigquery:** add internal listProjects API to core client ([#13429](#13429)) ([3580407](3580407)) * **bigtable:** route point read rows to shim ([#13542](#13542)) ([2c7e5f5](2c7e5f5)) * **deps:** add jspecify to shared dependencies ([#13412](#13412)) ([582053d](582053d)) * enable self-signed JWTs by default in ServiceOptions ([#13338](#13338)) ([110d2b7](110d2b7)) * **gapic-generator:** Add NullMarked annotation to generated classes ([#13517](#13517)) ([825dadd](825dadd)) * **google/cloud/agentregistry/v1:** onboard a new library ([#13509](#13509)) ([6728816](6728816)) * scale up connection worker pool based on latency ([#13384](#13384)) ([eb379b3](eb379b3)) * **spanner:** add settings for gRCP keep-alive ([#13643](#13643)) ([9d78307](9d78307)) * **spanner:** auth login support for Spanner Omni endpoints ([#13470](#13470)) ([55930b4](55930b4)) * **storage:** add checksum validation in the json read channel ([#13270](#13270)) ([872d7b7](872d7b7)) * **storage:** add checksum validation on json read paths ([#13269](#13269)) ([396b042](396b042)) * **storage:** add full object checksum validation for bidi flow ([#13266](#13266)) ([9113d80](9113d80)) * **storage:** add full object checksum validation for grpc flow ([#13265](#13265)) ([a5ba606](a5ba606)) * **storage:** Adding CumulativeHasher wrapper class for Full object … ([#13239](#13239)) ([bd40324](bd40324)) * **storage:** adding disabling option for bidi reads ([#13506](#13506)) ([591cae0](591cae0)) * **storage:** log additional bytes received from GCS in read path ([#13427](#13427)) ([9492aa2](9492aa2)) * **storage:** update compose sample to support deleteSourceObjects option ([#13493](#13493)) ([50a8658](50a8658)) * use self-signed JWTs in Spanner MutableCredentials ([#13381](#13381)) ([29543a2](29543a2)) ### Bug Fixes * Add logging-logback to gapic-libraries-bom ([#13663](#13663)) ([512e370](512e370)) * **auth:** Fix UserCredentials serialization clientSecret leak ([#13465](#13465)) ([2d9d01c](2d9d01c)) * **auth:** handle missing APPDATA on Windows gracefully ([#13471](#13471)) ([77e84a9](77e84a9)), closes [#12565](#12565) * **bigquery-jdbc:** add proper version to BigQueryDriver ([#13294](#13294)) ([9fd84fc](9fd84fc)) * **bigquery-jdbc:** avoid rollback on statement close in manual commit mode ([#13503](#13503)) ([79bbee1](79bbee1)) * **bigquery-jdbc:** correct `FilterTablesOnDefaultDataset` fallback logic ([#13625](#13625)) ([f7eb23d](f7eb23d)) * **bigquery-jdbc:** ensure largeResults are handled in PreparedStatement ([#13496](#13496)) ([d3e7a19](d3e7a19)) * **bigquery-jdbc:** ensure test uses unique dataset & cleans up ([#13587](#13587)) ([a6bb362](a6bb362)) * **bigquery-jdbc:** explicitly set location when available for temporary dataset ([#13508](#13508)) ([40cd0a7](40cd0a7)) * **bigquery-jdbc:** propagate connection proxy settings to auth library ([#13539](#13539)) ([87dda5d](87dda5d)) * **bigquery-jdbc:** shade org.slf4j ([#13547](#13547)) ([7ef1312](7ef1312)) * **bigquery-jdbc:** update format validation tests to create tables for the test ([#13588](#13588)) ([4bb3a61](4bb3a61)) * **bigquery-jdbc:** update Maven Central artifact link in the README ([#13563](#13563)) ([8707a65](8707a65)) * **bigquery:** retry cancel job and routine creation on transient HTTP 5xx errors ([#13416](#13416)) ([35eb041](35eb041)) * **bigquery:** route JOB_CREATION_REQUIRED through fast query path ([#13437](#13437)) ([64cde7f](64cde7f)) * **bigquery:** Update fast query path to allow jobTimeoutMs in the request ([#13502](#13502)) ([1a6f4d5](1a6f4d5)) * **bigtable:** honour session_load override when server-returned session_load is 0 ([#13629](#13629)) ([b56f9b8](b56f9b8)) * **bqjdbc:** pass rowsInPage with TableResult ([#13238](#13238)) ([203a91e](203a91e)) * **ci:** build dependencies before convergence check ([#13638](#13638)) ([9de3209](9de3209)) * **ci:** ignore SNAPSHOT suffix in flatten-plugin check ([#13639](#13639)) ([bb65627](bb65627)) * **ci:** run root install in dependency convergence check ([#13666](#13666)) ([86d35e4](86d35e4)) * **ci:** skip existing-versions-check for gapic-generator-java-root ([#13590](#13590)) ([a1f3b8a](a1f3b8a)) * **datastore:** Configure TraceExporter with Datastore credentials in E2E tests ([#13627](#13627)) ([d281a62](d281a62)) * **datastore:** disable built-in OpenTelemetry SDK when metrics export is disabled ([#13543](#13543)) ([53b7142](53b7142)) * **datastore:** reduce flakiness of E2E tracing tests ([#13645](#13645)) ([8afc0ab](8afc0ab)) * **deps:** update dependency com.google.apis:google-api-services-bigquery to v2-rev20260612-2.0.0 ([#13570](#13570)) ([ca5ff79](ca5ff79)) * **deps:** update dependency com.google.apis:google-api-services-dns to v1-rev20260616-2.0.0 ([#12796](#12796)) ([0dd60ca](0dd60ca)) * **deps:** update dependency com.google.cloud:google-cloud-pubsub-bom to v1.151.0 ([#12097](#12097)) ([d169006](d169006)) * **deps:** update dependency com.google.cloud:libraries-bom to v26.84.0 ([#12663](#12663)) ([95c7774](95c7774)) * **deps:** update first-party storage dependencies to v1-rev20260524-2.0.0 ([#13406](#13406)) ([bdeedb8](bdeedb8)) * exclude java-vertexai from existing versions check ([#13632](#13632)) ([edaa4e3](edaa4e3)) * fallback on VPC ([#13567](#13567)) ([e8c428d](e8c428d)) * **java-cloud-bom:** execute pre-install pass in GHA release notes generator ([#13608](#13608)) ([c365c10](c365c10)) * **release:** check remote tags using git ls-remote to prevent already exists error ([#13574](#13574)) ([6cf0567](6cf0567)) * **spanner:** fail fast when inline-begin DML returns no transaction id ([#13536](#13536)) ([94315ce](94315ce)) * **spanner:** pin inline read-write transactions to default endpoint ([#13562](#13562)) ([6908dee](6908dee)) * **spanner:** remove explicit tink version to resolve dependency convergence conflict ([#13602](#13602)) ([863c26e](863c26e)) * **spanner:** update dynamic channel pool default configuration ([#13646](#13646)) ([37b77c3](37b77c3)) ### Dependencies * Add org.bouncycastle:bcprov-jdk18on to java-shared-dependencies ([#13531](#13531)) ([e3d2082](e3d2082)) * **auth:** Update commons-codec to v1.18.0 ([#13598](#13598)) ([fd89957](fd89957)) * Update gapic-showcase to v0.41.0 ([#13582](#13582)) ([09faaac](09faaac)) * Upgrade google-http-client to v2.1.1 ([#13578](#13578)) ([6abef19](6abef19)) ### Documentation * add instructions for manual code generation checks in CI ([#13596](#13596)) ([f6162e2](f6162e2)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). --------- Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adding CumulativeHasher wrapper class for Full object checksum
Refer to: go/full_checksum_java