feat: Consistency polling + restore table for sync client in admin #1117
Conversation
No region tags are edited in this PR.This comment is generated by snippet-bot.
|
daniel-sanche
left a comment
There was a problem hiding this comment.
Looks like a good direction. What's the overal plan for this though? Were you planning on merging this now and adding tests and polish later, or will they come later in this PR?
| from google.cloud.bigtable.admin_v2.overlay.types import consistency, restore_table | ||
|
|
||
|
|
||
| class BigtableTableAdminClient(BaseBigtableTableAdminClient): |
There was a problem hiding this comment.
Do we also need this for the Async client?
There was a problem hiding this comment.
Yeah after I finish this PR and the second owlbot PR I'll go back and work on the async client as well.
| BaseBigtableTableAdminClient, | ||
| ) | ||
| from google.cloud.bigtable.admin_v2.overlay.types import consistency, restore_table | ||
|
|
There was a problem hiding this comment.
extensive comments will be useful here. We should have an explanation here at the top of the file explaining why we need to override these methods
There was a problem hiding this comment.
Let me write some comments in overlay/__init__.py explaining the purpose of this handwritten layer, then delve into specific details in this file.
| retry: OptionalRetry = gapic_v1.method.DEFAULT, | ||
| timeout: Union[float, object] = gapic_v1.method.DEFAULT, | ||
| metadata: Sequence[Tuple[str, Union[str, bytes]]] = (), | ||
| ) -> consistency.CheckConsistencyPollingFuture: |
There was a problem hiding this comment.
docstring?
Can the word await here be avoided? That could be complicated for asyncio
| @@ -0,0 +1,38 @@ | |||
| # Copyright 2025 Google LLC. | |||
There was a problem hiding this comment.
I'm a bit confused about the gapic folders in the file structure:
google
cloud
bigtable
data
admin
admin_v2
bigtable_v2
I remember some discussions around this before, but it still seems a bit awkward. Has this been locked in?
| default_retry: OptionalRetry = gapic_v1.method.DEFAULT, | ||
| polling: retries.Retry = polling.DEFAULT_POLLING, | ||
| **kwargs | ||
| ): |
There was a problem hiding this comment.
file-level comments and docstrings will be very helpful here
| class DefaultGarbageCollectionRule(BaseGarbageCollectionRule): | ||
| """Default garbage collection rule. | ||
|
|
||
| TODO: Add a sample for this. |
There was a problem hiding this comment.
are you planning on adding this before merging?
|
|
||
| .. literalinclude:: snippets_table.py | ||
| :start-after: [START bigtable_api_create_family_gc_max_versions] | ||
| :end-before: [END bigtable_api_create_family_gc_max_versions] |
There was a problem hiding this comment.
do these already exist?
If it's simple enough, it might be better to just include it directly, so users can read it on github. But if it's complex, it might be good to have it in a tested file like this
There was a problem hiding this comment.
I just looked at the snippet this is referring to, and what it's trying to do is different from what we're trying to do, so I agree with adding a new snippet directly into this.
| from google.cloud.bigtable.admin_v2.types import OptimizeRestoredTableMetadata | ||
|
|
||
|
|
||
| class RestoreTableOperation(operation.Operation): |
There was a problem hiding this comment.
again, file comments and docstrings would be helpful here
I was originally planning on getting this committed, then working on the owlbot stuff needed to integrate I'll definitely add comments and docstrings here, seeing as how a lot of the feedback here deals with these. This would be a good opportunity to polish the sync client stuff and work on async in the PR after the owlbot PR. |
|
|
||
| from .client import BigtableTableAdminClient | ||
|
|
||
| __all__ = ("BigtableTableAdminClient",) |
There was a problem hiding this comment.
what about the async client?
daniel-sanche
left a comment
There was a problem hiding this comment.
LGTM, to merge into the autogen-feature-branch. We might still have to discuss the overall file structure, but that can come as a follow-up
…ration (#1177) * chore: Removed old admin_v2 GAPIC layer (#1111) * feat!: Generated Selective GAPIC layer for Admin API (#1112) * chore: Updated service YAML by making all methods in BigtableInstanceAdmin public (#1113) * refactor: Refactored classic client to use new Admin API (#1114) * refactor: Refactored classic client to use new Admin API * added newline after gapic_version files * fix: Made generate_consistency_token and check_consistency public (#1116) methods * feat: Consistency polling + restore table for sync client in admin (#1117) * feat: Prototyped handwritten layer * Added newlines * linting * Added docstrings for restore table and consistency token polling; removed gc_rule * docs: owlbot related changes (#1133) * docs: owlbot related changes * Addressed PR feedback + made changes to toc.yml for docs pipeline * Fixed type hint * linting + added validation for admin section * linting + added noqas to owlbot lines * tests: Tests for sync client + fixes + client library versioning (#1132) * tests: Tests for sync client + fixes + client library versioning * Removed raise exception * linting + name changes in tests + added test for timeout * linting * Fixed tests on Python 3.7 * feat: Proto-plus modifications for enforcing strict oneofs (#1126) * feat: Proto-plus modifications for enforcing strict oneofs * Added template directory + changed unit tests to pytest * Finished README * linting * Added source of truth comment * feat: Reworked the wait_for_consistency call (#1144) * feat: Reworked the wait_for_consistency call * linting * Update google/cloud/bigtable/admin_v2/overlay/services/bigtable_table_admin/client.py Co-authored-by: Mattie Fu <[email protected]> * Improved documentation * linting again * linting --------- Co-authored-by: Mattie Fu <[email protected]> * feat: Async consistency polling harness (#1142) * feat: Async consistency polling harness * Fixed AsyncMock issue in Python 3.7 * Reworked async_consistency and added async client to __init__.py * linting * addressed review feedback * linting * feat: Restore Table LRO rework + async restore table (#1148) * chore(tests): system tests for autogen API (#1151) * tests: system tests for autogen API * Fixed async system tests * addressed review feedback * Fixed system test failure at the end of a test run * Linting * more linting * chore: Moved Admin API from google.cloud.bigtable.admin_v2 back to google.cloud.bigtable_admin_v2 (#1153) * chore: Removed autogenerated files from the feature branch (#1170) * chore: Merged selective GAPIC autogenerated changes into feature branch (#1175) * chore: Merged selective GAPIC owlbot changes into feature branch * linting * changed comment text * Removed redundant items * Fixed owlbot infinitely appending text * Added comments + fixed indentation in Owlbot * Added anonymous credentials to client tests * Fixed project ID issue in system tests * Fixed docstrings and skipped system tests on emulator. --------- Co-authored-by: Mattie Fu <[email protected]>
This commit is for the commit prototype step (Googlers see go/bigtable-admin-autogen-python-implementation for more details about the prototype). Kokoro might break during this step due to not enough test coverage (the unit tests all pass with the new admin shim), but additional docstrings, unit testing, and the async client, among other things, will be added after post-processor changes are finalized.