-
Notifications
You must be signed in to change notification settings - Fork 6.5k
Storage: HMAC key samples #2372
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding a few nits. Thanks @tritone, looks good!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
storage/cloud-client/hmac.py
Outdated
# project_id = 'Your Google Cloud project ID' | ||
# access_id = 'ID of an HMAC key' | ||
storage_client = storage.Client(project=project_id) | ||
hmac_key = storage_client.get_hmac_key_metadata(access_id, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Prefer not using hanging indents - see #7 under Style&Linting.
hmac_key = storage_client.get_hmac_key_metadata(access_id, | |
hmac_key = storage_client.get_hmac_key_metadata( | |
access_id, project_id=project_id) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed. Have we considered using yapf or another opinionated formatter that will make these decisions automatically?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Java has something like this (google-java-formatter), but I'm not aware of plans for Python. I'm okay with it as long as it's compatible with the google python style guides.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is a google project so I would assume it should be? :) I can run it by others on the team who are opinionated about python stuff.
service_account_email=service_account_email, | ||
project_id=project_id) | ||
print('The base64 encoded secret is {}'.format(secret)) | ||
print('Do not miss that secret, there is no API to recover it.') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
print('Do not miss that secret, there is no API to recover it.') | |
print('Do not lose this secret, as it cannot be recovered.') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This language was taken from the canonical sample, see https://github.com/googleapis/nodejs-storage/blob/master/samples/hmacKeyCreate.js#L48 . I would think that consistency is most important, unless we want to change it in all of them?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll leave the final decision up to @frankyn, but my concerns are that:
- It's currently improper english, which might make it harder for non-native english speakers to understand
- Saying "there is no API to recover it" seems to imply that there may be some other way to recover it, like a support ticket or something (which I assume it not the case)
storage/cloud-client/hmac_test.py
Outdated
Fixture to create a new HMAC key, and to guarantee all keys are deleted at | ||
the end of each test. | ||
""" | ||
hmac_key = hmac.create_key(PROJECT_ID, SERVICE_ACCOUNT_EMAIL) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This fixture is inconsistently handling fixtures - sometimes it uses the snippet to perform the action and sometimes not. We should be consistent.
I think it makes more sense to avoid using the snippet, so that we don't have to worry about the logs from another snippet inadvertently showing up in the test or logs somewhere.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought about this a bit-- the downside of not using the snippet is that then it's difficult to test the create_key snippet (because then I can't leverage the teardown in the fixture for the newly created key, so if assertions in the test fail the key will never be deleted). Any thoughts on how to get around this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you have a couple of options:
- For testing the create_key snippet, you could call create, then delete, then assert the output of create.
- You could reset
capsys
at the start of the function by callingreadouterr()
- If you switch
capsys
to the last parameter, it'll create the other fixtures before setting up the stdout capture (so you can use snippets without effecting the results of the test)
I think the first is probably preferable, as it leaves cleaner logs and having an explicit test for create_key
is a better indicator the problem is in the creation, rather than an error in a fixture bubbling up in a different test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good points, thanks for the advice. I'll take your first suggestion.
storage/cloud-client/hmac_test.py
Outdated
""" | ||
hmac_key = hmac.create_key(PROJECT_ID, SERVICE_ACCOUNT_EMAIL) | ||
yield hmac_key | ||
storage_client = storage.Client(project=PROJECT_ID) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We probably want to make this it's own fixture and scope it to module
to avoiding initializing a new one every time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are you referring specifically to storage_client? I could do this, but I guess I wasn't thinking of it as a resource that is expensive to initialize.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other tests such as https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/storage/cloud-client/acl_test.py and https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/storage/cloud-client/bucket_policy_only_test.py seem to use a similar approach with including storage.client in one function-level fixture.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah - some of our clients launch threads and tie up other resources (which is why I asked about close()
elsewhere). This client doesn't look too heavy, but there are a couple of network calls for oath and an http session being created, so it's probably worthwhile to reuse.
I'm actually not sure if you can nest fixtures, but if it doesn't work and since you don't have to close it, you can probably just move it to a module level variable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can nest fixtures, but I decided to go for the module-level variable because it seems pretty safe and it's less confusing to read.
storage/cloud-client/hmac_test.py
Outdated
Fixture to create a new HMAC key, and to guarantee all keys are deleted at | ||
the end of each test. | ||
""" | ||
hmac_key = hmac.create_key(PROJECT_ID, SERVICE_ACCOUNT_EMAIL) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you have a couple of options:
- For testing the create_key snippet, you could call create, then delete, then assert the output of create.
- You could reset
capsys
at the start of the function by callingreadouterr()
- If you switch
capsys
to the last parameter, it'll create the other fixtures before setting up the stdout capture (so you can use snippets without effecting the results of the test)
I think the first is probably preferable, as it leaves cleaner logs and having an explicit test for create_key
is a better indicator the problem is in the creation, rather than an error in a fixture bubbling up in a different test.
storage/cloud-client/hmac_test.py
Outdated
""" | ||
hmac_key = hmac.create_key(PROJECT_ID, SERVICE_ACCOUNT_EMAIL) | ||
yield hmac_key | ||
storage_client = storage.Client(project=PROJECT_ID) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah - some of our clients launch threads and tie up other resources (which is why I asked about close()
elsewhere). This client doesn't look too heavy, but there are a couple of network calls for oath and an http session being created, so it's probably worthwhile to reuse.
I'm actually not sure if you can nest fixtures, but if it doesn't work and since you don't have to close it, you can probably just move it to a module level variable.
No description provided.