feat(parameters): Configure max_age and decrypt parameters via environment variables #2088

leandrodamascena · 2023-04-05T22:06:54Z

Issue number: #2041

Summary

Changes

In this PR we changed the utility parameter code to accept environment variables to configure the default TTL and the decrypt parameter.

User experience

Before this PR the customer had to inform the max_age parameter every time they wanted a standard TTL different than 5 seconds (default value). If the user had retrieved dozens/hundreds of parameters, this made the developer's experience bad. This PR creates an environment variable called POWERTOOLS_PARAMETERS_MAX_AGE that accepts an integer value to be used as the default TTL.

This PR also introduces a new environment variable called POWERTOOLS_PARAMETERS_SSM_DECRYPT. This environment variable can be used to set a default value for the decrypt parameter in the SSM Provider. This is extremely useful in cases where the customer needs to recover dozens/hundreds of SecureString type parameters and you need to decrypt them. If a value for this environment variable is not informed, the default value False remains.

This PR aims to make the developer experience simpler. Below is a code snippet of how a Lambda template can be configured to use these environment variables.

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: HelloWorldFunction
      Handler: app.lambda_handler
      Runtime: python3.9
      Environment:
        Variables:
          POWERTOOLS_PARAMETERS_MAX_AGE: 10
          POWERTOOLS_PARAMETERS_SSM_DECRYPT: true
      Architectures:
      - x86_64
      Events:
        HelloWorld:
          Type: Api
          Properties:
            Path: /hello
            Method: get

Checklist

If your change doesn't seem to apply, please leave them unchecked.

Meet tenets criteria
I have performed a self-review of this change
Changes have been tested
Changes are documented
PR title follows conventional commit semantics

Is this a breaking change?

RFC issue number:

Checklist:

Migration process documented
Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

aws_lambda_powertools/utilities/parameters/base.py

heitorlessa · 2023-04-06T08:51:19Z

tests/functional/test_utilities_parameters.py

@@ -547,6 +547,44 @@ def test_ssm_provider_get_with_custom_client(mock_name, mock_value, mock_version
        stubber.deactivate()


+def test_ssm_provider_get_with_decrypt_environment_variable(monkeypatch, mock_name, mock_value, mock_version, config):


missing test for max_age env var?

Hei @heitorlessa, yes! It was missing and refactored the code adding a specific function to resolve the max_age and testing it. I tried to use the function resolve_env_var_choice but looks like that mypy doesn't like overload with very similar types like float and int.

leandrodamascena · 2023-04-11T16:22:03Z

@rubenfonseca can you review this PR, please?!

rubenfonseca

Great job! It's a little bit tedious because the same logic is present in multiple places. I left some suggestions for improvement.

aws_lambda_powertools/shared/constants.py

aws_lambda_powertools/utilities/parameters/appconfig.py

aws_lambda_powertools/utilities/parameters/base.py

aws_lambda_powertools/utilities/parameters/ssm.py

docs/index.md

docs/utilities/parameters.md

leandrodamascena · 2023-04-13T09:40:51Z

Great job! It's a little bit tedious because the same logic is present in multiple places. I left some suggestions for improvement.

Thank you for this super review, Ruben! I really liked it!
I have the same feeling: it's a bit distracting to use the same logic, but it seems like there's no other option (other than refactoring everything) to check if the customer is using the max_age/decrypt parameters when getting a specific parameter from a specific provider.

I addressed all feedback and let me know if I need to change something else.

codecov-commenter · 2023-04-13T09:57:45Z

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (3e313bb) 97.47% compared to head (639c24f) 97.48%.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #2088   +/-   ##
========================================
  Coverage    97.47%   97.48%           
========================================
  Files          147      147           
  Lines         6809     6835   +26     
  Branches       483      483           
========================================
+ Hits          6637     6663   +26     
  Misses         136      136           
  Partials        36       36

Impacted Files	Coverage Δ
aws_lambda_powertools/shared/constants.py	`100.00% <100.00%> (ø)`
aws_lambda_powertools/shared/functions.py	`96.55% <100.00%> (+0.12%)`	⬆️
...ambda_powertools/utilities/parameters/appconfig.py	`94.59% <100.00%> (+0.15%)`	⬆️
aws_lambda_powertools/utilities/parameters/base.py	`99.11% <100.00%> (+0.04%)`	⬆️
..._lambda_powertools/utilities/parameters/secrets.py	`100.00% <100.00%> (ø)`
aws_lambda_powertools/utilities/parameters/ssm.py	`97.59% <100.00%> (+0.18%)`	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

rubenfonseca

Just a small thing...

rubenfonseca · 2023-04-13T14:26:11Z

docs/index.md

+| **POWERTOOLS_PARAMETERS_MAX_AGE**         | Adjust how long values are kept in cache (in seconds)                                  | [Parameters](./utilities/parameters/#adjusting-cache-ttl)                           | `5`                   |
+| **POWERTOOLS_PARAMETERS_SSM_DECRYPT**     | Sets whether to decrypt or not values retrieved from AWS SSM Parameters Store          | [Parameters](./utilities/parameters/#ssmprovider)                                   | `false`               |


I think the environment variable names need to be updated too

No no Ruben, we don't need to. In the documentation is the name of the environment variable that will be configured in Lambda, what we changed was the constant in the constant.py file.
We put the constants to make easy the replication of the same value throughout the project's code.

I need more coffee

rubenfonseca

LGTM!

leandrodamascena added 2 commits April 5, 2023 22:23

feat(parameters) - environment default variables

149c5e5

feat(parameters) - environment default variables - maxage

b4c2d48

leandrodamascena requested a review from a team as a code owner April 5, 2023 22:06

leandrodamascena requested review from heitorlessa and removed request for a team April 5, 2023 22:06

boring-cyborg bot added commons documentation Improvements or additions to documentation tests labels Apr 5, 2023

pull-request-size bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Apr 5, 2023

github-actions bot added the feature New feature or functionality label Apr 5, 2023

leandrodamascena commented Apr 6, 2023

View reviewed changes

aws_lambda_powertools/utilities/parameters/base.py Outdated Show resolved Hide resolved

heitorlessa linked an issue Apr 6, 2023 that may be closed by this pull request

Feature request: ability to set maxAge and decrypt via environment variables for Parameters #2041

Closed

2 tasks

heitorlessa reviewed Apr 6, 2023

View reviewed changes

feat(parameters) - environment default variables - refactor maxage code

77c41a2

Merge branch 'develop' into feat/parameter-env-variable

02d3254

leandrodamascena requested a review from rubenfonseca April 11, 2023 16:31

leandrodamascena self-assigned this Apr 11, 2023

Merge branch 'develop' into feat/parameter-env-variable

2a6c90d

rubenfonseca suggested changes Apr 13, 2023

View reviewed changes

feat(parameters) - addressing Ruben's feedback

74f65f9

Merge branch 'develop' into feat/parameter-env-variable

639c24f

rubenfonseca suggested changes Apr 13, 2023

View reviewed changes

rubenfonseca self-requested a review April 14, 2023 07:29

rubenfonseca approved these changes Apr 14, 2023

View reviewed changes

leandrodamascena merged commit ed73aef into aws-powertools:develop Apr 14, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(parameters): Configure max_age and decrypt parameters via environment variables #2088

feat(parameters): Configure max_age and decrypt parameters via environment variables #2088

leandrodamascena commented Apr 5, 2023

heitorlessa Apr 6, 2023 •

edited

Loading

leandrodamascena Apr 11, 2023

leandrodamascena commented Apr 11, 2023

rubenfonseca left a comment

leandrodamascena commented Apr 13, 2023

codecov-commenter commented Apr 13, 2023

rubenfonseca left a comment

rubenfonseca Apr 13, 2023

leandrodamascena Apr 13, 2023

rubenfonseca Apr 14, 2023

rubenfonseca left a comment

		@@ -547,6 +547,44 @@ def test_ssm_provider_get_with_custom_client(mock_name, mock_value, mock_version
		stubber.deactivate()


		def test_ssm_provider_get_with_decrypt_environment_variable(monkeypatch, mock_name, mock_value, mock_version, config):

		\| POWERTOOLS_PARAMETERS_MAX_AGE \| Adjust how long values are kept in cache (in seconds) \| [Parameters](./utilities/parameters/#adjusting-cache-ttl) \| `5` \|
		\| POWERTOOLS_PARAMETERS_SSM_DECRYPT \| Sets whether to decrypt or not values retrieved from AWS SSM Parameters Store \| [Parameters](./utilities/parameters/#ssmprovider) \| `false` \|

feat(parameters): Configure max_age and decrypt parameters via environment variables #2088

feat(parameters): Configure max_age and decrypt parameters via environment variables #2088

Conversation

leandrodamascena commented Apr 5, 2023

Summary

Changes

User experience

Checklist

Acknowledgment

heitorlessa Apr 6, 2023 • edited Loading

Choose a reason for hiding this comment

leandrodamascena Apr 11, 2023

Choose a reason for hiding this comment

leandrodamascena commented Apr 11, 2023

rubenfonseca left a comment

Choose a reason for hiding this comment

leandrodamascena commented Apr 13, 2023

codecov-commenter commented Apr 13, 2023

Codecov Report

rubenfonseca left a comment

Choose a reason for hiding this comment

rubenfonseca Apr 13, 2023

Choose a reason for hiding this comment

leandrodamascena Apr 13, 2023

Choose a reason for hiding this comment

rubenfonseca Apr 14, 2023

Choose a reason for hiding this comment

rubenfonseca left a comment

Choose a reason for hiding this comment

heitorlessa Apr 6, 2023 •

edited

Loading