Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add util function to enable transparent local boto3 endpoints #40

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 31, 2022

Conversation

whummer
Copy link
Member

@whummer whummer commented Aug 31, 2022

  • Add enable_local_endpoints utility function to enable transparent local boto3 endpoints
  • The reverse operation disable_local_endpoints can be used to undo the patches
  • Slight project refactoring, migrate tests to pytest, add code linter/formatter
  • Bump version to 1.38 (released version 1.37 separately to get the AWS Transcribe endpoint change out)

TODO:

  • update changelog, and perhaps add a short note to the README

@whummer whummer force-pushed the transparent-local-endpoints branch from 4547b11 to 6422d40 Compare August 31, 2022 11:01
@whummer whummer force-pushed the transparent-local-endpoints branch from 90b0bd0 to 58df392 Compare August 31, 2022 11:21
Copy link
Member

@dominikschubert dominikschubert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Hope they'll add a better way to configure endpoints in the future but for now this should provide an awesome additional method to semi-transparently target localstack when needed 👍

"mwaa": "{proto}://{host}:4642",
"fis": "{proto}://{host}:4643",
"meteringmarketplace": "{proto}://{host}:4644",
"transcribe": "{proto}://{host}:4566",
}

# TODO remove service port mapping above entirely
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we should be able to remove these already right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! Great catch - planning to do that in a follow-up version (should not have a functional impact at this stage, but just to separate changes and versions a bit). 👍

Comment on lines +6 to +13
@pytest.fixture(scope="session", autouse=True)
def startup_localstack():
subprocess.check_output(["localstack", "start", "-d"])
subprocess.check_output(["localstack", "wait"])

yield

subprocess.check_output(["localstack", "stop"])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any particular reason for using this instead of the localstack python module? 🤔

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really, good point - basically just migrated that from the old nosetest setup. Will keep a note for a follow-up iteration 👌


# create default client, requests should fail
with pytest.raises(Exception):
boto3.client("s3").list_buckets()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't fail if the AWS credentials are set in the environment though.
Could make this more robust by manually specifying access key id & secret

Comment on lines +142 to +148
# skip adding host prefixes, to avoid making requests to, e.g., http://data-localhost:4566
is_sd = operation_model.service_model.service_name == "servicediscovery"
if is_sd and result == "data-":
return None
if operation_model.service_model.service_name == "mwaa" and result == "api.":
return None
return result
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stepfunctions has a similar prefix AFAIK (sync-)

Copy link
Member Author

@whummer whummer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the comments @dominikschubert , great points - will pick those up in a follow-up iteration, specifically when we get rid of the obsolete service ports 🧹

@whummer whummer merged commit 6972c42 into master Aug 31, 2022
@whummer whummer deleted the transparent-local-endpoints branch August 31, 2022 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants