IaC: Add AWS operations and CFN resources catalog #13027
Conversation
S3 Image Test Results (AMD64 / ARM64) 2 files 2 suites 8m 18s ⏱️ Results for commit 961dbee. ♻️ This comment has been updated with latest results. |
There was a problem hiding this comment.
This is nice 👌 super nice to see how easy it is to plug into this! this is only a pre-review as I got the notification, I'm still unsure how we are going to extend the behavior from Pro. Might be good to try it already to see how we could do it, and adapt the code in consequence? You might be on it already 😄
There was a problem hiding this comment.
I wonder if we should still catch exceptions in the skeleton instead of letting them bubbling up to the proper exception handlers.
This doesn't need to be acted upon in this PR in particular, but it feels a bit weird to have the logic duplicated.
There was a problem hiding this comment.
Will be addressed in this PR #13068
| def map_catalog_availability_to_exception( | ||
| service_name: str, | ||
| operation_name: str | None, | ||
| support_status: AwsServicesSupportInLatest | AwsServiceOperationsSupportInLatest, | ||
| ) -> AwsServiceAvailabilityException: |
There was a problem hiding this comment.
question: how will we extend this with more possible values in the Pro version? we might want to aim for it to be extended transparently, as we won't be able to plug into the Skeleton.on_not_implemented_error function directly to modify which function is being called.
We could probably patch it, and execute some code before deferring the logic to this function? maybe there are better ways. Patching might do, we're also using it for our AWS routing.
Thanks, yes, I will extend this in pro tomorrow :) I also want to see how the whole integration will work before merging this PR |
| for emulator_type in list(LocalstackEmulatorType): | ||
| if emulator_type in service: |
There was a problem hiding this comment.
same here with the emulator type
There was a problem hiding this comment.
very nice addition with the plugins 🚀 thanks for addressing the comments, and making this looking really great and extendable!
I believe the RemoteCatalogLoader is currently broken and will break the functionality all together, but once this is addressed, I think we would be good to go 👍
Only have a few comments, agreeing with Silvio, and added some more related to some Enum being returned. Nice work 👌
b2b69dc to
92e58e9
Compare
|
I removed integration to service and skeleton to merge this PR faster |
Test Results (amd64) - Integration, Bootstrap 5 files 5 suites 2h 33m 43s ⏱️ Results for commit 92e58e9. ♻️ This comment has been updated with latest results. |
| self.cfn_resources_in_latest = AwsCatalogRemoteStatePlugin.get_cfn_resources_catalog( | ||
| remote_catalog.cloudformation_resources | ||
| ) | ||
| self.cfn_resources_at_runtime = set(cfn_plugin_manager.list_names()) | ||
| self.services_at_runtime = AwsCatalogRemoteStatePlugin.get_aws_services_at_runtime() |
There was a problem hiding this comment.
question: would it make more sense to call those 2 methods via the self instance, even if they are static methods?
| "Failed to load catalog plugin with the latest LocalStack services support data, falling back to catalog without remote state: %s", | ||
| e, | ||
| ) | ||
| return plugin_manager.load("aws-catalog-runtime-only") |
There was a problem hiding this comment.
question: should we maybe have a different fallback for the Pro version?
Currently, if I understood well, the Runtime-only fallback doesn't do anything in Community, because it cannot access any licensing data, and won't provide additional features at all. I can see how this works for the 501 error, it will just fallback to what we had before.
For CloudFormation, I'm not sure what the behavior will be if we return None, but I'm sure we can handle it, so for now I don't see a big issue.
However, for the Pro "runtime fallback", we will have additional data even without the remote catalog: for example, if the service is in another tier, etc. So there we should probably share code for the runtime part between the 2 catalogs (remote and runtime-only). Not sure if I'm making sense here 😄
| except Exception as e: | ||
| LOG.debug( | ||
| "Failed to load catalog plugin with the latest LocalStack services support data, falling back to catalog without remote state: %s", | ||
| e, | ||
| ) | ||
| return plugin_manager.load("aws-catalog-runtime-only") |
There was a problem hiding this comment.
something I missed: I believe we should maybe raise a specific exception we would have subclassed ourselves in case we have a failure in the RemoteCatalogLoader, so that we could probably guard only against this? because here, if we have any kind of exceptions when instantiating the Catalog, we would fallback, even if the exception is something we should fix, like a Key error or something?
Motivation
This PR introduces a plugin AWS license catalog which will be used to customize error messages for unimplemented AWS services or operations.
Downloading and loading of the license catalog is not implemented in this PR and will be completed in the next PR.
Changes