-
Notifications
You must be signed in to change notification settings - Fork 542
feat(aws)!: Define primary key for eips #11728
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This PR has the following changes to source plugin(s) tables:
|
erezrokah
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR @kd7lxl 🙏 I think you should be able to use the transformers.WithPrimaryKeys option instead of redefining the column. See example
| Transform: transformers.TransformWithStruct(&types.Image{}, transformers.WithPrimaryKeys("Arn")), |
What do you think?
Looks like that worked great. Thanks for the hint. |
Indeed, that's exactly how I arrived at this solution. I tried using deterministic cq id but it had no effect on this table. Then I realized it was because this table was missing a primary key. |
Co-authored-by: Herman Schaaf <[email protected]>
🤖 I have created a release *beep* *boop* --- ## [20.0.0](plugins-source-aws-v19.2.0...plugins-source-aws-v20.0.0) (2023-07-12) ### ⚠ BREAKING CHANGES * **aws:** Define primary key for eips ([#11728](#11728)) * Upgrades the awspricing source plugin to use plugin-sdk v4. This version does not contain any user-facing breaking changes, but because it is now using CloudQuery gRPC protocol v3, it does require use of a destination plugin that also supports protocol v3. All recent destination plugin versions support this. ### This Release has the Following Changes to Tables - Table `aws_ec2_eips`: primary key constraint added to column `account_id` (:warning: breaking) - Table `aws_ec2_eips`: primary key constraint added to column `allocation_id` (:warning: breaking) - Table `aws_ec2_eips`: primary key constraint added to column `region` (:warning: breaking) - Table `aws_ec2_eips`: primary key constraint removed from column `_cq_id` (:warning: breaking) - Table `aws_networkmanager_global_networks` was added - Table `aws_networkmanager_links` was added - Table `aws_networkmanager_sites` was added - Table `aws_networkmanager_transit_gateway_registrations` was added ### Features * Add table_options support for aws_securityhub_findings table ([#11955](#11955)) ([c9eff12](c9eff12)) * **aws-policies:** Add in AWS security account contact query ([#11729](#11729)) ([c9d7294](c9d7294)) * **aws-policies:** Add sns logging of delivery status to AWS Policies ([#12074](#12074)) ([80f0b88](80f0b88)) * **aws-policies:** Update sqs encryption for aws foundational security policies ([#11777](#11777)) ([30d415c](30d415c)) * **aws-policies:** Update ssm queries for aws policies ([#12067](#12067)) ([2b9180f](2b9180f)) * **aws-services:** Support newly added regions ([#11922](#11922)) ([6680d7a](6680d7a)) * **aws-services:** Support newly added regions ([#12120](#12120)) ([15ea38c](15ea38c)) * **aws:** Add Support for `ecs:ListTasks` in `table_options` ([#11986](#11986)) ([3016c16](3016c16)), closes [#11981](#11981) * **aws:** Define primary key for eips ([#11728](#11728)) ([fa48d4a](fa48d4a)) * **aws:** Support networkmanager resources ([#12123](#12123)) ([a642ce0](a642ce0)) * Upgrades the awspricing source plugin to use plugin-sdk v4. This version does not contain any user-facing breaking changes, but because it is now using CloudQuery gRPC protocol v3, it does require use of a destination plugin that also supports protocol v3. All recent destination plugin versions support this. ([7d50d29](7d50d29)) ### Bug Fixes * **aws:** Skip fetching tags for `aws_kafka_cluster_operations` ([#11973](#11973)) ([2b62ba4](2b62ba4)) * **aws:** Validate table relations not just top level table ([#12121](#12121)) ([e13d931](e13d931)) * **deps:** Update github.com/apache/arrow/go/v13 digest to 5a06b2e ([#11857](#11857)) ([43c2f5f](43c2f5f)) * **deps:** Update github.com/cloudquery/arrow/go/v13 digest to 0a52533 ([#12091](#12091)) ([927cefa](927cefa)) * **deps:** Update github.com/cloudquery/arrow/go/v13 digest to a2a76eb ([#12104](#12104)) ([311f474](311f474)) * **deps:** Update github.com/cloudquery/arrow/go/v13 digest to df3b664 ([#11882](#11882)) ([9635b22](9635b22)) * **deps:** Update github.com/cockroachdb/cockroachdb-parser digest to c9c144e ([#11863](#11863)) ([1547efd](1547efd)) * **deps:** Update github.com/cockroachdb/logtags digest to 21c5414 ([#11864](#11864)) ([da48b1f](da48b1f)) * **deps:** Update github.com/gocarina/gocsv digest to 99d496c ([#11865](#11865)) ([c3de686](c3de686)) * **deps:** Update github.com/golang/geo digest to 6adc566 ([#11866](#11866)) ([edb7ed8](edb7ed8)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/networkfirewall to v1.28.3 ([#12079](#12079)) ([a27fa21](a27fa21)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/securityhub to v1.33.2 ([#12081](#12081)) ([e77f93e](e77f93e)) * **deps:** Update module github.com/aws/aws-sdk-go-v2/service/servicediscovery to v1.21.7 ([#12082](#12082)) ([01f8b59](01f8b59)) * **deps:** Update module github.com/cloudquery/plugin-pb-go to v1.5.0 ([#11850](#11850)) ([3255857](3255857)) * **deps:** Update module github.com/cloudquery/plugin-pb-go to v1.6.0 ([#11916](#11916)) ([421e752](421e752)) * **postgresql:** Rerun release please ([#12002](#12002)) ([9d12843](9d12843)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
By defining a primary key, the deterministic cq id feature can be used so that multiple syncs are able to associate the same resource.