Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@bbernays
Copy link
Collaborator

@bbernays bbernays commented Jul 11, 2023

Summary

Currently we are only validating that the top level table has no empty columns... This change ensures we validate all relations as well

@cq-bot cq-bot added the aws label Jul 11, 2023
@bbernays bbernays changed the title fix(aws): Validate relations, not just top level table fix(aws): Validate table relations not just top level table Jul 11, 2023
records := messages.GetInserts().GetRecordsForTable(table)
emptyColumns := schema.FindEmptyColumns(table, records)
if len(emptyColumns) > 0 {
t.Fatalf("empty columns: %v", emptyColumns)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe let's log the table name here too

Copy link
Member

@hermanschaaf hermanschaaf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch!

@bbernays bbernays added the automerge Automatically merge once required checks pass label Jul 11, 2023
@kodiakhq kodiakhq bot merged commit e13d931 into cloudquery:main Jul 11, 2023
kodiakhq bot pushed a commit that referenced this pull request Jul 12, 2023
🤖 I have created a release *beep* *boop*
---


## [20.0.0](plugins-source-aws-v19.2.0...plugins-source-aws-v20.0.0) (2023-07-12)


### ⚠ BREAKING CHANGES

* **aws:** Define primary key for eips ([#11728](#11728))
* Upgrades the awspricing source plugin to use plugin-sdk v4. This version does not contain any user-facing breaking changes, but because it is now using CloudQuery gRPC protocol v3, it does require use of a destination plugin that also supports protocol v3. All recent destination plugin versions support this.

### This Release has the Following Changes to Tables
- Table `aws_ec2_eips`: primary key constraint added to column `account_id` (:warning: breaking)
- Table `aws_ec2_eips`: primary key constraint added to column `allocation_id` (:warning: breaking)
- Table `aws_ec2_eips`: primary key constraint added to column `region` (:warning: breaking)
- Table `aws_ec2_eips`: primary key constraint removed from column `_cq_id` (:warning: breaking)
- Table `aws_networkmanager_global_networks` was added
- Table `aws_networkmanager_links` was added
- Table `aws_networkmanager_sites` was added
- Table `aws_networkmanager_transit_gateway_registrations` was added

### Features

* Add table_options support for aws_securityhub_findings table ([#11955](#11955)) ([c9eff12](c9eff12))
* **aws-policies:** Add in AWS security account contact query ([#11729](#11729)) ([c9d7294](c9d7294))
* **aws-policies:** Add sns logging of delivery status to AWS Policies ([#12074](#12074)) ([80f0b88](80f0b88))
* **aws-policies:** Update sqs encryption for aws foundational security policies ([#11777](#11777)) ([30d415c](30d415c))
* **aws-policies:** Update ssm queries for aws policies ([#12067](#12067)) ([2b9180f](2b9180f))
* **aws-services:** Support newly added regions ([#11922](#11922)) ([6680d7a](6680d7a))
* **aws-services:** Support newly added regions ([#12120](#12120)) ([15ea38c](15ea38c))
* **aws:** Add Support for `ecs:ListTasks` in `table_options` ([#11986](#11986)) ([3016c16](3016c16)), closes [#11981](#11981)
* **aws:** Define primary key for eips ([#11728](#11728)) ([fa48d4a](fa48d4a))
* **aws:** Support networkmanager resources ([#12123](#12123)) ([a642ce0](a642ce0))
* Upgrades the awspricing source plugin to use plugin-sdk v4. This version does not contain any user-facing breaking changes, but because it is now using CloudQuery gRPC protocol v3, it does require use of a destination plugin that also supports protocol v3. All recent destination plugin versions support this. ([7d50d29](7d50d29))


### Bug Fixes

* **aws:** Skip fetching tags for `aws_kafka_cluster_operations` ([#11973](#11973)) ([2b62ba4](2b62ba4))
* **aws:** Validate table relations not just top level table ([#12121](#12121)) ([e13d931](e13d931))
* **deps:** Update github.com/apache/arrow/go/v13 digest to 5a06b2e ([#11857](#11857)) ([43c2f5f](43c2f5f))
* **deps:** Update github.com/cloudquery/arrow/go/v13 digest to 0a52533 ([#12091](#12091)) ([927cefa](927cefa))
* **deps:** Update github.com/cloudquery/arrow/go/v13 digest to a2a76eb ([#12104](#12104)) ([311f474](311f474))
* **deps:** Update github.com/cloudquery/arrow/go/v13 digest to df3b664 ([#11882](#11882)) ([9635b22](9635b22))
* **deps:** Update github.com/cockroachdb/cockroachdb-parser digest to c9c144e ([#11863](#11863)) ([1547efd](1547efd))
* **deps:** Update github.com/cockroachdb/logtags digest to 21c5414 ([#11864](#11864)) ([da48b1f](da48b1f))
* **deps:** Update github.com/gocarina/gocsv digest to 99d496c ([#11865](#11865)) ([c3de686](c3de686))
* **deps:** Update github.com/golang/geo digest to 6adc566 ([#11866](#11866)) ([edb7ed8](edb7ed8))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/networkfirewall to v1.28.3 ([#12079](#12079)) ([a27fa21](a27fa21))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/securityhub to v1.33.2 ([#12081](#12081)) ([e77f93e](e77f93e))
* **deps:** Update module github.com/aws/aws-sdk-go-v2/service/servicediscovery to v1.21.7 ([#12082](#12082)) ([01f8b59](01f8b59))
* **deps:** Update module github.com/cloudquery/plugin-pb-go to v1.5.0 ([#11850](#11850)) ([3255857](3255857))
* **deps:** Update module github.com/cloudquery/plugin-pb-go to v1.6.0 ([#11916](#11916)) ([421e752](421e752))
* **postgresql:** Rerun release please ([#12002](#12002)) ([9d12843](9d12843))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automerge Automatically merge once required checks pass

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants