This PySpark utility compares JSON-based S3 source tables with Delta Lake tables to validate data integrity.
- Compares row counts
- Aligns records by primary key(s)
- Identifies mismatches at column level
- Outputs matched, mismatched, or missing tables
-
Notifications
You must be signed in to change notification settings - Fork 1
ramprasadowk/spark-s3-delta-comparator
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
Compare data between s3 to Datarbicks Delta table
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published