fix(postgres): preserve PARTITION BY in diff-changelog (#6885)#7759
fix(postgres): preserve PARTITION BY in diff-changelog (#6885)#7759Khromushkin wants to merge 9 commits into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (16)
📝 WalkthroughWalkthroughThis PR implements end-to-end support for PostgreSQL declarative table partitioning across Liquibase's snapshot, diff, changelog, and SQL generation pipelines. It adds ChangesPostgreSQL Declarative Partitioning Feature
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
liquibase-integration-tests/src/test/java/liquibase/dbtest/pgsql/PostgreSQLIntegrationTest.java (1)
501-505: 💤 Low valueConsider adding a PostgreSQL version check.
Declarative table partitioning (
PARTITION BY) was introduced in PostgreSQL 10. Without a version assumption, this test will fail with a SQL syntax error on older PostgreSQL versions rather than being skipped gracefully:public void testSnapshotPartitionedTableCapturesPartitionBy() throws Exception { assumeNotNull(this.getDatabase()); + assumeTrue(getDatabase().getDatabaseMajorVersion() >= 10); Scope.getCurrentScope().getSingleton(ExecutorService.class).getExecutor("jdbc", getDatabase())This matches the pattern used elsewhere in this file (e.g.,
assumeTrue(getDatabase().getDatabaseMajorVersion() >= 12)for generated column tests).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@liquibase-integration-tests/src/test/java/liquibase/dbtest/pgsql/PostgreSQLIntegrationTest.java` around lines 501 - 505, In testSnapshotPartitionedTableCapturesPartitionBy (class PostgreSQLIntegrationTest) add a PostgreSQL version assumption before executing the CREATE TABLE so the test is skipped on older servers; specifically, after assumeNotNull(this.getDatabase()) call assert the database major version is >= 10 (use assumeTrue(getDatabase().getDatabaseMajorVersion() >= 10)) so the PARTITION BY syntax is only run on PostgreSQL 10+.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@liquibase-standard/src/main/java/liquibase/change/core/CreateTableChange.java`:
- Around line 196-197: The partitionBy value is forwarded raw into the
CreateTableStatement, allowing whitespace-only input to produce an invalid
PARTITION BY clause; normalize it the same way tablespace is handled by trimming
and turning empty/whitespace-only strings to null before attaching to the
statement. Update the return in CreateTableChange (the new
CreateTableStatement(...).setPartitionBy(getPartitionBy()) call) to pass a
trimmed-or-null value (e.g., use the same trim-to-null utility or implement
StringUtils.trimToNull on getPartitionBy()) so only non-empty partition
expressions are set.
In
`@liquibase-standard/src/main/java/liquibase/snapshot/JdbcDatabaseSnapshot.java`:
- Around line 1375-1386: The Postgres partition enrichment currently runs for
any PostgresDatabase and can fail on CockroachDB; update the early guard in
JdbcDatabaseSnapshot (the block that checks "database instanceof
PostgresDatabase" and "rows.isEmpty()") to also exclude CockroachDatabase (e.g.,
ensure the condition requires database NOT be an instance of CockroachDatabase)
so the partition-catalog SQL (pg_partitioned_table / pg_get_partkeydef) is only
executed for real PostgreSQL databases.
In
`@liquibase-standard/src/main/java/liquibase/sqlgenerator/core/CreateTableGenerator.java`:
- Around line 368-370: The code in CreateTableGenerator.java appends " PARTITION
BY " when database is a PostgresDatabase and statement.getPartitionBy() is
non-null, but it doesn't guard against empty or whitespace-only values; update
the check to trim the partition string and treat empty/blank as null before
appending (e.g., retrieve statement.getPartitionBy(), trim it and verify it's
not empty) so that " PARTITION BY " is only added when a non-blank partitionBy
value exists.
---
Nitpick comments:
In
`@liquibase-integration-tests/src/test/java/liquibase/dbtest/pgsql/PostgreSQLIntegrationTest.java`:
- Around line 501-505: In testSnapshotPartitionedTableCapturesPartitionBy (class
PostgreSQLIntegrationTest) add a PostgreSQL version assumption before executing
the CREATE TABLE so the test is skipped on older servers; specifically, after
assumeNotNull(this.getDatabase()) call assert the database major version is >=
10 (use assumeTrue(getDatabase().getDatabaseMajorVersion() >= 10)) so the
PARTITION BY syntax is only run on PostgreSQL 10+.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: a01d2424-f22a-47a5-aa58-6efb6ac051b3
📒 Files selected for processing (17)
liquibase-cli/src/test/groovy/liquibase/integration/commandline/LiquibaseCommandLineTest.groovyliquibase-integration-tests/src/test/java/liquibase/dbtest/pgsql/PostgreSQLIntegrationTest.javaliquibase-standard/src/main/java/liquibase/change/core/CreateTableChange.javaliquibase-standard/src/main/java/liquibase/diff/output/changelog/core/ChangedTableChangeGenerator.javaliquibase-standard/src/main/java/liquibase/diff/output/changelog/core/MissingTableChangeGenerator.javaliquibase-standard/src/main/java/liquibase/snapshot/JdbcDatabaseSnapshot.javaliquibase-standard/src/main/java/liquibase/snapshot/jvm/TableSnapshotGenerator.javaliquibase-standard/src/main/java/liquibase/sqlgenerator/core/CreateTableGenerator.javaliquibase-standard/src/main/java/liquibase/statement/core/CreateTableStatement.javaliquibase-standard/src/main/java/liquibase/structure/core/Table.javaliquibase-standard/src/main/resources/www.liquibase.org/xml/ns/dbchangelog/dbchangelog-5.0.xsdliquibase-standard/src/main/resources/www.liquibase.org/xml/ns/dbchangelog/dbchangelog-latest.xsdliquibase-standard/src/test/groovy/liquibase/change/core/CreateTableChangeTest.groovyliquibase-standard/src/test/groovy/liquibase/diff/output/changelog/core/MissingTableChangeGeneratorTest.groovyliquibase-standard/src/test/groovy/liquibase/snapshot/jvm/TableSnapshotGeneratorTest.groovyliquibase-standard/src/test/java/liquibase/sqlgenerator/core/CreateTableGeneratorPostgresTest.javaliquibase-standard/src/test/resources/liquibase/change/ChangeDefinitionTest.yaml
…h guard, blank-input normalization, PG10 test gate (refs liquibase#6885, PR liquibase#7759) CodeRabbit posted 3 actionable findings + 1 nitpick on PR liquibase#7759 (liquibase#7759); all four addressed here. 1. JdbcDatabaseSnapshot.enrichPostgresqlTablesResult — exclude CockroachDB. `CockroachDatabase extends PostgresDatabase`, so our existing guard `if (!(database instanceof PostgresDatabase) ...)` allowed Cockroach targets to fall into the enrichment path and run `pg_get_partkeydef` / `pg_partitioned_table` queries against catalogs that Cockroach does not expose with the same semantics — the snapshot would fail. Added an explicit `database instanceof CockroachDatabase` check to the early-return guard. Note: PR liquibase#6901's index-USING enrichment a few methods up in the same file has the identical guard gap (CodeRabbit's claim that the index path already excludes Cockroach is incorrect — we re-verified). Same fix shape should land there in a separate follow-up; out of scope for liquibase#6885. 2. CreateTableChange.generateCreateTableStatement — trim-to-null partitionBy. The raw `getPartitionBy()` was forwarded into the statement without normalization, so a whitespace-only attribute value (`<createTable partitionBy=" ">`) would have made the SQL generator emit a stray, malformed `... PARTITION BY ` clause. Now wrapped in `StringUtils.trimToNull(...)`, mirroring how the same method normalizes `tablespace` via `StringUtil.trimToNull(getTablespace())` in `generateStatements`. 3. CreateTableGenerator.generateSql — defense-in-depth trim-to-null. Even with fix (2) above, a programmatically-constructed `CreateTableStatement` (from a future change type, or from external code that builds statements directly) could still arrive with a whitespace-only `partitionBy`. The generator now also runs `StringUtils.trimToNull` on the value before deciding whether to append the clause. 4. PostgreSQLIntegrationTest.testSnapshotPartitionedTableCapturesPartitionBy — gate on PostgreSQL 10+. Declarative partitioning syntax (`PARTITION BY`) was introduced in PostgreSQL 10. Other tests in this file gate on `assumeTrue(getDatabase().getDatabaseMajorVersion() >= N)` for feature-version-dependent setup (e.g. `>= 12` for generated columns); matching that pattern here so the test is gracefully skipped on pre-PG10 setups instead of failing with a SQL syntax error. Tests ----- - CreateTableGeneratorPostgresTest grew from 7 to 8 cases; new `testGenerateSqlNormalizesBlankPartitionByToNull` exercises six blank-shaped inputs ("", " ", " ", "\t", "\n", " \n\t ") and asserts none of them produce a `PARTITION BY` substring in the output. - All 126 tests across CreateTableGeneratorPostgresTest (8), CreateTableChangeTest (62), TableSnapshotGeneratorTest (3), MissingTableChangeGeneratorTest (5), and ChangeDefinitionTest (48) pass. Refs liquibase#6885 Refs PR liquibase#7759 Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
filipelautert
left a comment
There was a problem hiding this comment.
Hey Khromushkin, reviewed the changes. Architecture, tests, and pre-emptive justifications all look right — thanks for the thorough PR body, made review fast. One real blocker to address:
-
CreateTableGenerator.java:358-374—PARTITION BYis emitted afterTABLESPACE, but Postgres grammar requires the opposite order:(columns) [PARTITION BY ...] [USING ...] [WITH ...] [ON COMMIT ...] [TABLESPACE ...]. Any user with bothpartitionByANDtablespaceset will get a syntax error from Postgres. Swap the two blocks — move thepartitionByappend (lines 369-374) above the tablespace block (lines 358-366). -
Test the combination —
CreateTableGeneratorPostgresTestcovers 5 partition shapes but no test setspartitionBy+tablespacetogether. Add one asserting the emitted SQL is... PARTITION BY RANGE (col) TABLESPACE my_ts(in that order).
Re your drive-by CWE-316 fix: diagnosis is accurate (#7741 did merge May 21 with the credential-wipe finally block) and your fix is the right shape — confirmed no other integration-test sites have the multi-execute-on-same-scope pattern.
…quibase#6885, PR liquibase#7759 review) Addresses the sole blocker in @filipelautert's CHANGES_REQUESTED review on PR liquibase#7759: > `PARTITION BY` is emitted *after* `TABLESPACE`, but Postgres grammar > requires the opposite order: > `(columns) [PARTITION BY ...] [USING ...] [WITH ...] [ON COMMIT ...] [TABLESPACE ...]` > Any user with both `partitionBy` AND `tablespace` set will get a > syntax error from Postgres. Fix --- `CreateTableGenerator.generateSql` previously emitted the TABLESPACE block (lines 358-366) before the PARTITION BY block (lines 369-374). The two blocks are now swapped: PARTITION BY appends first, TABLESPACE appends after. Matches the canonical PG `CREATE TABLE` clause order documented at https://www.postgresql.org/docs/current/sql-createtable.html. The swap is purely textual: each block is self-contained, neither depends on the other's state, and the surrounding blocks (MySQL autoincrement options at line 353-356, MySQL COMMENT at line 376-378, Oracle ROWDEPENDENCIES at line 380-382) are unaffected. Test ---- `CreateTableGeneratorPostgresTest.testGenerateSqlEmitsPartitionByBeforeTablespace` — new test, asserts the exact emitted SQL is `CREATE TABLE public.part_with_ts (id INTEGER, created_at INTEGER) PARTITION BY RANGE (created_at) TABLESPACE my_ts` with both a string-equality check and a defensive index-order check (PARTITION BY index < TABLESPACE index) in case whitespace drifts later. `CreateTableGeneratorPostgresTest` grew from 8 to 9 cases; full `CreateTableGeneratorTest` (55 cases, all dialects) still passes — the swap is invisible to any statement that doesn't set partitionBy, which is everything outside Postgres. Regression check: 182 tests pass across CreateTableGeneratorPostgresTest, CreateTableGeneratorTest, CreateTableChangeTest, TableSnapshotGeneratorTest, MissingTableChangeGeneratorTest, ChangeDefinitionTest. Refs liquibase#6885 Refs PR liquibase#7759 Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
|
Thanks @filipelautert — appreciated the quick turnaround. Both items addressed in f1fbdc4:
Full |
filipelautert
left a comment
There was a problem hiding this comment.
Hey Khromushkin, both fixes look good — re-approving.
Heads-up unrelated to the partition work: #7760 (@v-petrovych's standalone fixture refresh, same three ALLOW_* flags) merged earlier today. Your drive-by cc332c791 is now duplicative with main — when you rebase to pick up the merge, just drop that commit. Your other drive-by a2dd2af06 (the CommandScope test fix) is unaffected and still needed.
…or (refs liquibase#6885) Adds a new `partitionBy` property on CreateTableStatement, surfaced as a `String` field with `getPartitionBy()` / `setPartitionBy(String)` (fluent setter returning the statement, matching the pre-existing `setTablespace` shape). When the target database is PostgresDatabase and the property is non-null, CreateTableGenerator.generateSql appends ` PARTITION BY <verbatim>` after the tablespace clause and before the MySQL COMMENT clause — i.e. the clause is emitted in exactly the slot the maintainer pointed at in liquibase#6885 (liquibase#6885, filipelautert's comment referencing CreateTableGenerator.java:357). For non-Postgres databases that receive a CreateTableStatement with partitionBy set, CreateTableGenerator.warn now emits a Warning ("partitionBy clause is only supported on PostgreSQL; <DB> will silently ignore it"). This mirrors the precedent in CreateIndexGenerator added by PR liquibase#6901 for the `using` clause: silently-dropped Postgres-specific clauses should be loud. Scope of this commit -------------------- This is the SQL-generation slice of the liquibase#6885 fix only. The structural model (Table.partitionBy), CreateTableChange wiring, snapshot enrichment via pg_partitioned_table / pg_get_partkeydef, and diff-side propagation through MissingTableChangeGenerator land in subsequent commits in this PR. The codebase compiles and the broader CreateTable* test suite (55 CreateTableGeneratorTest + 56 CreateTableChangeTest + 1 CreateTableGeneratorInformixTest + 1 CreateTableStatementTest) stays green after this commit; nothing observable changes for users yet (the new property is just plumbing until the change object exposes it). Pattern justification --------------------- The maintainer (filipelautert) explicitly named PR liquibase#6901 as the structural template: > You need to do something similar to what we have in > liquibase#6901 (in this case, adding > types for indexes), as to make a diff you need to implement SQL > generation and snapshotting - then diff will work. [...] you'll need a > new property to store the partitionned table info too. PR liquibase#6901 adds `USING <method>` for CREATE INDEX. The shape — optional Postgres-specific clause, captured from a system catalog and emitted in the DDL — maps 1:1 to PARTITION BY for CREATE TABLE. Departures from liquibase#6901 worth noting: - Field name: `partitionBy` (camel-cased SQL keyword), mirroring liquibase#6901's `using` for `USING`. Other reasonable names (`partitionedBy`, `partitioning`) lose the direct keyword mirror. - Single shared generator: liquibase#6901 extends a separate CreateIndexGeneratorPostgres class. For tables there is no CreateTableGeneratorPostgres today; the base CreateTableGenerator is already heavily branched per-database (MSSQL/SybaseASA, AbstractDb2/ Informix, MySQL, Oracle), so the PARTITION BY branch joins those rather than spawning a new class. This is consistent with the existing pattern for the other clauses in the same method. Tests ----- New: CreateTableGeneratorPostgresTest (3 cases) testGenerateSqlAppendsPartitionByClauseForPostgres partitionBy="RANGE (test_date_int)" on PostgresDatabase produces: "CREATE TABLE public.test_tbl_part (test_date_int INTEGER) PARTITION BY RANGE (test_date_int)" testGenerateSqlWithoutPartitionByOnPostgresProducesPlainCreateTable partitionBy=null on PostgresDatabase: no PARTITION BY substring in the output. (Guards against accidentally emitting an empty PARTITION BY.) testWarnEmittedForPartitionByOnNonPostgresDatabase partitionBy="RANGE (dt)" on MySQLDatabase: a Warning containing "partition" is emitted via warn(). Empty SqlGeneratorChain used as the downstream stub — we test the leaf generator's contribution. Regression check: 116/116 tests pass for the CreateTable* matcher (CreateTableGeneratorTest, CreateTableGeneratorInformixTest, CreateTableGeneratorPostgresTest, CreateTableStatementTest, CreateTableChangeTest). Refs liquibase#6885 Refs liquibase#1874 Refs liquibase#7215 Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…hange (refs liquibase#6885) Adds the structural-model and change-object pieces needed for liquibase#6885's diff/ generate-changelog fix to round-trip a Postgres declarative-partitioning specification through Liquibase. After this commit, the changelog XML form `<createTable tableName="foo" partitionBy="RANGE (created_at)">` parses and re-applies correctly; the snapshot side (reading partitionBy from a live database via pg_partitioned_table / pg_get_partkeydef) lands in the next commit. Changes ------- 1. liquibase.structure.core.Table New `getPartitionBy()` / `setPartitionBy(String)` backed by the DatabaseObject attribute map, mirroring the existing `tablespace` accessor pattern. Documented as "verbatim pg_get_partkeydef output for declaratively-partitioned PostgreSQL tables". Null for non-partitioned tables and on databases that do not support declarative partitioning. 2. liquibase.change.core.CreateTableChange - New `@Setter private String partitionBy;` field with `@DatabaseChangeProperty(supportsDatabase = "postgresql")` getter so the XSD and YAML-emitter pick it up automatically. - `generateCreateTableStatement()` now threads `getPartitionBy()` onto the produced CreateTableStatement via the fluent setter introduced in the previous commit. - `getExcludedFieldFilters(ChecksumVersion)` now version-gates the `partitionBy` exclusion. For V8 and earlier, partitionBy is excluded from the checksum so pre-existing changesets (which never had this field) keep their stored checksums after upgrading. For V9+, partitionBy contributes to the checksum so changes to the partition spec are caught by validation. Pattern mirrors CreateViewChange.java:165 (added in 4.22.0 alongside the V8→V9 bump). Tests ----- New cases in CreateTableChangeTest: "partitionBy round-trips through setter/getter" Trivial setter/getter sanity. "generateStatements threads partitionBy onto the CreateTableStatement" A CreateTableChange with partitionBy="RANGE (test_date_int)" must surface that string on the produced CreateTableStatement (which is what the CreateTableGenerator added in the previous commit consumes). "getExcludedFieldFilters version-gating for partitionBy" — @unroll, 3 sub-cases: V7: contains("partitionBy") == true (excluded — smooth upgrade) V8: contains("partitionBy") == true (excluded — smooth upgrade) V9: contains("partitionBy") == false (included — validate changes) All 62 CreateTableChangeTest cases pass; broader CreateTable* matcher covers 122 tests, 0 failures. Refs liquibase#6885 Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…-generated changelog (fixes liquibase#6885) Closes the user-visible bug from liquibase#6885. After this commit, `liquibase generate-changelog` and `liquibase diff-changelog` against a PostgreSQL database that contains declaratively-partitioned tables now preserve the PARTITION BY clause in the generated <createTable> change set. Pre-fix, the same command silently produced a plain CREATE TABLE; applying the generated changelog to a target database produced a non-partitioned heap copy of the source — a structural divergence not visible in the changelog text itself. Scope: parent-table PARTITION BY clause only. Child-partition (`CREATE TABLE child PARTITION OF parent FOR VALUES ...`) enumeration via pg_inherits / pg_get_expr(relpartbound) is deliberately out of scope for this PR and remains as raw <sql> change sets — same as today's user workaround. See the design discussion in the PR body for rationale. Changes ------- 1. liquibase.snapshot.JdbcDatabaseSnapshot.queryPostgres + new private enrichPostgresqlTablesResult(...) method queryPostgres' existing extract(databaseMetaData.getTables(..., {"TABLE", "PARTITIONED TABLE"})) already pulled partitioned-table rows into the cache (pg_class.relkind = 'p') but JDBC's getTables metadata does not expose the partition strategy or key columns. The new enrichment runs a targeted query against pg_partitioned_table joined to pg_class / pg_namespace, retrieving pg_get_partkeydef(c.oid) — which returns the verbatim right-hand side of a PARTITION BY clause: "RANGE (col)", "LIST (region)", "HASH (id)", "RANGE (lower(email))" for functional keys, etc. — and stamps it onto matching rows in the existing returnList as a synthetic PARTITION_BY column, keyed on (TABLE_SCHEM, TABLE_NAME). This mirrors the index-USING enrichment pattern added in PR liquibase#6901 (which lives in the same file under getIndexInfo). The schema and tableName scoping clauses mirror that pattern as well; the table is small (one row per partitioned parent) so the cost is negligible. queryPostgres' throws-clause gains DatabaseException (already declared on both callers, fastFetchQuery / bulkFetchQuery). 2. liquibase.snapshot.jvm.TableSnapshotGenerator.readTable New branch: when database instanceof PostgresDatabase, read the PARTITION_BY enrichment column off the CachedRow and call table.setPartitionBy(...). Existing readTable behaviour for all other dialects (and for non-partitioned PG tables, which have null PARTITION_BY after enrichment) is unchanged. 3. liquibase.diff.output.changelog.core.MissingTableChangeGenerator.fixMissing When the reference database is PostgresDatabase and the missing Table has a partitionBy attribute set, propagate it onto the produced CreateTableChange. This is the final hop that gets the value into the serialized <createTable> XML. Tests ----- New: TableSnapshotGeneratorTest (3 cases, Spock) readTable picks up PARTITION_BY enrichment column when database is PostgresDatabase — CachedRow with PARTITION_BY="RANGE (test_date_int)" yields table.getPartitionBy() == "RANGE (test_date_int)" readTable does not set partitionBy when PARTITION_BY column is absent — CachedRow without PARTITION_BY yields null partitionBy. readTable ignores PARTITION_BY column on non-Postgres databases (safety guard) — even with PARTITION_BY="..." on the row, MySQLDatabase yields null partitionBy. Guards against accidental cross-dialect leakage. New cases in MissingTableChangeGeneratorTest (Spock, 2 added): fixMissing propagates partitionBy from Postgres reference Table onto CreateTableChange — Table.partitionBy="RANGE (test_date_int)" with referenceDatabase=PostgresDatabase yields a CreateTableChange whose getPartitionBy() returns the same string. fixMissing leaves partitionBy null when reference Table has no partition spec — sanity guard that the propagation doesn't accidentally inject empty strings. Manual end-to-end verification against postgres:16 in colima: CREATE TABLE public.test_tbl_part (test_date_int integer NOT NULL, PRIMARY KEY (test_date_int)) PARTITION BY RANGE (test_date_int); CREATE TABLE public.test_tbl_part_2026 PARTITION OF public.test_tbl_part FOR VALUES FROM (20260101) TO (20270101); $ liquibase generate-changelog ... Before (current main, baseline): <createTable tableName="test_tbl_part"> <column name="test_date_int" type="INTEGER">...</column> </createTable> After (this commit): <createTable partitionBy="RANGE (test_date_int)" tableName="test_tbl_part"> <column name="test_date_int" type="INTEGER">...</column> </createTable> Diff confined to the new partitionBy attribute (plus changeSet id timestamps, which are wall-clock-dependent and not meaningful). Regression check: 249 tests pass across the *Table*, *Snapshot*, *Missing*, *Changed* matchers (CreateTableGeneratorTest, CreateTableChangeTest, TableSnapshotGeneratorTest, MissingTableChangeGeneratorTest, TableComparatorTest, CreateTableGeneratorInformixTest, CreateTableGeneratorPostgresTest, CreateTableStatementTest, plus the snapshot-id and parser tests caught by the glob). 0 failures. Fixes liquibase#6885 Closes liquibase#1874 Refs liquibase#7215 Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…or partitionBy attribute (refs liquibase#6885) Completes the user-facing surface of the liquibase#6885 fix: 1. XSD: `<xsd:attribute name="partitionBy" type="xsd:string"/>` on the `createTable` element in dbchangelog-latest.xsd. Without this, IDE XML validators (the Liquibase XSD ships with) flag `partitionBy` as an unknown attribute, which would be a poor user experience even though the runtime parser accepts unknown attributes via the existing `<xsd:anyAttribute namespace="##other" .../>` slot. 2. ChangeDefinitionTest.yaml: golden-file entry for the createTable `partitionBy` property under postgresql-only support. This file is regenerated against the @DatabaseChangeProperty annotation metadata by ChangeDefinitionTest at test time — without this entry, that test would fail because the auto-generated definition for createTable includes the new property and the YAML didn't. 3. PostgreSQLIntegrationTest.testSnapshotPartitionedTableCapturesPartitionBy End-to-end regression test. Mirrors PR liquibase#6901's pattern (testSnapshotIndexSnapshotsIndexFunction in the same file): a. Creates `CREATE TABLE part_test (id BIGINT, sold_at DATE NOT NULL) PARTITION BY RANGE (sold_at)` via raw SQL on the test database. b. Calls DiffGeneratorFactory.compare and produces a changelog via DiffToChangeLog.generateChangeSets. c. Asserts the resulting CreateTableChange.getPartitionBy() is set and contains both "RANGE" and "sold_at" (case-insensitive — the exact case returned by pg_get_partkeydef varies by PG version). d. Always drops the test table in a finally block so the assertion path is hermetic. Pre-fix, this test would have asserted false on step (c). Post-fix it asserts true. The test only runs against postgresql test systems (gated by assumeNotNull(this.getDatabase())). Manual end-to-end verification against postgres:16 in colima already showed the diff confined to the partitionBy attribute (see commit message of `fix(postgres): capture PARTITION BY in snapshot...`). Out of scope (deliberate) ------------------------- - Child partitions: `CREATE TABLE child PARTITION OF parent FOR VALUES ...` still emit as standalone <createTable> entries without partition-of linkage. Users continue to add `<sql>create table child partition of parent ...</sql>` change sets manually. A follow-up PR can add pg_inherits-based enumeration + a partitionOf attribute. Not in this PR because it doubles the surface area and the maintainer's guidance in liquibase#6885 referenced only the parent property. - ChangedTableChangeGenerator: no propagation added. That generator only produces SetTableRemarksChange today (and warns about tablespace changes). Postgres itself cannot ALTER a table's partition spec — changing it requires DROP+CREATE+data migration, which is outside Liquibase's diff-changelog scope. No new behaviour here. Refs liquibase#6885 Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…refs liquibase#6885) PR liquibase#6901 added the index `using` attribute to BOTH dbchangelog-latest.xsd AND the versioned dbchangelog-5.0.xsd (which is the schema URL emitted by default in changelog headers on Liquibase 5.x). The earlier commit "test(postgres): add liquibase#6885 integration test + XSD/YAML docs for partitionBy" on this branch only updated -latest.xsd, mirroring an incomplete reading of the liquibase#6901 pattern. Consequence pre-fix: PostgreSQLIntegrationTest.testRerunDiffChangeLog (and any other test that reparses a generated changelog with the canonical 5.0 namespace URL) fails XSD validation: ChangeLogParseException: ... cvc-complex-type.3.2.2: Attribute 'partitionBy' is not allowed to appear in element 'createTable'. Fix: add the same `<xsd:attribute name="partitionBy" type="xsd:string"/>` to dbchangelog-5.0.xsd in the same position (after rowDependencies, before the trailing <xsd:anyAttribute>), exactly mirroring the placement chosen in -latest.xsd and in PR liquibase#6901's createIndex update to the same file. No older versioned XSDs (1.0 through 4.33) are touched; those remain frozen with their historical content, same as the index-USING precedent. Refs liquibase#6885 Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
… password-clearing in CommandScope testStatusRunDuringUpdate reused a single CommandScope across three sequential .execute() calls (statusCheckBefore, statusCheckDuring, statusCheckAfter). PR liquibase#7741 ("CWE-316: clear password memory after use in legacy CLI + CommandScope", 4850342, 2026-05-21) added a finally block to CommandScope.execute() that overwrites credential-shaped argument values (password, passwd, secret, token, apikey, accesskey) with "*****" after the pipeline runs. This is the right hardening for the production single-execute use of CommandScope, but it makes a single CommandScope instance single-use: CommandScope scope = ...; // password=LiquibasePass1 scope.execute(); // password gets overwritten to "*****" scope.execute(); // 2nd run authenticates with "*****" -> FATAL Symptom on current upstream/main when running this test: liquibase.exception.DatabaseException: ... Connection could not be created to jdbc:postgresql://localhost:NNNN/lbcat: FATAL: password authentication failed for user "lbuser" at PostgreSQLIntegrationTest.testStatusRunDuringUpdate(line 93) Line 93 is the second commandScope.execute() — statusCheckDuring. The first .execute() (statusCheckBefore) succeeds; the second fails because the scope's credential argument has been wiped. Fix: build a fresh CommandScope per snapshot via the existing `getStatusCommandScope(changelogFile)` helper, which constructs a new scope and re-applies username/password from `testSystem`. Each .execute() runs against an unwiped credential storage. Test logic, assertions, and timing semantics are otherwise unchanged. Verified locally against postgres:15 in colima — the test now passes deterministically. Reproduced the failure on pure upstream/main (no other changes) before the fix to confirm this is a regression of the upstream commit, not something this branch introduced. This commit is independent of liquibase#6885 — surfacing it here only because it was the last red item blocking the local pre-push test run. If the maintainers prefer, this single commit can be cherry-picked into a standalone PR; it is fully self-contained and small. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…javadoc, message, deprecation, ChangedTable warning, edge-case tests Five small fixes surfaced by self-review against the patterns CodeRabbit, Copilot, and GitHub Advanced Security flag on Liquibase PRs (liquibase#7464, liquibase#6901, liquibase#7741). All defensible individually; rolled up to keep the polish off the load-bearing commits. 1. JdbcDatabaseSnapshot.enrichPostgresqlTablesResult — fix empty `{@link }` javadoc tag. Inside the method's javadoc the original referent of pg_get_partkeydef was written as `{@link }` (no target) because pg_get_partkeydef is a SQL function, not a Java identifier the javadoc tool can resolve. Replaced with `{@code pg_get_partkeydef(oid)}` — semantically a code reference, no broken link warning from javadoc:jar. 2. CreateTableGenerator.warn — clarify warning text. Pre: "partitionBy clause is only supported on PostgreSQL; X will silently ignore it" Post: "partitionBy is only supported on PostgreSQL; the clause will be dropped from the generated SQL for X" "silently ignore" implied the clause was emitted and the target db ignored it. In reality the CreateTableGenerator skips the append entirely for non-PG, so the clause is dropped at generation, not silently ignored at apply time. The existing test (testWarnEmittedForPartitionByOnNonPostgresDatabase) still passes — it asserts case-insensitive "partition" in the message. 3. TableSnapshotGenerator.readTable — use non-deprecated trimToNull for the new partitionBy read. The previous commit `fix(postgres): capture PARTITION BY in snapshot ...` added `liquibase.util.StringUtil.trimToNull(...)` which is marked `@Deprecated` (javadoc points at `org.apache.commons.lang3.StringUtils.trimToNull`). GitHub Advanced Security's code-scanning bot flags any new use of the deprecated overload in PR diffs (verified on PR liquibase#7464 line — issue 2245). Switched the one new call site to `org.apache.commons.lang3.StringUtils`. The five surrounding pre-existing `StringUtil.trimToNull` calls in readTable are left untouched — they're not new code and a wholesale migration of readTable is out of scope for liquibase#6885. 4. ChangedTableChangeGenerator.fixChanged — warn on partitionBy diff. Mirrors the existing tablespace-change warning a few lines above. Without this, a user who changes a table's `partitionBy` between two snapshots gets a silent no-op changelog (the diff sees the difference but neither emits a change nor warns). Postgres declarative partitioning can't be ALTER'd — the parent table has to be dropped and recreated. Tell the operator explicitly rather than producing a deceptively-empty diff. No unit test added: ChangedTableChangeGeneratorTest does not exist in the repo, and the existing tablespace-change warning is similarly untested at the unit level. Following precedent. 5. CreateTableGeneratorPostgresTest — add 4 verbatim-passthrough cases. pg_get_partkeydef returns several shapes that the existing single test (RANGE on a single column) didn't exercise: - testGenerateSqlEmitsVerbatimForMultiColumnRangeKey: "RANGE (a, b)" - testGenerateSqlEmitsVerbatimForFunctionalKey: "RANGE (lower(email))" - testGenerateSqlEmitsVerbatimForQuotedIdentifier: 'RANGE ("MixedCaseCol")' - testGenerateSqlEmitsListAndHashStrategiesVerbatim: "LIST (region)", "HASH (user_id)" All four assert the generator passes the partitionBy string through unchanged — confirming the design decision (D4 in the brief: verbatim, no normalization) is enforced by tests. Regression check: 6080 tests pass across liquibase-standard, 0 failures, 12 environment-dependent skips (normal). 7 cases now in CreateTableGeneratorPostgresTest (3 prior + 4 added). Refs liquibase#6885 Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…h guard, blank-input normalization, PG10 test gate (refs liquibase#6885, PR liquibase#7759) CodeRabbit posted 3 actionable findings + 1 nitpick on PR liquibase#7759 (liquibase#7759); all four addressed here. 1. JdbcDatabaseSnapshot.enrichPostgresqlTablesResult — exclude CockroachDB. `CockroachDatabase extends PostgresDatabase`, so our existing guard `if (!(database instanceof PostgresDatabase) ...)` allowed Cockroach targets to fall into the enrichment path and run `pg_get_partkeydef` / `pg_partitioned_table` queries against catalogs that Cockroach does not expose with the same semantics — the snapshot would fail. Added an explicit `database instanceof CockroachDatabase` check to the early-return guard. Note: PR liquibase#6901's index-USING enrichment a few methods up in the same file has the identical guard gap (CodeRabbit's claim that the index path already excludes Cockroach is incorrect — we re-verified). Same fix shape should land there in a separate follow-up; out of scope for liquibase#6885. 2. CreateTableChange.generateCreateTableStatement — trim-to-null partitionBy. The raw `getPartitionBy()` was forwarded into the statement without normalization, so a whitespace-only attribute value (`<createTable partitionBy=" ">`) would have made the SQL generator emit a stray, malformed `... PARTITION BY ` clause. Now wrapped in `StringUtils.trimToNull(...)`, mirroring how the same method normalizes `tablespace` via `StringUtil.trimToNull(getTablespace())` in `generateStatements`. 3. CreateTableGenerator.generateSql — defense-in-depth trim-to-null. Even with fix (2) above, a programmatically-constructed `CreateTableStatement` (from a future change type, or from external code that builds statements directly) could still arrive with a whitespace-only `partitionBy`. The generator now also runs `StringUtils.trimToNull` on the value before deciding whether to append the clause. 4. PostgreSQLIntegrationTest.testSnapshotPartitionedTableCapturesPartitionBy — gate on PostgreSQL 10+. Declarative partitioning syntax (`PARTITION BY`) was introduced in PostgreSQL 10. Other tests in this file gate on `assumeTrue(getDatabase().getDatabaseMajorVersion() >= N)` for feature-version-dependent setup (e.g. `>= 12` for generated columns); matching that pattern here so the test is gracefully skipped on pre-PG10 setups instead of failing with a SQL syntax error. Tests ----- - CreateTableGeneratorPostgresTest grew from 7 to 8 cases; new `testGenerateSqlNormalizesBlankPartitionByToNull` exercises six blank-shaped inputs ("", " ", " ", "\t", "\n", " \n\t ") and asserts none of them produce a `PARTITION BY` substring in the output. - All 126 tests across CreateTableGeneratorPostgresTest (8), CreateTableChangeTest (62), TableSnapshotGeneratorTest (3), MissingTableChangeGeneratorTest (5), and ChangeDefinitionTest (48) pass. Refs liquibase#6885 Refs PR liquibase#7759 Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…quibase#6885, PR liquibase#7759 review) Addresses the sole blocker in @filipelautert's CHANGES_REQUESTED review on PR liquibase#7759: > `PARTITION BY` is emitted *after* `TABLESPACE`, but Postgres grammar > requires the opposite order: > `(columns) [PARTITION BY ...] [USING ...] [WITH ...] [ON COMMIT ...] [TABLESPACE ...]` > Any user with both `partitionBy` AND `tablespace` set will get a > syntax error from Postgres. Fix --- `CreateTableGenerator.generateSql` previously emitted the TABLESPACE block (lines 358-366) before the PARTITION BY block (lines 369-374). The two blocks are now swapped: PARTITION BY appends first, TABLESPACE appends after. Matches the canonical PG `CREATE TABLE` clause order documented at https://www.postgresql.org/docs/current/sql-createtable.html. The swap is purely textual: each block is self-contained, neither depends on the other's state, and the surrounding blocks (MySQL autoincrement options at line 353-356, MySQL COMMENT at line 376-378, Oracle ROWDEPENDENCIES at line 380-382) are unaffected. Test ---- `CreateTableGeneratorPostgresTest.testGenerateSqlEmitsPartitionByBeforeTablespace` — new test, asserts the exact emitted SQL is `CREATE TABLE public.part_with_ts (id INTEGER, created_at INTEGER) PARTITION BY RANGE (created_at) TABLESPACE my_ts` with both a string-equality check and a defensive index-order check (PARTITION BY index < TABLESPACE index) in case whitespace drifts later. `CreateTableGeneratorPostgresTest` grew from 8 to 9 cases; full `CreateTableGeneratorTest` (55 cases, all dialects) still passes — the swap is invisible to any statement that doesn't set partitionBy, which is everything outside Postgres. Regression check: 182 tests pass across CreateTableGeneratorPostgresTest, CreateTableGeneratorTest, CreateTableChangeTest, TableSnapshotGeneratorTest, MissingTableChangeGeneratorTest, ChangeDefinitionTest. Refs liquibase#6885 Refs PR liquibase#7759 Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
f1fbdc4 to
ffd1406
Compare
|
Thanks @filipelautert for handling the rebase — appreciated, didn't expect it. Two follow-ups:
Anything else I can pick up? |
Impact
Description
PostgreSQL
diff-changelogandgenerate-changelogsilently dropped thePARTITION BYclause on declaratively-partitioned tables. The generatedchangelog described a plain heap table; applying it to a fresh database produced
a non-partitioned copy of the source. Structural divergence not visible in the
changelog text.
This PR adds first-class support for the parent-table
PARTITION BY <strategy> (<keys>)clause via a new optional
partitionByattribute on<createTable>. Capturedverbatim from
pg_get_partkeydef(c.oid)at snapshot time, round-tripped throughthe diff and SQL-generation paths, emitted in the changelog XML, and re-applied
on
liquibase update.Fixes #6885
Closes #1874
Refs #7215
Structural template
PR #6901 (
CREATE INDEX ... USING <method>) is the exact same shape — anoptional Postgres-specific clause captured from a system catalog and emitted in
DDL. @filipelautert explicitly named #6901 as the template in the #6885 thread,
with pointers to
TableSnapshotGenerator.java:83andCreateTableGenerator.java:357. This PR follows that template across 5 commits(the remaining 3 are documented under "Drive-by fixes" below).
Verification
Before (current
main, postgres:16 in colima):After:
liquibase-standard: 6080 unit tests, 0 failures, 12 environment-dependentskips.
PostgreSQLIntegrationTestagainst postgres:15 via TestSystemFactory:54/54 passing, 1 environment-dependent skip. Includes a new
testSnapshotPartitionedTableCapturesPartitionByintegration regression testthat fails on
mainbefore this PR and passes after.Release note
PostgreSQL
diff-changelogandgenerate-changelognow preserve thePARTITION BYclause on declaratively-partitioned tables. Previously thepartition strategy was silently dropped from the generated changelog,
producing a non-partitioned copy when applied to a target database. The new
behaviour is also expressible directly in handwritten changelogs via the
partitionByattribute on<createTable>— for example<createTable tableName="orders" partitionBy="RANGE (created_at)">.Things to be aware of
Checksum stability
The new
partitionByfield is version-gated inCreateTableChange.getExcludedFieldFilters(ChecksumVersion):V8and earlier: excluded from the checksum. Pre-existing<createTable>changesets that have already been applied keep their stored checksum after
upgrading — no
Validation Failed: Stored checksum ...errors and noforced
clear-checksumsruns.V9(current latest, since Liquibase 4.22.0): included in the checksum.Changes to the partition specification on V9-era changesets are caught by
validation rather than silently slipping through.
Mirrors the version-gating pattern in
CreateViewChange.java:165-180(addedalongside the V8→V9 bump). This is a more thoughtful approach than the simple
always-excludechosen forusingin PR #6901; happy to align with thatprecedent if reviewers prefer.
Verbatim partition spec storage
The string stored in
partitionByis whateverpg_get_partkeydef(c.oid)returns —
"RANGE (created_at)","LIST (region)","HASH (id)","RANGE (lower(email))"(functional),"RANGE (\"MixedCaseCol\")"(quoted),multi-column
"RANGE (a, b)". No upper-casing, normalization, or parsing.The output is already valid Postgres syntax for the right-hand side of
PARTITION BY. Tests inCreateTableGeneratorPostgresTestcover all fiveshapes.
Postgres-specific snapshot enrichment
JdbcDatabaseSnapshot.queryPostgresalready requestedTABLE_TYPE = 'PARTITIONED TABLE'from JDBC metadata, so partitioned-tablerows were in the cache — but JDBC does not expose the partition strategy or
key columns. A new
enrichPostgresqlTablesResultjoinspg_class,pg_namespace,pg_partitioned_tableand projectspg_get_partkeydef(c.oid)as a synthetic
PARTITION_BYcolumn on matching rows, keyed on(TABLE_SCHEM, TABLE_NAME). Mirrors PR #6901's index-USING enrichment in thesame file's
getIndexInfoextractor.The schema filter (and table filter, when scoped) are applied at SQL time
before the Java-side matching loop runs, so the matching is N×M with both
sides bounded by the partitioned-parent count in the schema — typically <10
even on large databases. No measurable cost.
Things to worry about (pre-emptive answers to common review prompts)
"Missing escape on
statement.getPartitionBy()in the SQL append"partitionByis a clause-tail expression(
"RANGE (a, b)","LIST (region)","HASH (lower(email))"), not anidentifier. The existing identifier-escapers
(
escapeColumnName,escapeTableName,escapeTablespaceName) don't apply;there is no expression-level parser/sanitizer in the codebase. Precedent for
verbatim concatenation of clause-tail expressions:
CreateIndexGeneratorPostgres.generateSql:53-55—buffer.append(" USING ").append(statement.getUsing())(PR feat: Add PostgreSQL Index Function Support via USING Clause #6901, mergedApril 2025, no escape requested at review)
CreateTableGenerator.generateSql:362,364—sql += " TABLESPACE " + statement.getTablespace()(long-standing)Trust boundary:
partitionByis set either bypg_get_partkeydef()at difftime (a trusted PG-internal source) or by a changelog author writing
<createTable partitionBy="...">(the same trust level that allows arbitrary<sql>blocks). No new attack surface introduced."ChangedTableChangeGenerator no-ops for partition-spec changes"
Postgres declarative partitioning cannot be added, removed, or modified via
ALTER TABLE — the parent table has to be dropped and recreated. The
ChangedTableChangeGeneratormirrors the existing tablespace behaviour:a
warningis logged when apartitionBydifference is detected, no changeis emitted. This is consistent with how
tablespacechanges are handled inthe same generator.
"Why not a separate
CreateTableGeneratorPostgres?"The base
CreateTableGenerator.generateSqlis already heavily branched perdatabase (
MSSQLDatabase,SybaseASADatabase,AbstractDb2Database,InformixDatabase,MySQLDatabase,OracleDatabase). A one-lineif (database instanceof PostgresDatabase) sql += " PARTITION BY ..."joins those rather than spawning a new class — consistent with existing
precedent.
Scope (deliberate exclusions)
Child partitions (
CREATE TABLE child PARTITION OF parent FOR VALUES ...)are out of scope for this PR. They remain as raw
<sql>change sets — thesame workaround users already employ. A follow-up PR can add
pg_inherits+pg_get_expr(relpartbound, oid)enumeration plus apartitionOf/partitionBoundattribute pair (or a new
<createPartition>change type). Doubling thesurface area into one PR would significantly increase the review burden and
risk "let's discuss the architecture" deferral — keeping V1 tight per the
maintainer guidance in #6885.
Drive-by fixes (cleanly cherry-pickable)
Two commits unrelated to #6885 — surfaced because they were blocking the
local pre-push test run, and isolating them into separate PRs would have
delayed shipping the load-bearing fix. If maintainers prefer, both are
self-contained:
cc332c791 test(cli): refresh LiquibaseCommandLineTest 'help output' fixture— three new
--allow-*global options (CWE-22/-78/-470 deprecation flags)were added on
mainrecently but the hardcodedexpectedHelpOutputinLiquibaseCommandLineTest.groovywas not regenerated. The test fails on aclean
maincheckout. Verified by stashing this branch's changes beforerunning the test. Mechanical regeneration from the live
liquibase --helpoutput.
a2dd2af06 test(postgres): fix testStatusRunDuringUpdate regression from CWE-316—
PostgreSQLIntegrationTest.testStatusRunDuringUpdatecallscommandScope.execute()three times on the sameCommandScopeinstance.PR CWE-316: clear password memory after use in legacy CLI + CommandScope #7741 (CWE-316, May 21) added a finally block in
CommandScope.execute()that overwrites credential argument values with
"*****"after thepipeline runs — making a single
CommandScopeinstance single-use. Secondcall gets
FATAL: password authentication failed for user "lbuser". Fixis one line per call site: build a fresh
CommandScopeper snapshot viathe existing
getStatusCommandScope(...)helper. Test logic, assertions,and timing semantics are otherwise unchanged.
Additional Context
afe00c4a4.in
TableSnapshotGenerator.java:83andCreateTableGenerator.java:357-356and named PR feat: Add PostgreSQL Index Function Support via USING Clause #6901 as the structural template. Both pointers are honored
verbatim.
🤖 Generated with Claude Code
The contributor stays accountable for all architecture and scope decisions on this PR. (this line typed without AI assistance, just to be clear 🙃) — @Khromushkin