Support PostgreSql UnNest Array to rows #5673

griffio · 2025-02-27T13:56:06Z

Support Unnest Function - expand an array to a set of rows

fixes #5346

🪹 🏗️ Work
🚧 🪺 In Progress

Add "unnest" to PostgreSql.bnf
Add mixins to help with column resolving as there are new aliases used by "unnest" in FROM
Update PostresqResolver to handle converting Array to T - this was not supported by IntermediateType
Add TableFunctionRowType to allow new concept of nested type to row type - not an actual table
Add fixture and integration tests

Prototype use project https://github.com/griffio/sqldelight-postgres-unnest

PostgreSQL allows a function (e.g unnest) to be written directly as a member of the FROM list. Allows multiple arrays to be unnested
SELECT unnest() allows a single array to be unnested

Examples:

Allows "Array DML" - e.g bulk inserts, updates and deletes

CREATE TABLE Business(
    id SERIAL PRIMARY KEY,
    name TEXT NOT NULL,
    zipcodes TEXT[] NOT NULL,
    headcounts INTEGER[] NOT NULL
);

CREATE TABLE Users (
   name TEXT NOT NULL,
   age INTEGER NOT NULL
);

select:
SELECT name, location.headcount, location.zipcode
FROM Business, UNNEST(zipcodes, headcounts) AS location(zipcode, headcount);

counts:
SELECT name, UNNEST(headcounts) AS headcount
FROM Business
ORDER BY headcount DESC;

array:
SELECT unnest('{1,2}'::INTEGER);

insertUsers:
INSERT INTO Users (name, age)
SELECT * FROM UNNEST(?::TEXT[], ?::INTEGER[])  AS u(name, age); 
// currently must explicitly add alias columns due to wildcard expansion in SqlDelight

updateUsers:
UPDATE Users
SET age = updates.updated_age
FROM UNNEST(?::TEXT[], ?::INTEGER[]) AS updates(name, updated_age)
WHERE Users.name = updates.name;

deleteUsers:
DELETE FROM Users
WHERE (name, age) IN (
  SELECT *
  FROM UNNEST(?::TEXT[], ?::INTEGER[]) AS u(name, age)
);

selectLocations:
SELECT DISTINCT b.*
FROM Business b
JOIN LATERAL UNNEST(b.zipcodes) AS loc(zipcode) ON loc.zipcode ILIKE '%' || :query || '%';

selectBusinessExists:
SELECT *
FROM business
WHERE EXISTS (
    SELECT 1
    FROM unnest(locations) AS loc 
    WHERE lower(loc) LIKE '%' || LOWER(:query::TEXT) || '%'
);

TODO
- Add current limitations and problems
  - PostgreSql allows functions declared after FROM e.g. FROM upper('a') or FROM unnest(...)
    - This is not supported in SqlDelight and is the actual issue

Add unnest to PostgreSql.bnf Add mixins to help with column resolving Update PostresqResolver to handle converting Array<T> to T Add TableFunctionRowType to allow new concept of Nested type to Row type

Try and support WHERE clause

Add where exists

For where clause

Update TableFunctionTableAliasMixin.kt

Add test for LATER UNNEST

fix conflicts

fix `The Kotlin type of the argument cannot be inferred, use CAST instead` LOWER and UPPER are no longer for TEXT but can be used with ranges so the bind param cannot be inferred

AlecKazakova

amazing, thank you

Angel-Ponce · 2025-05-15T20:19:41Z

What version should to be installed to use this feature?

griffio · 2025-05-16T08:05:21Z

What version should to be installed to use this feature?

Currently as snapshot version - you can try and see if it usable and report any issues

2.1.0-SNAPSHOT

repositories {
    maven(url = "https://oss.sonatype.org/content/repositories/snapshots")
    ...
}

And newly released in

2.1.0

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.github.jsqlparser:jsqlparser](https://github.com/JSQLParser/JSqlParser) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `5.2` -> `5.3` | | [app.cash.sqldelight](https://github.com/sqldelight/sqldelight) | plugin | misk/gradle/libs.versions.toml | gradle | minor | `2.0.2` -> `2.1.0` | | [app.cash.sqldelight:runtime](https://github.com/sqldelight/sqldelight) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.0.2` -> `2.1.0` | | [app.cash.sqldelight:mysql-dialect](https://github.com/sqldelight/sqldelight) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.0.2` -> `2.1.0` | | [app.cash.sqldelight:jdbc-driver](https://github.com/sqldelight/sqldelight) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.0.2` -> `2.1.0` | | [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.44` -> `2.31.45` | | [software.amazon.awssdk:sqs](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.44` -> `2.31.45` | | [software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.44` -> `2.31.45` | | [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.44` -> `2.31.45` | | [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.44` -> `2.31.45` | | [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.44` -> `2.31.45` | | [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.44` -> `2.31.45` | --- ### Release Notes <details> <summary>sqldelight/sqldelight (app.cash.sqldelight)</summary> ### [`v2.1.0`](https://github.com/sqldelight/sqldelight/blob/HEAD/CHANGELOG.md#210---2025-05-16) [Compare Source](sqldelight/sqldelight@2.0.2...2.1.0) ##### Added - \[WASM Driver] Add support for wasmJs to web worker driver ([#5534](sqldelight/sqldelight#5534) by \[Ilya Gulya]\[IlyaGulya]) - \[PostgreSQL Dialect] Support PostgreSql UnNest Array to rows ([#5673](sqldelight/sqldelight#5673) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] PostgreSql TSRANGE/TSTZRANGE support ([#5297](sqldelight/sqldelight#5297) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] PostgreSql Right Full Join ([#5086](sqldelight/sqldelight#5086) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] Postrgesql extract from temporal types ([#5273](sqldelight/sqldelight#5273) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] PostgreSql array contains operators ([#4933](sqldelight/sqldelight#4933) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] PostgreSql drop constraint ([#5288](sqldelight/sqldelight#5288) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] Postgresql type casting ([#5089](sqldelight/sqldelight#5089) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] PostgreSql lateral join operator for subquery ([#5122](sqldelight/sqldelight#5122) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] Postgresql ILIKE operator ([#5330](sqldelight/sqldelight#5330) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] PostgreSql XML type ([#5331](sqldelight/sqldelight#5331) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] PostgreSql AT TIME ZONE ([#5243](sqldelight/sqldelight#5243) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] Support postgresql order by nulls ([#5199](sqldelight/sqldelight#5199) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] Add PostgreSQL current date/time function support ([#5226](sqldelight/sqldelight#5226) by \[Drew Dobson]\[drewd]) - \[PostgreSQL Dialect] PostgreSql Regex operators ([#5137](sqldelight/sqldelight#5137) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] add brin gist ([#5059](sqldelight/sqldelight#5059) by \[Griffio]\[griffio]) - \[MySQL Dialect] Support RENAME INDEX for MySql dialect ([#5212](sqldelight/sqldelight#5212) by \[Oren Kislev]\[orenkislev-faire]) - \[JSON Extension] Add alias to json table function ([#5372](sqldelight/sqldelight#5372) by \[Griffio]\[griffio]) ##### Changed - \[Compiler] Generated query files return row counts for simple mutators ([#4578](sqldelight/sqldelight#4578) by \[Marius Volkhart]\[MariusV]) - \[Native Driver] Update NativeSqlDatabase.kt to change readonly flag for DELETE, INSERT, and UPDATE statements ([#5680](sqldelight/sqldelight#5680) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] Change PgInterval to String ([#5403](sqldelight/sqldelight#5403) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] Support SqlDelight modules to implement PostgreSql extensions ([#5677](sqldelight/sqldelight#5677) by \[Griffio]\[griffio]) ##### Fixed - \[Compiler] fix: notify queries when executing group statements with result ([#5006](sqldelight/sqldelight#5006) by \[Vitor Hugo Schwaab]\[vitorhugods]) - \[Compiler] Fix SqlDelightModule type resolver ([#5625](sqldelight/sqldelight#5625) by \[Griffio]\[griffio]) - \[Compiler] Fix 5501 insert object escaped column ([#5503](sqldelight/sqldelight#5503) by \[Griffio]\[griffio]) - \[Compiler] Compiler: Improve error message such that path links are clickable with the correct line & char position. ([#5604](sqldelight/sqldelight#5604) by \[Niklas Baudy]\[vanniktech]) - \[Compiler] Fix issue 5298: allow keywords to be used as table names - \[Compiler] fix named executes and add test - \[Compiler] Consider foreign key table constraints when sorting initialization statements ([#5325](sqldelight/sqldelight#5325) by \[Leon Linhart]\[TheMrMilchmann]) - \[Compiler] Align error underlines properly when tabs are involved ([#5224](sqldelight/sqldelight#5224) by \[Drew Dobson]\[drewd]) - \[JDBC Driver] Fix memory leak for connectionManager during end of transaction - \[JDBC Driver] Run SQLite migrations inside transaction as mentioned in documentation ([#5218](sqldelight/sqldelight#5218) by \[Lukáš Moravec]\[morki]) - \[JDBC Driver] Fix leaking connections after transaction commit / rollback ([#5205](sqldelight/sqldelight#5205) by \[Lukáš Moravec]\[morki]) - \[Gradle Plugin] Execute `DriverInitializer` before `GenerateSchemaTask` ([#5562](sqldelight/sqldelight#5562) by \[Emeka Nwagu]\[nwagu]) - \[Runtime] Fix crash in LogSqliteDriver when real driver is Async ([#5723](sqldelight/sqldelight#5723) by \[Eric Denman]\[edenman]) - \[Runtime] Fix StringBuilder capacity ([#5192](sqldelight/sqldelight#5192) by \[Jan Bína]\[janbina]) - \[PostgreSQL Dialect] PostgreSql create or replace view ([#5407](sqldelight/sqldelight#5407) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] Postgresql to_json ([#5606](sqldelight/sqldelight#5606) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] PostgreSql numeric resolver ([#5399](sqldelight/sqldelight#5399) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] sqlite windows function ([#2799](sqldelight/sqldelight#2799) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] PostgreSql SELECT DISTINCT ON ([#5345](sqldelight/sqldelight#5345) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] alter table add column if not exists ([#5309](sqldelight/sqldelight#5309) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] Postgresql async bind parameter ([#5313](sqldelight/sqldelight#5313) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] PostgreSql boolean literals ([#5262](sqldelight/sqldelight#5262) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] PostgreSql window functions ([#5155](sqldelight/sqldelight#5155) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] PostgreSql isNull isNotNull types ([#5173](sqldelight/sqldelight#5173) by \[Griffio]\[griffio]) - \[PostgreSQL Dialect] PostgreSql select distinct ([#5172](sqldelight/sqldelight#5172) by \[Griffio]\[griffio]) - \[Paging Extension] paging refresh initial load fix ([#5615](sqldelight/sqldelight#5615) by \[Eva]\[evant]) - \[Paging Extension] Add MacOS native targets ([#5324](sqldelight/sqldelight#5324) by \[Vitor Hugo Schwaab]\[vitorhugods]) - \[IntelliJ Plugin] K2 Support </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Never, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: 2c67f3c06e13d86329db5149abfceba98e3701a4

griffio force-pushed the add-5346-postgresql-unnest branch from 1afcdb6 to fb72bf6 Compare March 19, 2025 19:32

griffio force-pushed the add-5346-postgresql-unnest branch from fb72bf6 to c75e6b3 Compare May 15, 2025 13:03

griffio added 10 commits May 15, 2025 14:40

Grammar Mixins and Resolvers

21dd57a

Add unnest to PostgreSql.bnf Add mixins to help with column resolving Update PostresqResolver to handle converting Array<T> to T Add TableFunctionRowType to allow new concept of Nested type to Row type

Add fixture and Integration tests

6b63a34

TableOrSubqueryMixin

49dc545

Try and support WHERE clause

Update Test.s

0b0c96f

Add where exists

Integration test

f035c04

For where clause

Fixup for LATERAL

514cfbe

Update TableFunctionTableAliasMixin.kt

Update Test.s

10715f3

Add test for LATER UNNEST

Update PostgreSqlTypeResolver.kt

a734504

fix conflicts

Update Unnest.sq

762605d

fix `The Kotlin type of the argument cannot be inferred, use CAST instead` LOWER and UPPER are no longer for TEXT but can be used with ranges so the bind param cannot be inferred

Update PostgreSqlTest.kt

25d7493

griffio force-pushed the add-5346-postgresql-unnest branch from c75e6b3 to 25d7493 Compare May 15, 2025 13:40

griffio marked this pull request as ready for review May 15, 2025 13:44

AlecKazakova approved these changes May 15, 2025

View reviewed changes

AlecKazakova merged commit 17deef2 into sqldelight:master May 15, 2025
11 checks passed

griffio deleted the add-5346-postgresql-unnest branch May 15, 2025 17:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Support PostgreSql UnNest Array to rows #5673

Support PostgreSql UnNest Array to rows #5673

Uh oh!

griffio commented Feb 27, 2025 •

edited

Loading

Uh oh!

AlecKazakova left a comment

Uh oh!

Uh oh!

Angel-Ponce commented May 15, 2025

Uh oh!

griffio commented May 16, 2025 •

edited

Loading

Uh oh!

Uh oh!

Support PostgreSql UnNest Array to rows #5673

Support PostgreSql UnNest Array to rows #5673

Uh oh!

Conversation

griffio commented Feb 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlecKazakova left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Angel-Ponce commented May 15, 2025

Uh oh!

griffio commented May 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

griffio commented Feb 27, 2025 •

edited

Loading

griffio commented May 16, 2025 •

edited

Loading