Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Knex does not transform flat SQL result sets into nested objects #6289

@mercmobily

Description

@mercmobily

Practical action for this issue: update documentation and explain somewhere that knex will never implement returning of nested queries the way an ORM would.

Knex does not transform flat SQL result sets into nested objects

Knex is a SQL query builder, not an ORM (Object-Relational Mapper). One of the most common questions from users is how to get nested/hierarchical data structures from joined queries, where child records are grouped into arrays under their parent records. This issue consolidates all related questions and explains why this is outside Knex's scope.

The Core Issue

When you perform a JOIN query in SQL, the database returns a flat result set with repeated parent data for each child record. For example:

// This query
knex('user_groups')
  .leftJoin('users', 'users.user_group_id', 'user_groups.id')
  .where('user_groups.branch_id', 10)
  .select('*');

Returns a flat structure like:

[
  { id: 1, name: 'Admin', branch_id: 10, user_id: 1, user_name: 'Alice' },
  { id: 1, name: 'Admin', branch_id: 10, user_id: 2, user_name: 'Bob' },
  { id: 1, name: 'Admin', branch_id: 10, user_id: 3, user_name: 'Charlie' }
]

Many users expect or want nested results like:

[
  {
    id: 1,
    name: 'Admin',
    branch_id: 10,
    users: [
      { user_id: 1, user_name: 'Alice' },
      { user_id: 2, user_name: 'Bob' },
      { user_id: 3, user_name: 'Charlie' }
    ]
  }
]

Why Knex Doesn't Do This

As stated by maintainer @tgriesser in #61: "this is a limitation of sql... and not something knex will try to handle for you provide out of the box".

Knex's philosophy is to be a thin, portable SQL query builder that mirrors SQL semantics across different databases. Data transformation and object mapping are explicitly outside its scope. This design decision keeps Knex:

  • Lightweight - No complex data mapping or hydration logic
  • Predictable - What you see in SQL is what you get in JavaScript
  • Portable - Works identically across all supported databases
  • Fast - No transformation overhead
  • Simple - Easier to maintain and reason about

Available Solutions

If you need nested results from joined queries, you have several options:

1. Use an ORM that provides this feature

ORMs like Objection.js (built on Knex) and Bookshelf provide relationship management and nested data fetching:

// Objection.js example
const groups = await UserGroup.query()
  .where('branch_id', 10)
  .withGraphFetched('users');

As mentioned by maintainer @elhigu in #882: "objection.js and bookshelf allow fetching relations as nested data."

2. Use database-specific JSON functions

Modern databases provide native JSON aggregation functions that can build nested structures in SQL:

PostgreSQL:

knex('user_groups')
  .select([
    'user_groups.*',
    knex.raw('COALESCE(json_agg(users.*) FILTER (WHERE users.id IS NOT NULL), \'[]\') as users')
  ])
  .leftJoin('users', 'users.user_group_id', 'user_groups.id')
  .where('user_groups.branch_id', 10)
  .groupBy('user_groups.id');

MySQL (with `nestTables` driver option):

knex('user_groups')
  .leftJoin('users', 'users.user_group_id', 'user_groups.id')
  .where('user_groups.branch_id', 10)
  .options({ nestTables: true }); // MySQL-specific driver option

This returns objects grouped by table name:

[
  {
    user_groups: { id: 1, name: 'Admin', branch_id: 10 },
    users: { user_id: 1, user_name: 'Alice' }
  },
  // ... more rows
]

Note: These approaches are database-specific and require manual post-processing.

3. Manual transformation with JavaScript

Transform the flat results yourself using libraries like lodash or custom logic:

const results = await knex('user_groups')
  .leftJoin('users', 'users.user_group_id', 'user_groups.id')
  .where('user_groups.branch_id', 10)
  .select(
    'user_groups.id',
    'user_groups.name',
    'user_groups.branch_id',
    'users.id as user_id',
    'users.name as user_name'
  );

// Group manually
const grouped = results.reduce((acc, row) => {
  let group = acc.find(g => g.id === row.id);
  if (!group) {
    group = {
      id: row.id,
      name: row.name,
      branch_id: row.branch_id,
      users: []
    };
    acc.push(group);
  }
  if (row.user_id) {
    group.users.push({
      user_id: row.user_id,
      user_name: row.user_name
    });
  }
  return acc;
}, []);

4. Use a dedicated nesting library

Libraries like KnexNest (NestHydrationJS) provide automatic nesting based on column naming conventions:

const KnexNest = require('knex-nest');

const results = await knex('user_groups')
  .leftJoin('users', 'users.user_group_id', 'user_groups.id')
  .where('user_groups.branch_id', 10)
  .select(
    'user_groups.id AS _id',
    'user_groups.name AS _name',
    'user_groups.branch_id AS _branch_id',
    'users.id AS _users__id',
    'users.name AS _users__name'
  );

const nested = KnexNest(results);

5. Make separate queries

Often the cleanest approach is to fetch parent and child records separately:

// Fetch parent records
const groups = await knex('user_groups')
  .where('branch_id', 10)
  .select('*');

// Fetch all related users in one query
const groupIds = groups.map(g => g.id);
const users = await knex('users')
  .whereIn('user_group_id', groupIds)
  .select('*');

// Group users by parent ID
const usersByGroupId = users.reduce((acc, user) => {
  if (!acc[user.user_group_id]) acc[user.user_group_id] = [];
  acc[user.user_group_id].push(user);
  return acc;
}, {});

// Attach users to groups
const result = groups.map(group => ({
  ...group,
  users: usersByGroupId[group.id] || []
}));

This approach:

  • Avoids duplicate parent data in result sets
  • Often performs better than complex joins
  • Makes it easier to cache parent and child data separately
  • Is the pattern used by most ORMs internally

Common Related Issues

Column name collisions: When joining tables with identically named columns (like `id`), the child table values overwrite parent values. Solution: Use explicit column selection with aliases:

knex('posts')
  .leftJoin('comments', 'comments.post_id', 'posts.id')
  .select(
    'posts.id as post_id',
    'posts.name as post_name',
    'comments.id as comment_id',
    'comments.name as comment_name'
  );

Multiple executions: Remember that Knex query builders are "thenable" objects. Awaiting or calling `.then()` multiple times on the same query builder executes the query each time. Store the promise result, not the query builder.

Maintainer Position

This is a deliberate design decision that will not change. From the maintainers:

  • @tgriesser (creator): "this is a limitation of sql... and not something knex will try to handle for you provide out of the box"
  • @elhigu (maintainer): Directed users to ORMs like Objection.js and Bookshelf for this functionality

Knex intentionally remains a query builder, not an ORM. Users needing ORM features should use an ORM built on top of Knex (like Objection.js) or alongside it.

Related Issues (Duplicates)

Exact duplicates asking for nested results from joins:

Related to column collisions in joins:

Related to extending query builder for nested data:

Related ORM/relationship discussions:

Recommendation

If you're reading this because you opened an issue asking how to get nested results: Knex is not the right tool for this job. Use an ORM like Objection.js, use database-specific JSON functions, or transform the data manually. Knex will continue to return flat result sets that mirror standard SQL behaviour.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions