feat: pivot table row grouping and subtotals #10269

agha4to · 2024-05-30T23:00:33Z

Description:

Adds row grouping and subtotal functionality to Pivot Tables. Pivot Table and non-Pivot Table components have distinct implementations, and ideally we'd like to combine them. This code does not do that. Instead, it keeps them separate. The underlying Pivot Table data structures have been "retrofitted" rather than rewritten at this time. I think that task is better done in the future when combining the two components, to avoid too much wasted effort.

Reviewer actions

I have manually tested the changes in the preview environment
I have reviewed the code
I understand that "request changes" will block this PR from merging

netlify · 2024-05-30T23:00:37Z

👷 Deploy request for peaceful-bassi-cbf284 pending review.

Visit the deploys page to approve it

Name	Link
🔨 Latest commit	`175ae95`

packages/frontend/src/components/common/PivotTable/index.tsx

ZeRego

I would appreciate if you rebase this PR after the updates 🙏 🙇

agha4to · 2024-07-11T18:43:03Z

@ZeRego, thank you for your patience. Refactored per your recommendations and fixed some tests. Thanks again!

almeidabbm · 2024-07-12T15:42:52Z

Hi @agha4to 👋

I have been testing this and stumbled across a small bug:
I cannot change the order of the row dimensions. Don't think this is related to this PR itself, but allowing to do it would help with displaying the correct row grouping

Screen.Recording.2024-07-12.at.16.40.35.mov

agha4to · 2024-07-15T22:16:25Z

Hi @almeidabbm, I've been thinking about Rows order and trying to implement something but I'm coming up against a few design questions.

Currently in Pivots, even without this PR:

From the Results table, dragging columns reorders them in both the Results and Chart view
From the pivot Configure chart sidebar, reordering items under Columns reorders the pivoted items at the top of the Chart

What's proposed:

From the pivot Configure chart sidebar, reordering items under Rows should reorder columns in the Chart (It's somewhat confusing that reordering Rows would change column order, but it is standard for pivot tables.)

Questions:

In Configure chart, should Rows drags reflect in the Chart table only or also Results? A couple possibilities:

A) Change only Chart because 1) it is after all the Chart config and 2) if dragging Rows changed Results, how would dragging Columns be handled? If we go this route, should drags of Results columns override the Configure chart Rows order and change Chart?
B) Change both Chart and Results because it is confusing if the column orders differ between the Results and Charts. If so, again, how would dragging Columns in Chart config be handled to change Results?

What about Metrics? Shouldn't those columns also be draggable to reorder the Chart columns?

I think we need a design spec for all of this. There may be other loose ends to consider too.

packages/frontend/src/components/common/PivotTable/index.tsx

packages/frontend/src/hooks/pivotTable/pivotQueryResults.ts

almeidabbm

Couple things causing build errors related to types, otherwise types look good

packages/frontend/src/components/common/PivotTable/index.tsx

IrakliJani · 2024-09-25T11:26:01Z

render link: https://lightdash-pr-11681.onrender.com

agha4to · 2024-09-26T05:42:32Z

I heard y'all were busy. I don't think there's a rush--no problem. Afaik, no remaining issues. Here's a look at the recent shading changes:

Last row of Row # column is no longer shaded
Last row of dimension columns are no longer shaded
Last row of metric columns that do not have column totals are no longer shaded
Row total columns, not including last row, are shaded

Co-authored-by: Irakli Janiashvili <[email protected]>

fix: avoid unsafely inferred `finalHeaderInfoForColumns: any`

IrakliJani · 2024-09-27T09:46:31Z

@agha4to approving code changes ✅ great work 🙌

# [0.1281.0](0.1280.0...0.1281.0) (2024-09-27) ### Features * pivot table row grouping and subtotals ([#10269](#10269)) ([25d4be8](25d4be8))

lightdash-bot · 2024-09-27T17:02:33Z

🎉 This PR is included in version 0.1281.0 🎉

The release is available on:

0.1281.0
GitHub release

Your semantic-release bot 📦🚀

* feat: pivot table row grouping and subtotals * refactor: move pivot subtotal data structure retrofit * test: pivot query result test fixes * refactor: remove use of any in pivot data structure * fix: table drilldown for metrics as rows, build issues * fix: use null marker for subtotals * fix: font settings, spacing, and group component wrapping * Update packages/frontend/src/hooks/pivotTable/pivotQueryResults.ts Co-authored-by: Irakli Janiashvili <[email protected]> * fix: whitespace * test: update pivot table data model tests * chore: avoid mutating code fix: avoid unsafely inferred `finalHeaderInfoForColumns: any` * fix: row number col misalignment and checkbox state * feat: pivot cell shading tweaks * style: fix LightTable whitespace --------- Co-authored-by: Tony Orciuoli <[email protected]> Co-authored-by: Irakli Janiashvili <[email protected]>

# [0.1281.0](lightdash/lightdash@0.1280.0...0.1281.0) (2024-09-27) ### Features * pivot table row grouping and subtotals ([lightdash#10269](lightdash#10269)) ([25d4be8](lightdash@25d4be8))

agha4to changed the title ~~Pivot Table row grouping and subtotals~~ feat: pivot table row grouping and subtotals May 30, 2024

agha4to force-pushed the pivot-table-grouping-and-subtotals branch from 7b0b16e to f3b0b3d Compare May 30, 2024 23:07

ZeRego self-assigned this Jun 13, 2024

ZeRego marked this pull request as ready for review June 13, 2024 09:07

ZeRego reviewed Jun 14, 2024

View reviewed changes

packages/frontend/src/components/common/PivotTable/index.tsx Outdated Show resolved Hide resolved

ZeRego reviewed Jun 14, 2024

View reviewed changes

packages/frontend/src/components/common/PivotTable/index.tsx Outdated Show resolved Hide resolved

ZeRego reviewed Jun 14, 2024

View reviewed changes

ZeRego marked this pull request as draft June 24, 2024 09:38

agha4to force-pushed the pivot-table-grouping-and-subtotals branch from bc75347 to f0245ad Compare June 27, 2024 21:21

agha4to marked this pull request as ready for review June 27, 2024 21:28

12ian34 requested a review from ZeRego July 10, 2024 11:09

agha4to force-pushed the pivot-table-grouping-and-subtotals branch from 499d787 to ffd44f6 Compare July 11, 2024 18:34

rephus requested review from almeidabbm and removed request for ZeRego July 12, 2024 11:35

ZeRego assigned almeidabbm Jul 15, 2024

almeidabbm requested changes Jul 17, 2024

View reviewed changes

packages/frontend/src/components/common/PivotTable/index.tsx Outdated Show resolved Hide resolved

packages/frontend/src/hooks/pivotTable/pivotQueryResults.ts Outdated Show resolved Hide resolved

This comment was marked as resolved.

Sign in to view

agha4to requested a review from almeidabbm July 19, 2024 18:50

owlas requested a deployment to duplicate_pivot-table-grouping-and-subtotals - jaffle_db_pg_13 PR #10800 July 23, 2024 09:30 — with Render Abandoned

owlas deployed to duplicate_pivot-table-grouping-and-subtotals - headless-browser PR #10800 July 23, 2024 09:30 — with Render Active

owlas temporarily deployed to duplicate_pivot-table-grouping-and-subtotals - lightdash PR #10800 July 23, 2024 09:31 — with Render Destroyed

almeidabbm reviewed Jul 23, 2024

View reviewed changes

packages/frontend/src/components/common/PivotTable/index.tsx Outdated Show resolved Hide resolved

packages/frontend/src/components/common/PivotTable/index.tsx Outdated Show resolved Hide resolved

packages/frontend/src/components/common/PivotTable/index.tsx Outdated Show resolved Hide resolved

rephus requested a review from IrakliJani July 24, 2024 10:35

ZeRego assigned IrakliJani and unassigned ZeRego Jul 24, 2024

owlas temporarily deployed to duplicate_pivot-table-grouping-and-subtotals - headless-browser PR #10800 July 26, 2024 17:09 — with Render Destroyed

Tony Orciuoli and others added 13 commits September 25, 2024 23:06

feat: pivot table row grouping and subtotals

d85f47b

refactor: move pivot subtotal data structure retrofit

6c82c3b

test: pivot query result test fixes

17fed1d

refactor: remove use of any in pivot data structure

3ec9830

fix: table drilldown for metrics as rows, build issues

cb53b8c

fix: use null marker for subtotals

1fdc6e8

fix: font settings, spacing, and group component wrapping

181402a

Update packages/frontend/src/hooks/pivotTable/pivotQueryResults.ts

7d64221

Co-authored-by: Irakli Janiashvili <[email protected]>

fix: whitespace

4f21fcc

test: update pivot table data model tests

b05d4dc

chore: avoid mutating code

0995dda

fix: avoid unsafely inferred `finalHeaderInfoForColumns: any`

fix: row number col misalignment and checkbox state

9277a54

feat: pivot cell shading tweaks

3f715c1

agha4to force-pushed the pivot-table-grouping-and-subtotals branch from 18cb3c5 to 3f715c1 Compare September 26, 2024 06:09

style: fix LightTable whitespace

175ae95

owlas temporarily deployed to pivot-table-grouping-and-subtotals - lightdash PR #11681 September 27, 2024 09:44 — with Render Destroyed

owlas temporarily deployed to pivot-table-grouping-and-subtotals - headless-browser PR #11681 September 27, 2024 09:44 — with Render Destroyed

IrakliJani approved these changes Sep 27, 2024

View reviewed changes

agha4to requested a review from almeidabbm September 27, 2024 15:28

almeidabbm approved these changes Sep 27, 2024

View reviewed changes

IrakliJani merged commit 25d4be8 into lightdash:main Sep 27, 2024
54 of 57 checks passed

lightdash-bot pushed a commit that referenced this pull request Sep 27, 2024

chore(release): 0.1281.0

2e4cca5

# [0.1281.0](0.1280.0...0.1281.0) (2024-09-27) ### Features * pivot table row grouping and subtotals ([#10269](#10269)) ([25d4be8](25d4be8))

lightdash-bot added the released label Sep 27, 2024

agha4to deleted the pivot-table-grouping-and-subtotals branch November 13, 2024 18:32

feat: pivot table row grouping and subtotals #10269

feat: pivot table row grouping and subtotals #10269

Uh oh!

Conversation

agha4to commented May 30, 2024

Description:

Reviewer actions

Uh oh!

netlify bot commented May 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

👷 Deploy request for peaceful-bassi-cbf284 pending review.

Uh oh!

Uh oh!

Uh oh!

ZeRego left a comment

Choose a reason for hiding this comment

Uh oh!

agha4to commented Jul 11, 2024

Uh oh!

almeidabbm commented Jul 12, 2024

Uh oh!

agha4to commented Jul 15, 2024

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

almeidabbm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

IrakliJani commented Sep 25, 2024

Uh oh!

agha4to commented Sep 26, 2024

Uh oh!

IrakliJani commented Sep 27, 2024

Uh oh!

Uh oh!

lightdash-bot commented Sep 27, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

netlify bot commented May 30, 2024 •

edited

Loading