Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@rephus
Copy link
Collaborator

@rephus rephus commented May 21, 2025

Closes: #14809

Description:

image

image

This PR introduces the concept of "default filters" as an alternative to "required filters" with the ability to mark them as optional.

Key changes:

  • Added support for default_filters in model metadata (alias for required_filters)
  • Added a required: false flag for filters that should be suggested but not enforced
  • Modified the backend to only enforce filters that don't have required: false
  • Updated the frontend to apply default filters when creating a new chart
  • Added tests for the new filter parsing functionality

This allows models to suggest default filters to users without forcing them to be applied, improving the user experience when creating new charts.

Example usage in YAML:

meta: 
  default_filters: 
    - event: 'lightdash_server_saved_chart_created'
      required: false 
Screencast.from.2025-05-20.17-14-20.webm

Reviewer actions

  • I have manually tested the changes in the preview environment
  • I have reviewed the code
  • I understand that "request changes" will block this PR from merging

@owlas owlas requested a deployment to 05-21-feat_default_filters - jaffle_db_pg_13 PR #14898 May 21, 2025 07:13 — with Render Abandoned
@owlas owlas deployed to 05-21-feat_default_filters - headless-browser PR #14898 May 21, 2025 07:13 — with Render Active
Copy link
Collaborator Author

rephus commented May 21, 2025

This stack of pull requests is managed by Graphite. Learn more about stacking.

export type RequiredFilter = {
[key: string]: AnyType;
} & {
required?: boolean;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since we can accept all different values within [key: string] , required can't be used like this:

{
    [key: string]: AnyType;
    required?: boolean;
}

There is a small edge case, which is that you can't have a required: false filter on a required named dimension... A workaround would be to implement an alias (like req) but I'd wait until this becomes an issue...

@rephus rephus force-pushed the 05-21-feat_default_filters branch from 92b48e8 to 3f813e1 Compare May 21, 2025 08:42
@owlas owlas deployed to 05-21-feat_default_filters - headless-browser PR #14898 May 21, 2025 08:42 — with Render Active
@owlas owlas deployed to 05-21-feat_default_filters - headless-browser PR #14898 May 21, 2025 11:55 — with Render Active
@rephus rephus requested review from magnew and removed request for joaoviana May 21, 2025 12:26
@owlas owlas deployed to 05-21-feat_default_filters - headless-browser PR #14898 May 22, 2025 06:24 — with Render Active
M  packages/common/src/types/filterGrammar.test.ts
M  packages/common/src/types/filterGrammar.ts
M  packages/common/src/utils/filters.ts
M  packages/frontend/src/components/Explorer/FiltersCard/FiltersCard.tsx
M  packages/frontend/src/components/common/Filters/index.tsx
@owlas owlas deployed to 05-21-feat_default_filters - headless-browser PR #14898 May 22, 2025 06:47 — with Render Active
@owlas owlas temporarily deployed to 05-21-feat_default_filters - lightdash PR #14898 May 22, 2025 06:47 — with Render Destroyed
@owlas owlas deployed to 05-21-feat_default_filters - headless-browser PR #14898 May 22, 2025 06:48 — with Render Active
Copy link
Contributor

@magnew magnew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I put a question that I think I see the answer to.

But I left it just to ask: should we should some UI indication of the required filters?I guess it could be construed as a data-security problem to show that data is being silently filtered. But it could also be a bit confusing not to mention it.

meta:
default_filters:
- timestamp: 'inThePast 5 years'
required: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this timestamp one operate? Does it just basically work behind the scenes with no UI? I see the other one in the UI. Do we not expect to see any indication of this to users or am I just missing it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should also appear, but perhaps the dbt didn't refresh .
image
This one is also enforced in the backend

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, cool. Yup, I see it now!

if (data && data.tables[tableName]) {
const requiredFilters = data.tables[tableName].requiredFilters;
// We only force required filters that are not explicitly set to false
// requiredFilters with required:false will be added on the UI, but not enforced on the backend
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this answers my other question

M  packages/common/src/types/filterGrammar.test.ts
M  packages/common/src/types/filterGrammar.ts
@owlas owlas deployed to 05-21-feat_default_filters - headless-browser PR #14898 May 23, 2025 07:28 — with Render Active
M  packages/common/src/types/filterGrammar.ts
@owlas owlas deployed to 05-21-feat_default_filters - headless-browser PR #14898 May 23, 2025 08:29 — with Render Active
@owlas owlas deployed to 05-21-feat_default_filters - headless-browser PR #14898 May 23, 2025 09:11 — with Render Active
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
@owlas owlas temporarily deployed to 05-21-feat_default_filters - headless-browser PR #14898 May 23, 2025 09:29 — with Render Destroyed
@rephus rephus requested a review from almeidabbm May 23, 2025 13:59
@rephus
Copy link
Collaborator Author

rephus commented May 23, 2025

API tests are failing because I modified content in jaffle shop

@rephus
Copy link
Collaborator Author

rephus commented May 23, 2025

Waiting until @almeidabbm confirms this is compatible with his filters with joins feature.

@almeidabbm
Copy link
Contributor

This allows models to suggest default filters to users without forcing them to be appli

All is working @rephus

image

Including default_filters on joined models!
image

Only thing I noticed is that when you refresh dbt the UI doesn't update right away, we need to manually reload the page.
With required filters when I update then, the UI updates as well after DBT refresh

@rephus rephus merged commit 5854caa into main May 26, 2025
43 of 49 checks passed
@rephus rephus deleted the 05-21-feat_default_filters branch May 26, 2025 07:56
lightdash-bot pushed a commit that referenced this pull request May 26, 2025
# [0.1651.0](0.1650.0...0.1651.0) (2025-05-26)

### Bug Fixes

* filtering latest dashboard version on validation for charts within dashboard ([#15000](#15000)) ([6edf349](6edf349))

### Features

* default filters ([#14898](#14898)) ([5854caa](5854caa))
@lightdash-bot
Copy link
Collaborator

🎉 This PR is included in version 0.1651.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

I want to supply 'recommended' filters in my explore, beside 'required' ones

6 participants