Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat: support filtering, sorting & pagination when loading CoValues#2946

Draft
nrainhart wants to merge 59 commits into
mainfrom
feat/colist-query-api
Draft

feat: support filtering, sorting & pagination when loading CoValues#2946
nrainhart wants to merge 59 commits into
mainfrom
feat/colist-query-api

Conversation

@nrainhart
Copy link
Copy Markdown
Contributor

Description

We want to extend the resolve query API to allow filtering, sorting and paginating CoList elements:

const Ranking = co.list(co.map({
  name: z.string(),
  score: z.number(),
  date: z.date(),
}));

Ranking.load(ranking.id, {
  resolve: {
    $where: {
      'name': 'Darts',
    },
    $orderBy: { 'score': 'desc' },
    $limit: 20,
    $offset: 40,
});

This will provide a simpler way to perform these operations and pave the way for implementing indexes. For now, we're loading all the CoList elements and performing the filter/sort/pagination operations in memory, but longer term the goal is to use indexes to avoid fetching the CoList elements.

There are some pending things to do before merging this PR:

  • add support for filtering, sorting & pagination in Svelte & Vue
  • migrate one or more examples to use the new APIs
  • write docs

But this PR can already be reviewed.

Key design decisions

  • Extended resolve query API with new options:
    • $where for filtering CoLists elements
      • Comparison operators: $eq, $ne, $gt, $gte, $lt, $lte
      • Logical operators: $and, $or, $not
    • $orderBy for sorting
    • $limit & offset for pagination
  • CoLists now have an optional query view. The query view gives us a way to decouple the elements from a jazz-tools CoList from the elements in the underlying RawCoList. This means users will see only the elements returned by their resolve query, but will still be able to perform operations on the CoList that will be mapped to the underlying RawCoList.
    • The logic for creating and invalidating the query view is pretty naive, but has worked well enough on my tests. We can go with this simple implementation for now and optimize it later if needed (which may not be necessary if we end up using indexes most of the time).
  • In order for pagination to work, I modified useCoState to re-create the subscription scope if the resolve query is updated

Tests

  • Tests have been added and/or updated

Checklist

  • I've updated the part of the docs that are affected the PR changes
  • I've generated a changeset, if a version bump is required
  • I've updated the jsDoc comments to the public APIs I've modified, or added them when missing

The goal is to store a single index-related reference in the CoValue's history
@nrainhart nrainhart requested review from aeplay and gdorsi September 22, 2025 15:50
@vercel
Copy link
Copy Markdown

vercel Bot commented Sep 22, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
clerk-demo Ready Ready Preview Comment Sep 23, 2025 5:45pm
design-system Ready Ready Preview Comment Sep 23, 2025 5:45pm
file-upload-demo Ready Ready Preview Comment Sep 23, 2025 5:45pm
form-demo Ready Ready Preview Comment Sep 23, 2025 5:45pm
gcmp-homepage Ready Ready Preview Comment Sep 23, 2025 5:45pm
image-upload-demo Ready Ready Preview Comment Sep 23, 2025 5:45pm
jazz-chat Ready Ready Preview Comment Sep 23, 2025 5:45pm
jazz-chat-1 Ready Ready Preview Comment Sep 23, 2025 5:45pm
jazz-chat-2 Ready Ready Preview Comment Sep 23, 2025 5:45pm
jazz-filestream Ready Ready Preview Comment Sep 23, 2025 5:45pm
jazz-image-upload Ready Ready Preview Comment Sep 23, 2025 5:45pm
jazz-inspector Ready Ready Preview Comment Sep 23, 2025 5:45pm
jazz-multi-cursors Ready Ready Preview Comment Sep 23, 2025 5:45pm
jazz-organization Ready Ready Preview Comment Sep 23, 2025 5:45pm
jazz-paper-scissors Ready Ready Preview Comment Sep 23, 2025 5:45pm
jazz-richtext Ready Ready Preview Comment Sep 23, 2025 5:45pm
jazz-richtext-prosekit Ready Ready Preview Comment Sep 23, 2025 5:45pm
jazz-richtext-tiptap Ready Ready Preview Comment Sep 23, 2025 5:45pm
jazz-todo Ready Ready Preview Comment Sep 23, 2025 5:45pm
jazz-version-history Ready Ready Preview Comment Sep 23, 2025 5:45pm
music-demo Ready Ready Preview Comment Sep 23, 2025 5:45pm
passkey-demo Ready Ready Preview Comment Sep 23, 2025 5:45pm
passphrase-auth-demo Ready Ready Preview Comment Sep 23, 2025 5:45pm
quint-ui Ready Ready Preview Comment Sep 23, 2025 5:45pm
reactions-demo Ready Ready Preview Comment Sep 23, 2025 5:45pm
1 Skipped Deployment
Project Deployment Preview Comments Updated (UTC)
jazz-homepage Ignored Ignored Preview Sep 23, 2025 5:45pm

Comment on lines -131 to -133
private constructor(
init: { header: CoValueHeader } | { id: RawCoID },
node: LocalNode,
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're never providing a header, so I removed that option

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, nice.

cursor[bot]

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

Comment on lines -131 to -133
private constructor(
init: { header: CoValueHeader } | { id: RawCoID },
node: LocalNode,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, nice.

subscription.branchOwnerId !==
options?.unstable_branch?.owner?.$jazz.id ||
(subscription.subscription &&
subscription.subscription.resolve !== resolveQuery)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: why are we relying on useStructuralEquals vs calling structuralEquals here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This useLayoutEffect is re-subscribing to the context manager each time it runs, so I wanted to detect the change in the resolve query earlier. I see there's still an integration test failing due to an infinite render loop, so I need to look into this some more

Comment thread packages/jazz-tools/src/tools/subscribe/SubscriptionScope.ts Outdated
// Need all these checks because the migration can trigger new syncronous updates
if (queryFields.length > 0 && value !== "unavailable") {
this.requestCoListChildrenLoad();
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it ok for ths to run also on non-coList values?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heh, $where and $orderBy can only be used with CoLists that contain CoMaps right now. I'll work to extend it to CoLists with plain JSON objects as well

Copy link
Copy Markdown
Collaborator

@gdorsi gdorsi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when a value that doesn't match the filter criteria is updated?

cursor[bot]

This comment was marked as outdated.

@nrainhart
Copy link
Copy Markdown
Contributor Author

What happens when a value that doesn't match the filter criteria is updated?

Right now the subscription update will still be triggered. I'll modify this to prevent the update from happening if the query view didn't change.

@gdorsi gdorsi marked this pull request as draft October 27, 2025 23:47
@gdorsi gdorsi mentioned this pull request Mar 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants