Allow to filter queries by child entity #3184

kamilkisiela · 2022-01-25T09:07:09Z

Given the schema:

type Purpose @entity {
  id: ID!
  sender: Sender!
  purpose: String!
  createdAt: BigInt!
  transactionHash: String!
}

type Sender @entity {
  id: ID!
  address: Bytes!
  purposes: [Purpose!] @derivedFrom(field: "sender")
  createdAt: BigInt!
  purposeCount: BigInt!
}

The sender_ is added to Purpose_filter input object.

input Purpose_filter {
    ...
+  sender_: Sender_filter
}

The Sender_filter input object is untouched.

Example

query {
  purposes(
    where: {
      purpose_contains: "...",
      sender_: {
        id: "..."
      }
    }
  ) {
    id
    sender {
      id
      purposeCount
    }
  }
}

select
  'Purpose' as entity,
  to_jsonb(c.*) as data
from
  (
    select
      c.*
    from
      "sgd1"."purpose" c
    where
      c.block_range @ > $ 1
      and c."purpose" like $ 2
      and (
        exists (
          select
            1
          from
            "sgd1"."sender" as i
          where
            c."sender" = i."id"
            and i."block_range" @ > $ 3
            and i."id" = $ 4
        )
      )
    group by
      c.id
    order by
      c."id"
    limit
      100
  ) c

Child filters are limited to be only 1 level deep, so no Child(Child(...)) and this pull request DOES NOT implement a support for interface types.

Continues #3022

lutter · 2022-03-07T23:57:58Z

Could you rebase this to latest master when you get a chance?

dotansimha · 2022-03-08T15:33:12Z

@lutter done! :)

Note that this PR is only for filtering, for sorting we have this one: #3096 (targeted into this PR, not master, and @kamilkisiela had some questions there before proceeding).

store/postgres/src/relational_queries.rs

graph/src/components/store/mod.rs

lutter

Getting closer! I actually hadn't thought about the issue that forced you to use group by but I think using the exists(select 1 ..) strategy I outlined in the comment will make this easier.

We'll also need tests for all this; the best place for them is probably graphql/tests/query.rs - I recently revamped them to (hopefully) make it easier to add tests there. The tests for this should use child filters that traverse Musician.bands, Musician.written_songs, Band.members, Song.written_by and SongStat.song. For example, what I have in mind for Musician.bands is something like musicians(where: { bands_: { name: "something" } }) { id }

graph/src/components/store/mod.rs

store/postgres/src/relational_queries.rs

graphql/tests/query.rs

graphql/src/schema/api.rs

lutter

This is getting really, really close. There's a few smallish things to touch up, but other than that, this is almost ready.

lutter · 2022-06-17T18:51:01Z

graph/src/components/store/mod.rs

 impl CheapClone for EntityType {}

+#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq, PartialOrd, Ord, Hash)]
+pub struct EntityFilterDerivative(bool);


This would be better as an enum AttributeStorage { Direct, Derived }; even clearer would be to make EntityFilter::Child a struct, i.e.

Child { attr: Attribute, entity_type: EntityType, filter: Box<EntityFilter>, derived: bool, },

graphql/src/schema/api.rs

lutter · 2022-06-17T19:01:30Z

graphql/src/schema/api.rs

+            return Some(input_values);
+        }
+
+        let input_field_type = input_field_type.unwrap();


This, from line 401, would be clearer and avoid an unwrap() like this:

let input_field_type = match input_field_type { None => { let mut input_values: Vec<InputValue> = vec![]; if let Some(parent) = parent_type_name { extend_with_child_filter_input_value(field, &parent, &mut input_values); } return Some(input_values); } Some(input_field_type) => input_field_type, };

Or, even better to do what follows in the match, too and avoid the early return:

let mut input_values: Vec<InputValue> = match input_field_type { None => { vec![] } Some(input_field_type) => vec![ "", "not", "contains", "contains_nocase", "not_contains", "not_contains_nocase", ] .into_iter() .map(|filter_type| { input_value( &field.name, filter_type, Type::ListType(Box::new(Type::NonNullType(Box::new( input_field_type.clone(), )))), ) }) .collect(), }; if let Some(parent) = parent_type_name { extend_with_child_filter_input_value(field, &parent, &mut input_values); } Some(input_values)

This could also be a let mut input_values = input_field_type.map(|input_field_type| ...).unwrap_or(vec![]) but I am fine with either

lutter · 2022-06-17T22:12:26Z

graphql/src/store/query.rs

+        Type::NonNullType(inner) => resolve_type_name(inner),
+        Type::NamedType(name) => name,
+    }
+}


This is get_base_type from graph/src/data/graphql (just do something like use graph::data::graphql::TypeExt as _ to be able to use it)

graphql/tests/query.rs

lutter · 2022-06-17T22:22:51Z

store/postgres/src/relational_queries.rs

+            Child(attr, entity, child_filter, _) => {
+                if child_filter_ancestor {
+                    return Err(StoreError::QueryExecutionError(
+                        "Only a single level sub filter is allowed".to_string(),


How about something like Child filters can not be nested?

lutter · 2022-06-17T22:23:22Z

store/postgres/src/relational_queries.rs

+
+        out.push_sql("exists (select 1");
+
+        out.push_sql(" from ");


Very minor but these two push_sql could be collapsed into one

lutter · 2022-06-17T22:24:07Z

store/postgres/src/relational_queries.rs

+            let parent_column = self.table.primary_key();
+
+            if child_column.is_list() {
+                // Type A: p.id = any(c.{parent_field})


Shouldn't that be c.id = any(i.{parent_field}) ? And similar in the comments for the other cases

lutter · 2022-06-17T22:25:56Z

store/postgres/src/relational_queries.rs

+        out.push_sql(child_prefix);
+        out.push_identifier(BLOCK_RANGE_COLUMN)?;
+        out.push_sql(" @> ");
+        out.push_bind_param::<Integer, _>(&self.block)?;


This needs to use BlockRangeColumn::contains - the way it is written now only works for mutable entities, but would fail for immutable ones since they don't have a block_range column, only a block$ column

lutter · 2022-06-20T22:12:54Z

This is good to go - there's one small thing around how the commits are structured/named: see this for how we usually do that. With this PR, it might be best to just squash everything into a mega commit all: Add child filters

LukvonStrom · 2022-06-24T14:35:05Z

@ALL (@lutter, @kamilkisiela) when is the ETA for merge of this awesome addition to the graphprotocol? :)

kamilkisiela · 2022-06-27T11:29:55Z

@lutter I squashed all commits and renamed it

mhpaler · 2022-08-06T16:47:17Z

@kamilkisiela @lutter and anyone else involved in this PR. Kinda late to the party on this one, but I'd like to offer a HUGE thank you for getting this into the graph-node. It's just such an amazing boost to querying. Thank you!!

dotansimha changed the title ~~Filter by child entity~~ graph, graphql: filter by child entity Jan 26, 2022

dotansimha mentioned this pull request Jan 29, 2022

refactor: prefer GraphQL SDL for base/meta schema #3203

Closed

kamilkisiela force-pushed the kamil-filter-by-child branch from 9afb13b to 46d2872 Compare March 3, 2022 17:02

tilacog self-requested a review March 4, 2022 01:11

dotansimha force-pushed the kamil-filter-by-child branch from 46d2872 to 771a995 Compare March 8, 2022 15:30

dotansimha requested a review from lutter March 8, 2022 16:19

lutter reviewed Mar 9, 2022

View reviewed changes

store/postgres/src/relational_queries.rs Outdated Show resolved Hide resolved

store/postgres/src/relational_queries.rs Outdated Show resolved Hide resolved

store/postgres/src/relational_queries.rs Outdated Show resolved Hide resolved

lutter reviewed Mar 9, 2022

View reviewed changes

store/postgres/src/relational_queries.rs Outdated Show resolved Hide resolved

dotansimha force-pushed the kamil-filter-by-child branch 2 times, most recently from 809b6fd to f34918a Compare March 13, 2022 14:48

kamilkisiela force-pushed the kamil-filter-by-child branch from f34918a to 2a77b5d Compare April 5, 2022 10:51

kamilkisiela force-pushed the kamil-filter-by-child branch 2 times, most recently from 1e9f27b to 5523cf7 Compare April 27, 2022 12:35

dotansimha linked an issue May 2, 2022 that may be closed by this pull request

Filtering on the basis of child entities #2960

Closed

dotansimha force-pushed the kamil-filter-by-child branch from 5523cf7 to 6e04d05 Compare May 3, 2022 06:27

dotansimha changed the title ~~graph, graphql: filter by child entity~~ feat(graph,graphql): filter by child entity May 3, 2022

dotansimha force-pushed the kamil-filter-by-child branch from 6e04d05 to 6f9312f Compare May 3, 2022 06:28

kamilkisiela force-pushed the kamil-filter-by-child branch 3 times, most recently from b863adb to 461a827 Compare May 4, 2022 09:38

dotansimha requested review from azf20, lutter and saihaj May 4, 2022 13:39

kamilkisiela commented May 5, 2022

View reviewed changes

graph/src/components/store/mod.rs Outdated Show resolved Hide resolved

dizzyd assigned dizzyd and lutter and unassigned dizzyd May 9, 2022

lutter reviewed May 10, 2022

View reviewed changes

graph/src/components/store/mod.rs Outdated Show resolved Hide resolved

store/postgres/src/relational_queries.rs Outdated Show resolved Hide resolved

store/postgres/src/relational_queries.rs Outdated Show resolved Hide resolved

kasparkallas mentioned this pull request May 23, 2022

[RFR] create account token snapshot log for each account token snapshot updates superfluid-org/protocol-monorepo#830

Merged

kamilkisiela force-pushed the kamil-filter-by-child branch from 461a827 to 2c30a78 Compare May 23, 2022 12:37

kamilkisiela commented May 24, 2022

View reviewed changes

store/postgres/src/relational_queries.rs Outdated Show resolved Hide resolved

kamilkisiela force-pushed the kamil-filter-by-child branch from f465e1e to 56e515c Compare May 27, 2022 12:13

schmidsi mentioned this pull request May 30, 2022

Is it possible to query with a nested where syntax? graphprotocol/support#61

Closed

kamilkisiela force-pushed the kamil-filter-by-child branch from 56e515c to 21bbe8b Compare June 14, 2022 12:36

kamilkisiela mentioned this pull request Jun 14, 2022

all: GraphQL API Versioning #3185

Merged

1 task

kamilkisiela force-pushed the kamil-filter-by-child branch 3 times, most recently from e6ab8cd to 322beea Compare June 15, 2022 12:22

kamilkisiela commented Jun 15, 2022

View reviewed changes

graphql/tests/query.rs Outdated Show resolved Hide resolved

graphql/src/schema/api.rs Outdated Show resolved Hide resolved

kamilkisiela force-pushed the kamil-filter-by-child branch 2 times, most recently from 2a61917 to bdc00dc Compare June 17, 2022 07:41

kamilkisiela mentioned this pull request Jun 17, 2022

feat(graph,graphql): filter by child entity (interfaces) #3677

Merged

lutter requested changes Jun 17, 2022

View reviewed changes

lutter approved these changes Jun 20, 2022

View reviewed changes

kamilkisiela changed the title ~~feat(graph,graphql): filter by child entity~~ Allow to filter queries by child entity Jun 27, 2022

kamilkisiela force-pushed the kamil-filter-by-child branch from 474af82 to b68d755 Compare June 27, 2022 11:28

all: Add child filters

976dc62

kamilkisiela force-pushed the kamil-filter-by-child branch from b68d755 to 976dc62 Compare June 27, 2022 11:29

tilacog merged commit 4c7cb15 into master Jun 27, 2022

dotansimha mentioned this pull request Jul 10, 2022

Order by nested relation graphprotocol/graph-client#145

Closed

0xdavinchee mentioned this pull request Jul 19, 2022

[SUBGRAPH] Refactor schema since we can filter queries by child entities superfluid-org/protocol-monorepo#974

Closed

kamilkisiela deleted the kamil-filter-by-child branch August 8, 2022 16:27

Allow to filter queries by child entity #3184

Allow to filter queries by child entity #3184

Uh oh!

Conversation

kamilkisiela commented Jan 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Example

Uh oh!

lutter commented Mar 7, 2022

Uh oh!

dotansimha commented Mar 8, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lutter left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lutter left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lutter commented Jun 20, 2022

Uh oh!

LukvonStrom commented Jun 24, 2022

Uh oh!

kamilkisiela commented Jun 27, 2022

Uh oh!

mhpaler commented Aug 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

kamilkisiela commented Jan 25, 2022 •

edited

Loading

mhpaler commented Aug 6, 2022 •

edited

Loading