Effects #2826
Merged
Effects #2826
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dd92c2b to
a8c0587
Compare
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## master #2826 +/- ##
==========================================
+ Coverage 90.36% 90.48% +0.11%
==========================================
Files 283 287 +4
Lines 28072 28747 +675
==========================================
+ Hits 25367 26011 +644
- Misses 2705 2736 +31
... and 5 files with indirect coverage changes Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report in Codecov by Sentry. |
AviAvni
suggested changes
Feb 8, 2023
src/commands/cmd_query.c
Outdated
| UndoLog *undolog = QueryCtx_GetUndoLog(); | ||
| uint n = UndoLog_Length(*undolog); | ||
|
|
||
| if(n > 2048 || n == 0) { |
src/commands/cmd_query.c
Outdated
|
|
||
| if(n > 2048 || n == 0) { | ||
| // either no changes or too many changes, do not use effects | ||
| return false; |
There was a problem hiding this comment.
we don't check if there is non-deterministic change
src/configuration/config.h
Outdated
| Config_VKEY_MAX_ENTITY_COUNT, | ||
| Config_DELTA_MAX_PENDING_CHANGES | ||
| Config_DELTA_MAX_PENDING_CHANGES, | ||
| Config_REPLICATE_EFFECTS |
There was a problem hiding this comment.
are you sure we want it to changed during runtime?
did you test what happen if you change this during traffic?
also need to test that we can load aof file mixed with effect and queries
src/graph/graph_hub.c
Outdated
| bool has_indecise = GraphContext_HasIndices(gc); | ||
| UndoLog *undo_log = (log == true) ? QueryCtx_GetUndoLog() : NULL; | ||
|
|
||
| for (uint i = 0; i < n; i++) { |
There was a problem hiding this comment.
maybe
if(log == true) {
for (uint i = 0; i < n; i++) {
// add each deleted edge to undo-log
UndoLog_DeleteEdge(undo_log, edges + i);
}
}
if(has_indecise == true) {
for (uint i = 0; i < n; i++) {
// remove edge from relevant indices
_DeleteEdgeFromIndices(gc, edges + i);
}
}
There was a problem hiding this comment.
I don't want to perform 2N operations
|
we need to add summary of the changes and then update matrix size so the matrix policy will be not or only resize when applying effect |
a21b477 to
db97417
Compare
LiorKogan
reviewed
Apr 6, 2023
| total execution time / number of changes: 5ms / 5 = 1ms. | ||
| if the average modification time is greater then `EFFECTS_THRESHOLD` the query | ||
| will be replicated to both replicas and AOF as a graph effect otherwise the original | ||
| query will be replicated. |
There was a problem hiding this comment.
Suggestion: Add:
0: always replicate the effect; -1: always replicate the original query.
There was a problem hiding this comment.
Please add:
(Since RedisGraph v2.12)
AviAvni
pushed a commit
that referenced
this pull request
Apr 30, 2023
* update with master * wait for replica * wait after each query * refactor test * effect configuration * wip * determine when to use effects * remove stats * skip replica config * adjust constraint test * use readonly query whenever possible * debuging * pass array and len to update labels * set edge endpoints * mostly repositioning * remove debugging prints
AviAvni
added a commit
that referenced
this pull request
Apr 30, 2023
* Effects (#2826) * update with master * wait for replica * wait after each query * refactor test * effect configuration * wip * determine when to use effects * remove stats * skip replica config * adjust constraint test * use readonly query whenever possible * debuging * pass array and len to update labels * set edge endpoints * mostly repositioning * remove debugging prints * Update graph.query.md (#3029) BFS: 3 arguments instead of 4 * Clear errno before strtol() (#3021) Co-authored-by: Roi Lipman <[email protected]> * Validate with fix (#3018) * wip * fix validations * touchup * no if branch when children are not visited * migrate and add tests, fix RETURN validation * doc addition --------- Co-authored-by: Roi Lipman <[email protected]> * Refactor execution-plan freeing mechanism (#2946) * Refactor execution-plan freeing mechanism Co-authored-by: razmon <[email protected]> * use rax to store labels * fix comments * add comment * addressing review comments * addressed review comments * fix * leak fix * free ops deepest first * addressing review comments * Update execution_plan.c remove dict init * Update execution_plan.c include setjmp * touchup * touchup --------- Co-authored-by: Ofir Moskovich <[email protected]> Co-authored-by: Avi Avni <[email protected]> Co-authored-by: Roi Lipman <[email protected]> * Fix optimize label scan (2) (#3034) * introduce swap candidates to Label-Scan * doc-fix * fix * tests fix * Update test_optimizations_plan.py * Update scan_functions.h * move context to heap * fix leak * for index scan as well * addressing review comments * touchup * Update scan_functions.c * Update scan_functions.c --------- Co-authored-by: Roi Lipman <[email protected]> * Fix LockWrite crash (#3039) * JoinConsume(): Propagaget reset if stream depleted * WIP: Test FOREACH * fix foreach crash * fix tests * fix --------- Co-authored-by: Avi Avni <[email protected]> * Fix for Xenial (OpenMP 4.0/4.5) (#3017) * Fix for Xenial (OpenMP 4.5) * fixes 1 * fixes 2 * fixes 3 * fixes 4 * fixes 5 * fixes 6 * fixes 7 --------- Co-authored-by: Avi Avni <[email protected]> * accumulate updates (#3005) * accumulate updates * fix leaks * add test * unify default dictType definitions * fix * fix 2 * add test * wip fix * wip fix * fix crash * add effect log * early review * fix build * fix test * fixes * fix * refactoring in progress * refactoring continue * write effects directly into buffer * effects-buffer * no need to clone value * no need to clone string * address review comments * address review * fix * address review comment * address reivew * fix perf issue * fix build * refactoring * fix perf * fix build * fix * fix leak * remove redundant flags * compute change against new attribute-set * fix perf * address review * fix * fix * adding comments --------- Co-authored-by: razmon <[email protected]> Co-authored-by: Raz Monsonego <[email protected]> Co-authored-by: swilly22 <[email protected]> Co-authored-by: Roi Lipman <[email protected]> * Fix rewrite `DELETE` clauses (#3067) * fix and test * add example --------- Co-authored-by: Avi Avni <[email protected]> * Reattach on reset (#3070) * formatting * reattach iterator on reset * disable graphblas memory pool (#3068) * bump version 2.12.1 --------- Co-authored-by: Roi Lipman <[email protected]> Co-authored-by: Lior Kogan <[email protected]> Co-authored-by: nafraf <[email protected]> Co-authored-by: Raz Monsonego <[email protected]> Co-authored-by: Ofir Moskovich <[email protected]> Co-authored-by: Rafi Einstein <[email protected]> Co-authored-by: razmon <[email protected]> Co-authored-by: swilly22 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Support for GRAPH.EFFECT replicating modifications to both replicas and AOF