Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Introduction of ijson to reduce memory consumption #563

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Dec 9, 2021
Merged

Introduction of ijson to reduce memory consumption #563

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
f3da896
initial ijson commit
gkorland Dec 6, 2021
932ec7e
Code arrangement
Dec 7, 2021
edd93d4
Implement lasts unimplimented manager API for IValue
Dec 7, 2021
6781c41
fmt fixes
Dec 7, 2021
8e48af7
return jsonpath library to point to generic_json_path
Dec 8, 2021
1aba8a9
Made backend configurable on start using JSON_BACKEND module argument…
Dec 8, 2021
055b85a
run tests on both backends
Dec 8, 2021
cf6e5a5
json_init -> json_init_config
Dec 8, 2021
35331a9
Merge branch 'master' into gkorland-ijson
Dec 8, 2021
1fc8249
fmt fixes
Dec 8, 2021
6449501
tests fixes
Dec 8, 2021
c09bf21
fmt fixes
Dec 8, 2021
4d667d6
disable memory test as its currently incorrect
Dec 8, 2021
59d3cfe
review fixes
Dec 9, 2021
c2b831e
update redismodule-rs to 0.25
Dec 9, 2021
c5314cc
fix 6.0 tests
Dec 9, 2021
a7b76e3
Skip short read test on 6.0 (too slow)
oshadmi Dec 9, 2021
a338d20
Skip module keyspace notification test (introduced in redis 6.2)
oshadmi Dec 9, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
193 changes: 104 additions & 89 deletions Cargo.lock
View file
Open in desktop

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,12 @@ name = "rejson"
[dependencies]
log = "0.4"
bson = "0.14"
ijson = "0.1.3"
serde_json = "1.0"
serde = "1.0"
libc = "0.2"
jsonpath_lib = { git = "https://github.com/RedisJSON/jsonpath.git", branch = "generic_json_path" }
redis-module = { version="0.24", features = ["experimental-api"]}
redis-module = { version="0.25", features = ["experimental-api"]}
itertools = "0.10.1"
[features]
# Workaround to allow cfg(feature = "test") in redismodue-rs dependencies:
Expand Down
28 changes: 1 addition & 27 deletions src/c_api.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,10 @@ use std::{
use crate::commands::KeyValue;
use jsonpath_lib::select::select_value::{SelectValue, SelectValueType};
use jsonpath_lib::select::Selector;
use redis_module::{raw as rawmod, RedisError};
use redis_module::raw as rawmod;
use redis_module::{Context, RedisString, Status};
use serde_json::Value;

use crate::manager::{Manager, ReadHolder};
use crate::redisjson::RedisJSON;

// extern crate readies_wd40;
// use crate::readies_wd40::{BB, _BB, getenv};
Expand Down Expand Up @@ -172,30 +170,6 @@ pub fn json_api_get_boolean<M: Manager>(_: M, json: *const c_void, val: *mut c_i

//---------------------------------------------------------------------------------------------

pub fn value_from_index(value: &Value, index: size_t) -> Result<&Value, RedisError> {
match value {
Value::Array(ref vec) => {
if index < vec.len() {
Ok(vec.get(index).unwrap())
} else {
Err(RedisError::Str("JSON index is out of range"))
}
}
Value::Object(ref map) => {
if index < map.len() {
Ok(map.iter().nth(index).unwrap().1)
} else {
Err(RedisError::Str("JSON index is out of range"))
}
}
_ => Err(RedisError::Str("Not a JSON Array or Object")),
}
}

pub fn get_type_and_size(value: &Value) -> (JSONType, size_t) {
RedisJSON::get_type_and_size(value)
}

#[allow(clippy::not_unsafe_ptr_arg_deref)]
pub fn set_string(from_str: &str, str: *mut *const c_char, len: *mut size_t) -> c_int {
if !str.is_null() {
Expand Down
Loading