Key/Value Datastore for Persistent Memory
This is experimental pre-release software and should not be used in production systems. APIs and file formats may change at any time without preserving backwards compatibility. All known issues and limitations are logged as GitHub issues.
pmemkv is a local/embedded key-value datastore optimized for persistent memory.
Rather than being tied to a single language or backing implementation, pmemkv
provides different options for storage engines and language bindings.
pmemkv provides multiple storage engines with vastly different implementations. Since all
engines conform to the same common API, any engine can be used with common pmemkv utilities
and language bindings. Engines are requested at runtime by name.
Contributing a new engine
is easy and encouraged!
| Engine | Description | Thread-Safe? |
|---|---|---|
| kvtree2 (default) | Hybrid B+ persistent tree (latest version) | No |
| kvtree | Hybrid B+ persistent tree (2017 version) | No |
| blackhole | Accepts everything, returns nothing | Yes |
pmemkv is written in C and C++. Developers can either use native C++ classes directly, or use our extern "C" API, or use one of several high-level language bindings that are based on the extern "C" API.
#include <libpmemkv.h>
#include <string.h>
#include <assert.h>
using namespace pmemkv;
int main() {
// open the datastore
KVEngine* kv = new KVEngine("kvtree2", "/dev/shm/mykv", 8388608); // 8 MB
// put new key
KVStatus s = kv->Put("key1", "value1");
assert(s == OK);
// read last key back
string value;
s = kv->Get("key1", &value);
assert(s == OK && value == "value1");
// close the datastore
delete kv;
return 0;
}#include <libpmemkv.h>
#include <string.h>
#include <assert.h>
int main() {
#define VAL_LEN 64
char value[VAL_LEN];
/* open the datastore */
KVEngine* kv = kvengine_open("kvtree2", "/dev/shm/mykv", 8388608);
/* put new key */
int32_t len = strlen("value1");
KVStatus s = kvengine_put(kv, "key1", "value1", &len);
assert(s == OK);
/* read last key back */
len = VAL_LEN;
s = kvengine_get(kv, "key1", VAL_LEN, value, &len);
assert(s == OK && !strcmp(value, "value1"));
/* close the datastore */
kvengine_close(kv);
return 0;
}These bindings are maintained in separate GitHub repos, but are still kept
in sync with the main pmemkv distribution.
- Java - https://github.com/pmem/pmemkv-java
- JNI - https://github.com/pmem/pmemkv-jni
- Node.js - https://github.com/pmem/pmemkv-nodejs
- Ruby - https://github.com/pmem/pmemkv-ruby
The pmemkv_bench utility provides some simple read & write benchmarks. This is
much like the db_bench utility included with LevelDB and RocksDB, although the
list of supported parameters is slightly different.
pmemkv_bench
--engine=<name> (storage engine name, default: kvtree2)
--db=<location> (path to persistent pool, default: /dev/shm/pmemkv)
(note: file on DAX filesystem, DAX device, or poolset file)
--db_size_in_gb=<integer> (size of persistent pool to create in GB, default: 0)
(note: always use 0 with poolset or device DAX configs)
--histogram=<0|1> (show histograms when reporting latencies)
--num=<integer> (number of keys to place in database, default: 1000000)
--reads=<integer> (number of read operations, default: 1000000)
--threads=<integer> (number of concurrent threads, default: 1)
--value_size=<integer> (size of values in bytes, default: 100)
--benchmarks=<name>, (comma-separated list of benchmarks to run)
fillseq (load N values in sequential key order)
fillrandom (load N values in random key order)
overwrite (replace N values in random key order)
readseq (read N values in sequential key order)
readrandom (read N values in random key order)
readmissing (read N missing values in random key order)
deleteseq (delete N values in sequential key order)
deleterandom (delete N values in random key order)
Benchmarking on emulated persistent memory:
PMEM_IS_PMEM_FORCE=1 ./bin/pmemkv_bench --db=/dev/shm/pmemkv --db_size_in_gb=1
Benchmarking on filesystem DAX:
PMEM_IS_PMEM_FORCE=1 ./bin/pmemkv_bench --db=/mnt/pmem/pmemkv --db_size_in_gb=1
Benchmarking on device DAX:
./bin/pmemkv_bench --db=/dev/dax1.0
Benchmarking with poolset:
PMEM_IS_PMEM_FORCE=1 ./pmemkv_bench --db=~/pmemkv.poolset