Maxim Sokhatsky
Live
KVS
Abstract
Erlang Term
Database
@5HT
How not to
store a cat?
SQL -- relational algebra for cats
ORM -- trying to do OLAP in Java
XML -- suck <and> blow waste
JSON -- JavaScript database
How our fathers stored cats?
BerkleyDB -- 1986
LDAP -- X.500 Interface to KV
CAP
CA -- SQL RDBMS, Mnesia (non-distr)
CP -- BDB,Redis,BigTable,MongoDB
AP -- Dynamo,Riak,Couch,Cassandra
Consistency
GOSSIP AP -- Eventual
PAXOS CP -- Hadoop
RAFT CP -- etcd
ZAB CP -- ZooKeeper HBase Storm
2PC CP -- SQL RDBMS
Erlang AP KV Boom
CouchDB Couchbase Riak
Mnesia KAI LeoDB
Hibari HanoiDB
Pitfalls
How to store a
cat-to-cat subscriptions?
Secondary Indexes
1. LevelDB -- HanoiDB, Riak
2. DETS -- Mnesia, KAI
Pitfalls
How to put each cat in a box?
Consistency
1. Reconcile -- Editing History
2. Sequence Lock -- Erlang Process
REST
Riak/WebMachine
CouchDB/Own
FarWest/Cowboy
Abstractions
Feuerlabs/kvdb
project-fifo/fifo-db
KVS is...
Statically Typed
Mutable
Social Schema Based
Key-Value
Term Store
with Secondary Indexes
and REST ... Abstract DB
Social Schema
Users Products Reviews Feeds
Comments Groups Likes Acl
Teams Tags Messages
Payments Accounts
Transactions Purchases
KVS
Riak / Mnesia / KAI Backends
Cowboy/N2O REST
JSON/BERT/...
Records <-> Maps <-> Proplists
1500 LOC
KVS
#user { id, mail, name }
#product { id, name }
#review { id, body, product }
KVS
#user { id, next, prev, ... }
#product { id, next, prev, ... }
#review { id, next, prev, ... }
KVS core
#container { id, top, count }
#iterator { id, container,
prev, next
feeds = [] }
KVS Core
#subscription { who,
whom,
what }
Macros
-define(CONTAINER, id, top, count=0).
-define(ITERATOR(Container),
id, container=Container,
feed_id, prev, next, feeds=[]).
-record(feed, {?CONTAINER, aclver}).
-record(user, {?ITERATOR(feed), name, email}).
-record(product, {?ITERATOR(feed), name}).
Schema
-define(CONTAINERS, [
{feed, record_info(fields, feed)},
{acl, record_info(fields, acl)},
{transaction, record_info(fields, transaction)},
{payment, record_info(fields, payment)} ]).
API
POST/PUT kvs:add(#user{}).
DELETE kvs:remove(user,2).
PUT/PATCH kvs:put(#user{id=2}).
GET kvs:entries(Feed,user).
GET kvs:get(user,2).
GET kvs:all(user).
Feed Server
For each Container
we spawn Erlang Lock Process
for Feed Write Operations
ordering and consistensy
Console
> kvs:join('root@synrc.com').> kvs:initialize().> kvs:init_db().> kvs:dir().[{table,"user"}, {table,"feed"}, {table,"acl"}, {table,"transaction"}, ... ]
> kvs:join('root@synrc.com').> kvs:initialize().> kvs:init_db().> kvs:dir().[{table,"user"}, {table,"feed"}, {table,"acl"}, {table,"transaction"}, ... ]Console
> rr(kvs).> kvs:add(#user{id="[email protected]"}).> kvs:entries( kvs:get(feed,users), user,undefined).[#user{id="[email protected]"}]> kvs:get(user,"mes@ua.fm").{ok,#user{id="[email protected]"}
> rr(kvs).> kvs:add(#user{id="[email protected]"}).> kvs:entries( kvs:get(feed,users), user,undefined).[#user{id="[email protected]"}]> kvs:get(user,"mes@ua.fm").{ok,#user{id="[email protected]"}1500 LOC
@5HT @doxtop
https://github.com/synrc/kvs
KVS
By Maxim Sokhatsky
KVS
KVS is statically typed, Erlang Term, abstract database with secondary indexes support and REST handler. It has default schema that was designed for social applications, blog engines, app stores.
- 9,028