NOSQL DATABASES
DATA COLLECTION & DESIGN OF
EXPERIMENTS
14/02/2019
Andon Tchechmedjiev 1
1 LGI2P, IMT Mines Ales
OUTLINE
1. Background
2. What is NOSQL?
3. The CAP Theorem
4. Types of NOSQL DBMS
BACKGROUND
Background
Background
I Relational databases → mainstay of business
I Web-based applications caused spikes
I explosion of social media sites (Facebook, Twitter) with
large data needs
I rise of cloud-based solutions such as Amazon S3 (simple
storage solution)
I Hooking RDBMS to web-based application becomes
trouble
3
Background
Issues with scaling up
I Best way to provide ACID and rich query models is to have
the dataset on a single machine
I Limits to scaling up (or vertical scaling: make a ”single”
machine more powerful) → dataset is just too big!
I Scaling out (or horizontal scaling: adding more
smaller/cheaper servers) is a better choice
I Different approaches for horizontal scaling (multi-node
database):
I Master/Slave
I Sharding (partitioning)
4
Background
Scaling out RDBMS: Master/Slave
I Master/Slave
I All writes are written to the master
I All reads performed against the replicated slave databases
I Critical reads may be incorrect as writes may not have been
propagated down
I Large datasets can pose problems as master needs to
duplicate data to slaves
5
Background
Scaling out RDBMS: Sharding
I Sharding (Partitioning)
I Scales well for both reads and writes
I Not transparent, application needs to be partition-aware
I Can no longer have relationships/joins across partitions
I Loss of referential integrity across shards
6
Background
Other ways to scale out RDBMS
I Multi-Master replication
I INSERT only, not UPDATES/DELETES
I No JOINs, thereby reducing query time
I This involves de-normalizing data
I In-memory databases
7
WHAT IS NOSQL?
What is NOSQL?
What is NOSQL?
I The Name:
I Stands for Not Only SQL
I The term NOSQL was introduced by Carl Strozzi in 1998 to
name his file-based database
I It was again re-introduced by Eric Evans when an event
was organized to discuss open source distributed databases
I Eric states that ”... but the whole point of seeking
alternatives is that you need to solve a problem that
relational databases are a bad fit for...”
9
What is NOSQL?
What is NoSQL?
I Key features (advantages):
I Non-relational
I Doesn’t require schema
I Data are replicated to multiple nodes (so, identical and
fault tolerant) and can be partitioned:
◦ Nodes taht are down can easily be replaced
◦ No single point of failure
I Horizontal scalability
I Cheap, easy to implement (open-source)
I Massive write performance
I Fast key-value access
10
What is NOSQL?
What is NoSQL?
11
What is NOSQL?
What is NOSQL?
I Disadvantages:
I Don’t fully support relational features
◦ No join, group by, order by operations (except within
partitions)
◦ No referential integrity constraints across partitions
I No(t always) declarative query language (e.g., SQL) →
more programming
I Relaxed ACID (see CAP theorem) → fewer guarantees
I No easy integration with other applications that support
SQL
12
What is NOSQL?
Who is using them?
13
What is NOSQL?
3 major papers for NOSQL
I Three major papers were the ”seeds” of the NOSQL
movement:
I BigTable (Google)
I DynamoDB (Amazon)
◦ Ring partition and replication
◦ Gossip protocol (discovery and error detection)
◦ Distributed key-value data stores
◦ Eventual consistency
I CAP Theorem
14
What is NOSQL?
The Perfect Storm
I Large datasets, acceptance of alternatives, and
dynamically-typed data has come together in a “perfect
storm”
I Not a backlash against RDBMS
I SQL is a rich query language that cannot be rivaled by the
current list of NOSQL offerings
15
THE CAP THEOREM
The CAP Theorem
CAP Theorem
I Suppose three properties of a distributed system (sharing
data):
I Consistency: All copies have same value
I Availability: reads and writes always succeed
I Partition-tolerance: system properties (consistency and/or
availability) hold even when network failures prevent some
machines from communicating with others
17
The CAP Theorem
CAP Theorem
18
The CAP Theorem
CAP Theorem
I Brewer’s CAP Theorem:
I For any system sharing data, it is “impossible” to guarantee
simultaneously all of these three properties
I You can have at most two of these three properties for any
shared-data system
I Very large systems will “partition” at some point:
I That leaves either C or A to choose from (traditional DBMS
prefers C over A and P )
I In almost all cases, you would choose A over C (except in
specific applications such as order processing)
19
The CAP Theorem
CAP Theorem
I Consistency
I 2 types of consistency:
1. Strong consistency – ACID (Atomicity, Consistency,
Isolation, Durability)
2. Weak consistency – BASE (Basically Available Soft-state
Eventual consistency)
20
The CAP Theorem
CAP Theorem
I ACID – A DBMS is expected to support ”ACID
transaction”, processes that are:
I Atomicity: Either the whole process is done or none is
I Consistency: Only valid data are written
I Isolation: One operation at a time
I Durability: Once commited it always stays that way.
I CAP
I Consistency: All data on the cluster have the same copies
I Availability: The cluster always accepts reads and writes
I Partition tolerance: guaranteed properties are maintained
even when network failures prevent some machines from
communicating with others
21
The CAP Theorem
CAP Theorem
I A consistency model determines rules for visibility and
apparent order of updates
I Example:
I Row X is replicated on nodes M and N
I Client A writes row X to node N
I Some period of time t elapses
I Client B reads row X from node M
I Does client B see the write from client A?
I Consistency is a continuum with tradeoffs
I For NOSQL, the answer would be: ”maybe”
I CAP theorem states: ”strong consistency can’t be achieved
at the same time as availability and partition-tolerance”
22
The CAP Theorem
CAP Theorem
I Eventual consistency
I When no updates occur for a long period of time,
eventually all updates will propagate through the system
and all the nodes will be consistent
I Cloud computing
I ACID is hard to achieve, moreover, it is not always required,
e.g. for blogs, status updates, product listings, etc.
23
TYPES OF NOSQL DBMS
Types of NOSQL DBMS
NOSQL categories
I Key-value
I Example: DynamoDB, Voldermort, Scalaris
I Document-based
I Example: MongoDB, CouchDB
I Column-based
I Example: BigTable, Cassandra, Hbased
I Graph-based
I Example: Neo4J, InfoGrid
I ”No-schema” is a common characteristics of most NOSQL
storage systems
I Provide ”flexible” data types
25
Types of NOSQL DBMS
NOSQL categories and CAP
26
Types of NOSQL DBMS
Key-value
I Focus on scaling to huge amounts of data
I Designed to handle massive load
I Based on Amazon’s dynamo paper
I Data model: (global) collection of Key-value pairs
I Dynamo ring partitioning and replication
I Example: (DynamoDB)
I items having one or more attributes (name, value)
I An attribute can be single-valued or multi-valued like set.
I items are combined into a table
27
Types of NOSQL DBMS
Key-value
I Basic API access:
I get(key): extract the value given a key
I put(key, value): create or update the value given its key
I delete(key): remove the key and its associated value
I execute(key, operation, parameters): invoke an
operation to the value (given its key) which is a special data
structure (e.g. List, Set, Map, etc.)
28
Types of NOSQL DBMS
Key-value
I Pros:
I very fast
I very scalable (horizontally distributed to nodes based on
key)
I simple data model
I eventual consistency
I fault-tolerance
I Cons:
I Can’t model more complex data structure such as objects
29
Types of NOSQL DBMS
Key-value
Name Producer Data model Querying
SimpleDB Amazon Set of couples (key, {attribute}), Restricted SQL; select, delete,
where attribute is a couple (name, GetAttributes, and PutAttributes
value) operations
Redis Salvatore Set of couples (key, value), where primitive operations for each value
Sanfilippo value is simple typed value, list, or- type
dered (according to ranking) or un-
ordered set, hash value
Dynamo Amazon Like SimpleDB Simple get operation and put in a
context
Voldemort LinkeId Like SimpleDB Similar to Dynamo
30
Types of NOSQL DBMS
Document-based
I Can model more complex objects
I Inspired by Lotus Notes
I Data model: collection of documents
I Document: JSON (JavaScript Object Notation is a data model, key-value
pairs, which supports objects, records, structs, lists, array, maps, dates,
Boolean with nesting), XML, other semi-structured formats.
31
Types of NOSQL DBMS
Document-based
I Example: (MongoDB) document
{
Name:'Jaroslav',
Address:'Malostranske nám. 25, 118 00 Praha 1',
Grandchildren: {Claire: "7", Barbara: "6", "Magda: "3", "Kirsten: "1", "Otis: "3", Richard: "1“}
Phones: [ “123-456-7890”, “234-567-8963” ]
}
32
Types of NOSQL DBMS
Document-based
Name Producer Producer Data model Data model Querying Querying
MongoDB MongoDB 10gen 10gen Object-structured doc- Object-structured doc- Manipulations with
uments stored in collec- uments stored in collec- objects in collections
tions;each object has a tions;each object has a (find object or objects
primary key called Ob- primary key called Ob- via simple selections
jectId jectId and logical expressions,
delete, update,)
Couchbase Couchbase Couchbase1 Couchbase1 Document as a list Document as a list By key and key range,
of named (structured) of named (structured) views via Javascript and
items (JSON document) items (JSON document) MapReduce
33
Types of NOSQL DBMS
Column-based
I Based on Google’s BigTable paper
I Like column oriented relational databases (store data in column order)
but with a twist
I Tables similarly to RDBMS, but handle semi-structured
I Data model:
I Collection of Column Families
I Column family = (key, value) where value = set of related columns
(standard, super)
I Indexed by row key, column key and timestamp
34
Types of NOSQL DBMS
Column-based
35
Types of NOSQL DBMS
Column-based
I Allow key-value pairs to be stored (and retrieved on key) in a massively
parallel system
I Storing principle: big hashed distributed tables
I Properties: partitioning (horizontally and/or vertically), high
availability etc. completely transparent to application
I Better: extensible records
36
Types of NOSQL DBMS
Column-based
I One column family can have variable numbers of columns
I Cells within a column family are sorted ”physically”
I Very sparse, most cells have null values
I Comparison: RDBMS vs column-based NOSQL
I Query on multiple tables
I RDBMS: must fetch data from several places on disk and
glue together
I Column-based NOSQL: only fetch column families of those
columns that are required by a query (all columns in a
column family are stored together on the disk, so multiple
rows can be retrieved in one read operation → data locality)
37
Types of NOSQL DBMS
Column-based
Example: (Cassandra column family–timestamps removed for simplicity)
UserProfile = {
Cassandra = { emailAddress:”
[email protected]” , age:”20”}
TerryCho = { emailAddress:”
[email protected]” , gender:”male”}
Cath = { emailAddress:”
[email protected]” , age:”20”,gender:”female”,address:”Seoul”}
}
38
Types of NOSQL DBMS
Column-based
Name Producer Producer Data model Querying
BigTable BigTable Google Set of couples (key, value) Selection (by combination of row,
column, and time stamp ranges)
HBase HBase Apache Groups of columns (a BigTable JRUBY IRB-based shell (similar to
clone) SQL)
Hypertable Hypertable Hypertable Like BigTable HQL (Hypertext Query Language)
CASSANDRACASSANDRAApache (origi- Columns, groups of columns corre- Simple selections on key, range
nally Facebook) sponding to a key (supercolumns) queries, column or columns ranges
PNUTS PNUTS Yahoo (hashed or ordered) tables, typed Selection and projection from a sin-
arrays, flexible schema gle table (retrieve an arbitrary sin-
gle record by primary key, range
queries, complex predicates, order-
ing, top-k)
39
Types of NOSQL DBMS
Graph-based
I Focus on modeling the structure of data (interconnectivity)
I Scales to the complexity of data
I Inspired by mathematical Graph Theory (G=(E,V))
I Data model:
I (Property Graph) nodes and edges
◦ Nodes may have properties (including ID)
◦ Edges may have labels or roles
I Key-value pairs on both
I Interfaces and query languages vary
I Single-step vs path expressions vs full recursion
I Example: ´Neo4j, FlockDB, Pregel, InfoGrid . . .
40
Types of NOSQL DBMS
Conclusion
I NOSQL database cover only a part of data-intensive cloud
applications (mainly Web applications)
I Problems with cloud computing:
I SaaS (Software as a Service or on-demand software)
applications require enterprise-level functionality,
including ACID transactions, security, and other features
associated with commercial RDBMS technology, i.e.
NOSQL should not be the only option in the cloud
I Hybrid solutions:
◦ Voldemort with MySQL as one of storage backend
◦ deal with NOSQL data as semi-structured data
◦ Integrating RDBMS and NOSQL via SQL/XML
41
Types of NOSQL DBMS
Conclusion
I next generation of highly scalable and elastic RDBMS:
NewSQL databases (from April 2011)
I they are designed to scale out horizontally on shared
nothing machines,
I still provide ACID guarantees,
I applications interact with the database primarily using
SQL,
I the system employs a lock-free concurrency control scheme
to avoid user shut down,
I the system provides higher performance than available
from the traditional systems.
I Examples: MySQL Cluster (most mature solution), VoltDB,
Clustrix, ScalArc, etc.
42
Types of NOSQL DBMS
References
I Rajshekhar Sunderraman
I http://tinman.cs.gsu.edu/~raj/8711/sp13/
berkeleydb/finalpres.ppt
I Tobias Ivarsson
I http://www.slideshare.net/thobe/nosql-for-dummies
I Jennifer Widom
I http:
//www.stanford.edu/class/cs145/ppt/cs145nosql.pptx
I Ruoming Jin
I http://www.cs.kent.edu/~jin/Cloud12Spring/
HbaseHivePig.pptx
I Seth Gilbert
I http://lpd.epfl.ch/sgilbert/pubs/
BrewersConjecture-SigAct.pdf
I Patrick McFadin
I http://www.slideshare.net/patrickmcfadin/ 43
