Data Segment

Compression

Julian Dyke
Independent Consultant
Web Version

1 © 2005 Julian Dyke juliandyke.com

 Agenda
1. Introduction
2. What is data compression?
3. Data segment compression
1. Functionality
2. Syntax
3. Implementation
4. Performance
4. Conclusion

2 © 2005 Julian Dyke juliandyke.com

 What is data compression?
 Data is
 compressed when it is written to a block
 decompressed when it is read from the block

 Compression requires
 lessdata storage to hold the compressed data
 more CPU to compress and decompress the data

3 © 2005 Julian Dyke juliandyke.com

 Why use data compression?
 Data compression
 Increases number of rows in each block
 Reduces number of blocks required to store data
 For a full table scan reduces number of logical
(and probably physical) I/Os required
 For a table access by ROWID increases
probability that block is already in buffer cache
 Improves buffer cache hit ratio
 Potentially reduces backup and recovery times

4 © 2005 Julian Dyke juliandyke.com

 When does Oracle use compression?
 Oracle compresses some data types including
 VARCHAR2
 NUMBER
 RAW

 Oracle does not compress

 DATE
 CHAR

 Compression is often achieved by

 using length byte(s)
 trimming unused characters/bytes

5 © 2005 Julian Dyke juliandyke.com

 When does Oracle use compression?
 Oracle also compresses
 Length bytes in table blocks
 Length bytes in index blocks
 NULL values
 NULL values at end of each row
 Index branch blocks (suffix compressed)
 Index leaf blocks (optionally prefix compressed)

 In addition some data structures implicitly

compress data
 IOTs
 Index Clusters

6 © 2005 Julian Dyke juliandyke.com

 Data Segment Compression
 Introduced in Oracle 9.2
 Intended for
 DSS environments
 Read-only tables

 Not intended for

 OLTP environments
 Environments with any DML activity subsequent
to data loading
 Data is compressed at block level
 Direct path load must be used

7 © 2005 Julian Dyke juliandyke.com

 Restrictions
 Data segment compression cannot be used
with
 IOTs
 IOT overflow segments
 IOT mapping tables
 Index clustered tables
 Hash clustered tables
 Hash partitions
 Hash / list subpartitions
 External Tables

8 © 2005 Julian Dyke juliandyke.com

 Block Level Compression
 Compression is applied at block level

 Blocks will only be compressed if

 datais sufficiently large to fill the block
 rows have low enough cardinality

 Columns will be reordered within each block to

achieve optimal compression ratios

 A segment may contain

 compressed and uncompressed blocks
 blocks compressed on different columns

9 © 2005 Julian Dyke juliandyke.com

 Direct Path Load
 Direct path load bypasses much on the work
done by conventional load

 Direct path load

 reserves extents from above HWM
 formats rows into blocks
 inserts blocks back into table
 adjusts HWM

 No other transactions can be active on the table

whilst load is in progress

1 © 2005 Julian Dyke juliandyke.com

 Direct Path Load
 In Oracle 9.2 the following statements can use
direct path loads
 CREATE TABLE AS SELECT
 INSERT /* + APPEND */
 ALTER TABLE MOVE

 In addition the following features can use direct

path loads
 Materialized View Refresh
 SQL*Loader
 Online reorganisation

11 © 2005 Julian Dyke juliandyke.com

 Creating New Tables
 Tables are compressed using COMPRESS
clause

CREATE TABLE t1
(
c01 NUMBER,
c02 VARCHAR2(30)
)
COMPRESS;

 Default is for tables to be uncompressed

 This is equivalent to using the NOCOMPRESS
clause

1 © 2005 Julian Dyke juliandyke.com

 Successful Compression
 If conditions are met then these statements
create compressed data blocks
CREATE TABLE t2 COMPRESS AS
SELECT * FROM t1;

CREATE TABLE t2 COMPRESS AS

SELECT * FROM t1 WHERE ROWNUM < 1;
INSERT /*+ APPEND */ INTO t2
SELECT * FROM t1;

1 © 2005 Julian Dyke juliandyke.com

 Unsuccessful Compression
 These statements will not create compressed
data blocks
CREATE TABLE t2 AS
SELECT * FROM t1 WHERE ROWNUM < 1;
INSERT INTO t2 VALUES ('DBA_TABLES',46);

CREATE TABLE t2 COMPRESS AS

SELECT * FROM t1 WHERE ROWNUM < 1;
INSERT INTO t2
SELECT * FROM t1;

1 © 2005 Julian Dyke juliandyke.com

 Altering Existing Tables
 Compression can be specified for an existing
table
ALTER TABLE t1 COMPRESS;

 Existing blocks are not compressed

 New blocks will be compressed if direct path load is used
 Similarly

ALTER TABLE t1 NOCOMPRESS;

 disables compression for new data blocks, but

does not change existing data blocks

1 © 2005 Julian Dyke juliandyke.com

 Moving Existing Tables
 Tables can be moved using
ALTER TABLE t1 MOVE COMPRESS;

 This command
 Creates a new segment
 Uses direct load to copy and compress blocks
 Drops old segment

1 © 2005 Julian Dyke juliandyke.com

 Data Dictionary Views
 Not modified in Oracle 9.2

 Data segment compression flag is missing from

 ALL_TABLES
 DBA_TABLES
 USER_TABLES

 Data segment compression is recorded by

setting a bit in TAB$.SPARE1

 Affects Håkan factor – maximum number of

rows that can be held on block

1 © 2005 Julian Dyke juliandyke.com

 Data Dictionary Views
 In Oracle 9.2.0.1 the following will list all
compressed tables in the database
SELECT
u.name AS owner,
o.name AS table_name
FROM
sys.tab$ t,
sys.obj$ o,
sys.user$ u
WHERE BITAND (t.spare1, 131072) = 131072
AND o.obj# = t.obj#
AND o.owner# = u.user#;

1 © 2005 Julian Dyke juliandyke.com

 Tablespace Defaults
 Data segment compression can be specified at
tablespace level
CREATE TABLESPACE ts01 DEFAULT COMPRESS;

 All new objects created will have compression enabled

 Data segment compression can also be specified for existing tablespaces

ALTER TABLESPACE ts01 DEFAULT COMPRESS;

1 © 2005 Julian Dyke juliandyke.com

 Data Dictionary Views
 In Oracle 9.2.0.1 the DBA_TABLESPACES view
was not been updated to include data segment
compression

 Use the following to identify tablespaces with

compression enabled
SELECT name
FROM sys.ts$
WHERE BITAND (flags, 64) = 64;

2 © 2005 Julian Dyke juliandyke.com

 Partitioned Tables
 Data segment compression can also be used
with range or list partitioned tables

CREATE TABLE t1 (c01 NUMBER, c02 VARCHAR2(200))

PARTITION BY RANGE (c01)
(
PARTITION p1 VALUES LESS THAN (100),
PARTITION p2 VALUES LESS THAN (200)
)
COMPRESS;

 Oracle 9.2 cannot compress hash or composite

partitioned tables

2 © 2005 Julian Dyke juliandyke.com

 Partitioned Tables
 Can also create a table with some partitions
compressed and others uncompressed
CREATE TABLE t1 (c01 NUMBER, c02 VARCHAR2(200))
PARTITION BY RANGE (c01)
(
PARTITION p1 VALUES LESS THAN (100) COMPRESS,
PARTITION p2 VALUES LESS THAN (200)
)
COMPRESS;

 Compression can also be specified for new

partitions added to an existing table
ALTER TABLE t1
ADD PARTITION p3
VALUES LESS THAN (300) COMPRESS;

2 © 2005 Julian Dyke juliandyke.com

 Partitioned Tables
 Existing partitions can be specified as
compressed/uncompressed using
ALTER TABLE t1 MODIFY PARTITION p1 COMPRESS;
ALTER TABLE t1 MODIFY PARTITION p1 NOCOMPRESS;

 These do not affect existing rows

 An existing uncompressed partition can be
compressed using
ALTER TABLE t1 MOVE PARTITION p1 COMPRESS;
 This
 creates new segment,
 copies and compresses all the rows
 drops old segment

2 © 2005 Julian Dyke juliandyke.com

 Data Dictionary Views
 DBA_PART_TABLES.DEF_COMPRESSION
contains
NONE Compression never enabled
ENABLED Compression enabled at table level
DISABLED Compression has been enabled at table
level and subsequently disabled
N/A Partitioned IOT

 DBA_TAB_PARTITIONS.COMPRESSION
contains
ENABLED Compressed enabled for partition
DISABLED Otherwise

2 © 2005 Julian Dyke juliandyke.com

 Nested Tables
 Compression can be specified for storage table
of a nested table
CREATE TABLE t1 (c1 NUMBER, c2 TY2)
NESTED TABLE c2 STORE AS t2 COMPRESS;

 In Oracle 9.2. DBA_NESTED_TABLES has not

been updated to indicate that the storage table
has been compressed

2 © 2005 Julian Dyke juliandyke.com

 Materialized Views
 Compression can be specified for materialized
views

CREATE MATERIALIZED VIEW mv1

COMPRESS
BUILD IMMEDIATE
ENABLE QUERY REWRITE
AS
SELECT c1, c2, SUM (c3)
FROM t1
GROUP BY c1, c2;

2 © 2005 Julian Dyke juliandyke.com

 Materialized Views
 Compression can be also specified for existing
materialized views

ALTER MATERIALIZED VIEW mv1 COMPRESS;

 Data will be compressed the next time the materialized view is

refreshed
 e.g

EXECUTE dbms_mview.refresh ('MV1');

2 © 2005 Julian Dyke juliandyke.com

 SQL*Loader
 SQL*Loader can create data segment
compressed blocks using direct path loads
 Specified using the parameter

DIRECT = TRUE

 Conventional loads using SQLLDR do not

generate compressed blocks

2 © 2005 Julian Dyke juliandyke.com

 PCTFREE
 Default value of PCTFREE for compressed
tables is 0
 Can be overridden manually e.g.

CREATE TABLE t1 (c01 NUMBER)

COMPRESS PCTFREE 10;

 In general the default behaviour is preferable

2 © 2005 Julian Dyke juliandyke.com

 Compression Ratios
 Compression ratios vary with
 number of rows
 number of columns
 cardinality of rows

 Compression ratios can be improved by sorting

table on low cardinality columns prior to
loading

 Can also be improved by using larger block

sizes

3 © 2005 Julian Dyke juliandyke.com

 Compression Ratios
 For example – loading SALES table from sales
history demo schema
$ORACLE_HOME/demo/schema/sales_history

 Table contains 1016271 rows

Block Uncompressed Compressed Ratio %

Size Size (Blocks) Size (Blocks)
2048 18777 13433 71.5
4096 8983 6106 68.0
8192 4398 2850 64.7
16384 2179 1353 62.0

3 © 2005 Julian Dyke juliandyke.com

 Implementation
 Each compressed block contains two tables
 Symbol table – contains one row for each
individual column value or set of column values
 Row table – one row for each row in block

 Each column in row table can be a

 referenceto the symbol table if column is
compressed
 column value if column is not compressed

 Compression is performed at block-level only –

no inter-block references

3 © 2005 Julian Dyke juliandyke.com

 Data Segment Compression
Block Common Header 20 bytes

Transaction Header 24 bytes + 24 bytes per ITL entry

Data Header 14 bytes

Compressed Data Header 16 bytes (variable)

Table directory 8 bytes
Row directory 2 bytes per row

Free Space

Row Data

Symbol Table

Tail 4 bytes

3 © 2005 Julian Dyke juliandyke.com

 Compressed Block Header
 Compressed blocks include an extra header.
 Header length is variable
 Depends on
 number of columns
 order in which they are compressed

 Example of header from block dump

r0_9ir2=0x0
mec_kdbh9ir2=0x1
r1_9ir2=0x0
76543210
flag_9ir2=------OC
fcls_9ir2[5]={ 0 32768 32768 32768 32768 }
perm_9ir2[5]={ 0 1 4 2 3 }

3 © 2005 Julian Dyke juliandyke.com

 Length Bytes
 Column length bytes behave differently in
compressed blocks

<= 200 reference (single column values)

column count (multi-column values)
> 200 AND < 250 length is value - 200
250 (0xFA) length is contained in next two bytes
251 (0xFB) reference is contained in next two bytes

3 © 2005 Julian Dyke juliandyke.com

 Length Bytes
 Examples
Byte(s) - Hex Bytes (s) – Decimal Value
C9 201 1
CA 202 2
CB 203 3
CC 204 4
F8 248 48
F9 249 49
FA 00 32 250 00 50 50
FA 00 33 250 00 51 51
FA 0F 9F 250 15 159 3999
FA 0F A0 250 15 160 4000

3 © 2005 Julian Dyke juliandyke.com

 Example
 Monaco Grand Prix Winners 1993-2002

Year Driver Team

1993 Ayrton Senna McLaren
1994 Michael Schumacher Benetton
1995 Michael Schumacher Benetton
1996 Olivier Panis Ligier
1997 Michael Schumacher Ferrari
1998 Mika Hakkinen McLaren
1999 Michael Schumacher Ferrari
2000 David Coulthard McLaren
2001 Michael Schumacher Ferrari
2002 David Coulthard McLaren

3 © 2005 Julian Dyke juliandyke.com

 Example - Uncompressed Data Block

2002 David Coulthard McLaren

2001 Michael Schumacher Ferrari
2000 David Coulthard McLaren
1999 Michael Schumacher Ferrari
Row 1998 Mika Hakkinen McLaren
Data 1997 Michael Schumacher Ferrari
1996 Olivier Panis Ligier
1995 Michael Schumacher Benetton
1994 Michael Schumacher Benetton
1993 Ayrton Senna McLaren

3 © 2005 Julian Dyke juliandyke.com

 Example - Compressed Data Block
4 0 2002
1 2 2001
4 0 2000
1 2 1999
Row 4 Mika Hakkinen 1998 Row
Data 1 2 1997 Data
Ligier Olivier Panis 1996
3 2 1995
3 2 1994
4 Ayrton Senna 1993
4 4 McLaren
3 2 Benetton
Symbol Symbol
2 5 Michael Schumacher
Table Table
1 3 Ferrari
0 2 David Coulthard

3 © 2005 Julian Dyke juliandyke.com

 Performance
 Performance tests
 Cost of inserting compressed rows
 Cost of selecting compressed rows

 Tested using
 Oracle 9.2.0.1
 Sun Ultra Enterprise 450 – 4 CPUs
 8192 byte block size

 Test data adapted from sales history demo

$ORACLE_HOME/demo/schema/sales_history

 SALES table contains 1016271 rows

4 © 2005 Julian Dyke juliandyke.com

 Inserting Compressed Rows
 Loading the entire file into an empty table

Compressed Blocks Elapsed TIme CPU Time

(Secs) (Secs)
No 4398 31.77 4.13
Yes 2850 71.08 43.86

 Compressed data is 35% smaller

 Reduction in logical and physical I/O more than
offset by increase in CPU time to compress blocks
 Statistics from V$SYSSTAT

4 © 2005 Julian Dyke juliandyke.com

 Selecting Compressed Rows
 Selecting all rows from table
SELECT SUM (quantity_sold) FROM sales;

Compressed Blocks Elapsed TIme CPU Time

(Secs) (Secs)
No 4398 3.41 2.77
Yes 2850 3.78 3.53

 Reduction in logical and physical I/O more than

offset by increase in CPU time to compress
blocks
 Statistics from trace file

4 © 2005 Julian Dyke juliandyke.com

 Caveat
 Updating rows is VERY expensive
 Rows must be decompressed before they can
be updated
 In this example the ALL_OBJECTS view
contained 7253 rows
CREATE TABLE t1 PCTFREE 0 AS
SELECT owner, object_name,
subobject_name, object_id
FROM all_objects;

 creates a 28 block table

 The same statement with a COMPRESS clause
creates a 23 block table
4 © 2005 Julian Dyke juliandyke.com
 Caveat
 The statement
UPDATE t2 SET owner = owner;

 decompresses all blocks

 After the update statement is executed the table contains 79 blocks

 Once blocks are decompressed, rollback will

not recompress them
 Use read-only tablespaces to prevent
inadvertent updates
 Deletes do not display this characteristic

4 © 2005 Julian Dyke juliandyke.com

 Conclusions
 DSS / read only feature
 Good compression ratios
 Only works with direct path load
 High CPU usage
 High elapsed times
 Updates are disproportionately expensive
 Documentation is weak
 Data dictionary views need enhancing

4 © 2005 Julian Dyke juliandyke.com

 Thank you for your interest

For more information and to provide feedback

please contact me
My e-mail address is:
[email protected]

My website address is:

www.juliandyke.com

4 © 2005 Julian Dyke juliandyke.com