Data Warehousing and OLAP Technology for Data Mining
Data Warehousing and OLAP Technology for Data Mining
What is a data warehouse? A multi-dimensional data model From data warehousing to data mining
Based on the slides developed by J Han and M Kamber
What is Data Warehouse?
Data WarehouseSubject-Oriented
Defined in many different ways, but not rigorously. A decision support database that is maintained separately from the organizations operational database Support information processing by providing a solid platform of consolidated, historical data for analysis. A data warehouse is a subject-oriented, integrated, time-variant, and nonvolatile collection of data in support of managements decision-making process.W. H. Inmon Data warehousing: The process of constructing and using data warehouses
Organized around major subjects, such as customer, product, sales. Focusing on the modeling and analysis of data for decision makers, not on daily operations or transaction processing. Provide a simple and concise view around particular subject issues by excluding data that are not useful in the decision support process.
Data WarehouseIntegrated
Data WarehouseTime Variant
Constructed by integrating multiple, heterogeneous data sources relational databases, flat files, on-line transaction records Data cleaning and data integration techniques are applied. Ensure consistency in naming conventions, encoding structures, attribute measures, etc. among different data sources
The time horizon for the data warehouse is significantly longer than that of operational systems.
Operational database: current value data. Data warehouse data: provide information from a historical perspective (e.g., past 5-10 years)
E.g., Hotel price: currency, tax, breakfast covered, etc.
When data is moved to the warehouse, it is converted.
Every important element in the data warehouse contains time, explicitly or implicitly
Data WarehouseNon-Volatile
From Tables and Spreadsheets to Data Cubes
A physically separate store of data transformed from the operational environment. Operational update of data does not occur in the data warehouse environment.
A data warehouse is based on a multidimensional data model which views data in the form of a data cube A data cube, such as sales, allows data to be modeled and viewed in multiple dimensions
Does not require transaction processing, recovery, and concurrency control mechanisms
Dimension tables, such as item (item_name, brand, type), or time(day, week, month, quarter, year) Fact table contains measures (such as dollars_sold) and keys to each of the related dimension tables
Requires only two operations in data accessing:
initial loading of data and access of data.
In data warehousing literature, an n-D base cube is called a base cuboid. The top most 0-D cuboid, which holds the highest-level of summarization, is called the apex cuboid. The lattice of cuboids forms a data cube.
Cube: A Lattice of Cuboids
all time item location supplier
Conceptual Modeling of Data Warehouses
Modeling data warehouses: dimensions & measures
0-D(apex) cuboid
Star schema: A fact table in the middle connected to a set of dimension tables Snowflake schema: A refinement of star schema where some dimensional hierarchy is normalized into a set of smaller dimension tables, forming a shape similar to snowflake Fact constellations: Multiple fact tables share dimension tables, viewed as a collection of stars, therefore called galaxy schema or fact constellation
1-D cuboids
time,item
time,location
item,location
location,supplier
time,supplier time,item,location
2-D cuboids
item,supplier
time,location,supplier
3-D cuboids
time,item,supplier
item,location,supplier
4-D(base) cuboid
time, item, location, supplier
Example of Star Schema
time
time_key day day_of_the_week month quarter year
Example of Snowflake Schema
time item
item_key item_name brand type supplier_type time_key day day_of_the_week month quarter year
item Sales Fact Table time_key item_key branch_key location_key units_sold dollars_sold avg_sales
item_key item_name brand type supplier_key
supplier
supplier_key supplier_type
Sales Fact Table time_key item_key branch_key
branch
branch_key branch_name branch_type
location location_key units_sold dollars_sold avg_sales Measures
location_key street city province_or_street country
branch
branch_key branch_name branch_type
location
location_key street city_key
city
city_key city province_or_street country
Measures
Example of Fact Constellation
time
time_key day day_of_the_week month quarter year
Example of Star Schema
time
time_key day day_of_the_week month quarter year
item Sales Fact Table
time_key item_key branch_key
item_key item_name brand type supplier_type
Shipping Fact Table
time_key item_key shipper_key from_location
item Sales Fact Table time_key item_key branch_key
item_key item_name brand type supplier_type
branch
branch_key branch_name branch_type
location_key units_sold dollars_sold avg_sales Measures
location
location_key street city province_or_street country
to_location dollars_cost units_shipped shipper
shipper_key shipper_name location_key shipper_type
branch
branch_key branch_name branch_type
location location_key units_sold dollars_sold avg_sales Measures
location_key street city province_or_street country
Multidimensional Data
A Sample Data Cube
du ct
TV PC VCR sum 1Qtr 2Qtr
Sales volume as a function of product, month, and region
gi on
Dimensions: Product, Location, Time Hierarchical summarization paths Industry Region Year
Date
3Qtr 4Qtr sum
Total annual sales of TV in U.S.A. U.S.A Canada Mexico sum
Pr o
Category Country Quarter
Product
Product
City Office
Month Day
Week
Month
Cuboids Corresponding to the Cube
all
product
Browsing a Data Cube
0-D(apex) cuboid country
date, country
date
product,country
1-D cuboids
product,date
2-D cuboids 3-D(base) cuboid
product, date, country
Visualization OLAP capabilities Interactive manipulation
Country
Re
