CS3492 – DATABASE MANAGEMENT SYSTEMS

ACADEMIC YEAR 2024-2025 (EVEN)

Department of Computer Science and Engineering

UNIT-IV - IMPLEMENTATION TECHNIQUES

RAID – File Organization – Organization of Records in Files – Data dictionary Storage – Column
Oriented Storage– Indexing and Hashing –Ordered Indices – B+ tree Index Files – B tree Index Files –
Static Hashing – Dynamic Hashing – Query Processing Overview – Algorithms for Selection, Sorting
and join operations – Query optimization using Heuristics - Cost Estimation.

4.1 RAID
RAID: Redundant Arrays of Independent Disks
RAID (Redundant Array of Independent Disks) in a Database Management System (DBMS) is a
technology that combines multiple physical disk drives into a single logical unit for data storage. The
main purpose of RAID is to improve data reliability, availability, and performance. There are different
levels of RAID, each offering a balance of these benefits.
Key Evaluation Points for a RAID System
 Reliability: How many disk faults can the system tolerate?
 Availability: What fraction of the total session time is a system in uptime mode, i.e. how available
is the system for actual use?
 Performance: How good is the response time? How high is the throughput (rate of processing
work)?
 Capacity: Given a set of N disks each with B blocks, how much useful capacity is available to the
user?

Software and hardware RAID systems

Hardware RAID: The hardware-based array manages the RAID subsystem independently from
the host. It presents a single disk per RAID array to the host.
Software RAID: Software RAID implements the various RAID levels in the kernel disk code. It
offers the cheapest possible solution, as expensive disk controller cards.

SRMMCET / CSE / DBMS 1

 CS3492 – DATABASE MANAGEMENT SYSTEMS

The RAID is demonstrated in the Fig. 4.1

Fig. 4.1 RAID Controller

Different RAID Levels
 RAID-0 (Stripping)
 RAID-1 (Mirroring)
 RAID-2 (Bit-Level Stripping with Dedicated Parity)
 RAID-3 (Byte-Level Stripping with Dedicated Parity)
 RAID-4 (Block-Level Stripping with Dedicated Parity)
 RAID-5 (Block-Level Stripping with Distributed Parity)
 RAID-6 (Block-Level Stripping with two Parity Bits)

1. RAID-0 (Stripping)
RAID level 0 divides data into block units and writes them across a number of disks. As data is
placed across multiple disks, it is also called ―data Striping. The advantage of distributing data over
disks is that if different I/O requests are pending for two different blocks of data, then there is a
possibility that the requested blocks are on different disks. There is no parity checking of data. So if data
in one drive gets corrupted then all the data would be lost. Thus RAID 0 does not support data recovery.
Spanning is another term that is used with RAID level 0 because the logical disk will span all the
physical drives. RAID-0 implementation requires minimum 2 disks.

Fig. 4.2 RAID-0

Advantages
 I/O performance is greatly improved by spreading the I/O load across many channels & drives.
 Best performance is achieved when data is striped across multiple controllers with only one
driver per controller

SRMMCET / CSE / DBMS 2

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Disadvantages
 It is not fault-tolerant, failure of one drive will result in all data in an array being lost

RAID LEVEL 1: Mirroring (or shadowing)

It is also known as disk mirroring, this configuration consists of at least two drives that duplicate
the storage of data. There is no striping. Read performance is improved since either disk can be read at
the same time. Write performance is the same as for single disk storage. Every write is carried out on
both disks. If one disk in a pair is fails, data still available in the other. Data loss would occur only if a
disk fails, and its mirror disk also fails before the system is repaired Probability of combined event is
very small.

Fig. 4.3 RAID-1

RAID LEVEL 2: Hamming Code ECC

This configuration uses striping across disks, with some disks storing error checking and
correcting (ECC) information. It has no advantage over RAID 3 and is no longer. Each bit of data word
is written to a data disk drive (4 in this example: 0 to 3). Each data word has its Hamming Code ECC
word recorded on the ECC disks. On Read, the ECC code verifies correct data or corrects single disk
errors.

Fig. 4.4 RAID-2

Advantages-
 On the fly' data error correction
 Extremely high data transfer rates possible
 The higher the data transfer rate required, the better the ratio of data disks to ECC disks.
Disadvantages-
 Very high ratio of ECC disks to data disks with smaller word sizes
 Entry level cost very high
 Requires very high transfer rate requirement to justify.
 No commercial implementations exist

SRMMCET / CSE / DBMS 3

 CS3492 – DATABASE MANAGEMENT SYSTEMS

RAID LEVEL 3: Bit-Interleaved Parity

A single parity bit is enough for error correction, not just detection, since we know which disk
has failed when writing data; corresponding parity bits must also be computed and written to a parity bit
disk. To recover data in a damaged disk, compute XOR of bits from other disks. I/O operation addresses
all the drives at the same time, RAID 3 cannot overlap I/O. For this reason, RAID 3 is best for single-
user systems with long record applications. The data block is subdivided and written on the data disks.
Stripe parity is generated on Writes, recorded on the parity disk and checked on Reads. RAID Level 3
requires a minimum of 3 drives to implement.
Advantages
 Very high Read data transfer rate
 Very high Write data transfer rate
 Disk failure has an insignificant impact on throughput
 Low ratio of ECC (Parity) disks to data disks means high efficiency
Disadvantages
 Transaction rate equal to that of a single disk drive at best (if spindles are synchronized).
 Controller design is fairly complex.
 Very difficult and resource intensive to do as a 'software' RAID

Fig. 4.5 RAID-3

RAID LEVEL 4: Block-Interleaved Parity

When writing data block, corresponding block of parity bits must also be computed and written
to parity disk. To find value of a damaged block, compute XOR of bits from corresponding blocks
(including parity block) from other disks. Each entire block is written onto a data disk. Parity for same
rank blocks is generated on Writes, recorded on the parity disk and checked on Reads. RAID Level 4
requires a minimum of 3 drives to implement.

Fig. 4.6 RAID-4

SRMMCET / CSE / DBMS 4

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Advantages
 Very high Read data transaction rate
 Low ratio of ECC (Parity) disks to data disks means high efficiency
 High aggregate Read transfer rate
 Low ratio of ECC (Parity) disks to data disks means high efficiency
Disadvantages
 Quite complex controller design
 Worst Write transaction rate and Write aggregate transfer rate
 Difficult and inefficient data rebuild in the event of disk failure
 Block Read transfer rate equal to that of a single disk

RAID LEVEL 5:
RAID 5 uses striping as well as parity for redundancy. It is well suited for heavy read and low
write operations. Block-Interleaved Distributed Parity; partitions data and parity among all N + 1 disks,
rather than storing data in N disks and parity in 1 disk.

Fig. 4.7 RAID-5

RAID LEVEL 6:
This technique is similar to RAID 5, but includes a second parity scheme that is distributed
across the drives in the array. The use of additional parity allows the array to continue to function even
if two disks fail simultaneously. However, this extra protection comes at a cost. P+Q Redundancy
scheme; similar to Level 5, but stores extra redundant information to guard against multiple disk
failures. Better reliability than Level 5 at a higher cost; not used as widely.

4.2 File Organization

File Organization refers to the logical relationships among various records that constitute the file,
particularly with respect to the means of identification and access to any specific record. In simple
terms, storing the files in a certain order is called File Organization.
Objective of file organization
 It contains an optimal selection of records, i.e., records can be selected as fast as possible.
 To perform insert, delete or update transaction on the records should be quick and easy.

SRMMCET / CSE / DBMS 5

 CS3492 – DATABASE MANAGEMENT SYSTEMS

 The duplicate records cannot be induced as a result of insert, update or delete.

 For the minimal cost of storage, records should be stored efficiently.

4.3 Organization of Records in Files

 The File is a collection of records. Using the primary key, we can access the records. The type
and frequency of access can be determined by the type of file organization which was used for
a given set of records.
 File organization is a logical relationship among various records. This method defines how file
records are mapped onto disk blocks.
 File organization is used to describe the way in which the records are stored in terms of
blocks, and the blocks are placed on the storage medium.
 The first approach to map the database to the file is to use the several files and store only one
fixed length record in any given file. An alternative approach is to structure our files so that
we can contain multiple lengths for records.
 Files of fixed length records are easier to implement than the files of variable length records.

Types of File Organizations

 Sequential File Organization
 Heap File Organization
 Hash File Organization
 B+ Tree File Organization
 Clustered File Organization
 ISAM (Indexed Sequential Access Method)

Sequential File Organization

The easiest method for file Organization is the Sequential method. In this method, the file is
stored one after another in a sequential manner. There are two ways to implement this method:
a. Pile File Method:
It is a quite simple method. In this method, we store the record in a sequence, i.e., one after
another. Here, the record will be inserted in the order in which they are inserted into tables. In
case of updating or deleting of any record, the record will be searched in the memory blocks.
When it is found, then it will be marked for deleting, and the new record is inserted

Fig. 4.8 Pile File Method

SRMMCET / CSE / DBMS 6

 CS3492 – DATABASE MANAGEMENT SYSTEMS

b. Sorted File Method:

In this method, the new record is always inserted at the file's end, and then it will sort the
sequence in ascending or descending order. Sorting of records is based on any primary key or any other
key. In the case of modification of any record, it will update the record and then sort the file, and lastly,
the updated record is placed in the right place.

Fig. 4.9 Sorted File Method

Heap file organization

 It is the simplest and most basic type of organization. It works with data blocks. In heap file
organization, the records are inserted at the file's end. When the records are inserted, it doesn't
require the sorting and ordering of records.
 When the data block is full, the new record is stored in some other block. This new data block
need not to be the very next data block, but it can select any data block in the memory to store
new records. The heap file is also known as an unordered file.
 In the file, every record has a unique id, and every page in a file is of the same size. It is the
DBMS responsibility to store and manage the new records.

Fig. 4.10 Heap file organization

Hash File Organization
Hash File Organization uses the computation of hash function on some fields of the records. The
hash function's output determines the location of disk block where the records are to be placed. When a
record has to be received using the hash key columns, then the address is generated, and the whole
record is retrieved using that address. In the same way, when a new record has to be inserted, then the
address is generated using the hash key and record is directly inserted. The same process is applied in
the case of delete and update.

SRMMCET / CSE / DBMS 7

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Fig. 4.11 Hash File Organization

B+ File Organization
 B+ tree file organization is the advanced method of an indexed sequential access method. It
uses a tree-like structure to store records in File.
 It uses the same concept of key-index where the primary key is used to sort the records. For
each primary key, the value of the index is generated and mapped with the record.
 The B+ tree is similar to a binary search tree (BST), but it can have more than two children. In
this method, all the records are stored only at the leaf node. Intermediate nodes act as a pointer
to the leaf nodes. They do not contain any records.

Fig. 4.12 B+ File Organization

Indexed sequential access method (ISAM)

ISAM method is an advanced sequential file organization. In this method, records are stored in
the file using the primary key. An index value is generated for each primary key and mapped with the
record. This index contains the address of the record in the file.

Fig. 4.13 Indexed sequential access method

SRMMCET / CSE / DBMS 8

 CS3492 – DATABASE MANAGEMENT SYSTEMS

4.4 Data dictionary Storage

Definition: Data dictionary is mini database management system that manages the metadata.
 Data dictionaries are helpful to the database administrators in management of database.
 The data in the database dictionary is maintained by several programs and generates the reports if
required.
 The data dictionary is integrated with the database systems in which the data is controlled by the
data dictionaries and is made available to the DBMS software.

The general structure of data dictionary is as shown in the Fig. 4.14.

Fig. 4.14 Structure of data dictionary

Following type of information is maintained by data dictionary.

 Description of the schema of the database.
 Detailed information about the physical database design.
 Description of database users, their roles and their access rights.
 Description of database transactions.
 The description of the relationship between database transactions and data items
referenced by them.
 Information about the usage statistics. That means, how many times the queries are raised
to the database, how many transactions are made by the DBMS.

For example - Consider a Student database, in which various fields are RollNo, FirstName, LastName,
and CourseID. The data dictionary for this database ban maintains the information about this database.
The data dictionary contains the dog column names, Data type of each field and the description of each
column of the database.

Active and Passive Data Dictionaries

Active data dictionary: - Active Data dictionary is managed automatically by the database management
system. They are consistent with current structure. In the active data dictionary, when any modification
or changes is executed by the DBMS, then this dictionary it also gets modified by the DBMS
automatically. Most of the active data dictionaries are derived from system catalog.

SRMMCET / CSE / DBMS 9

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Passive data dictionary: - Passive data dictionary is used only for documentation purpose. Passive
dictionary is a self-contained application and set of files used for documenting the data processing
environment. The process of maintaining or modification of the database is manual. It is managed by the
users of the database systems.

4.5 Column Oriented Storage

Column oriented storage which is also called as columnar database management system that stores data
in columns rather than rows.
 The advantage of column oriented storage is to retrieve the result of query very efficiently.
 It also improves disk I/O performances.
Example:

Fig. 4.15 Example for Row and Column Oriented Storage

Advantages of Column Databases

There are several benefits that go along with columnar databases:
 Column stores are excellent at compression and therefore are efficient in terms of storage.
This means you can reduce disk resources while holding massive amounts of information in
a single column
 Since a majority of the information is stored in a column, aggregation queries are quite fast,
which is important for projects that require large amounts of queries in a small amount of
time.
 Scalability is excellent with column-store databases. They can be expanded nearly infinitely,
and are often spread across large clusters of machines, even numbering in thousands. That
also means that they are great for Massive Parallel Processing

SRMMCET / CSE / DBMS 10

 CS3492 – DATABASE MANAGEMENT SYSTEMS

 Load times are similarly excellent, as you can easily load a billion-row table in a few
seconds. That means you can load and query nearly instantly.
 Large amounts of flexibility as columns do not necessarily have to look like each other. That
means you can add new and different columns without disrupting the whole database. That
being said, entering completely new record queries requires a change to all tables.

Disadvantages of Column Databases

As it usually is in life, nothing is perfect and there are a couple of disadvantages to using
column-oriented databases as well:
 Designing an indexing schema that’s effective is difficult and time consuming. Even then, the
said schema would still not be as effective as simple relational database schemas.
 While this may not be an issue for some users, incremental data loading is suboptimal and
should be avoided if possible.
 This goes for all NoSQL database types and not just columnar ones. Security vulnerabilities in
web applications are ever present and the fact that NoSQL databases lack inbuilt security
features doesn’t help. If security is your number one priority, you should either look into
relational databases you could employ or employ a well-defined schema if possible.
 Online Transaction Processing (OLTP) applications are also not compatible with columnar
databases due to the way data is stored.

4.6 Indexing and Hashing

 Indexing is used to optimize the performance of a database by minimizing the number of
disk accesses required when a query is processed.
 The index is a type of data structure. It is used to locate and access the data in a database
table quickly.
Index structure:
Indexes can be created using some database columns.

 The first column of the database is the search key that contains a copy of the primary key or
candidate key of the table. The values of the primary key are stored in sorted order so that
the corresponding data can be accessed easily.
 The second column of the database is the data reference. It contains a set of pointers holding
the address of the disk block where the value of the particular key can be found.

SRMMCET / CSE / DBMS 11

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Fig. 4.16 Indexing Methods

Hashing in DBMS
In a huge database structure, it is very inefficient to search all the index values and reach the
desired data. Hashing technique is used to calculate the direct location of a data record on the disk
without using index structure. In this technique, data is stored at the data blocks whose address is
generated by using the hashing function. The memory location where these records are stored is known
as data bucket or data blocks.

In this, a hash function can choose any of the column value to generate the address. Most of the
time, the hash function uses the primary key to generate the address of the data block. A hash function is
a simple mathematical function to any complex mathematical function. We can even consider the
primary key itself as the address of the data block. That means each row whose address will be the same
as a primary key stored in the data block.

Fig. 4.17 Hasing

The above diagram shows data block addresses same as primary key value. This hash function
can also be a simple mathematical function like exponential, mod, cos, sin, etc. Suppose we have hash
function mod (5) to determine the address of the data block. In this case, it applies mod (5) hash
function on the primary keys and generates 3, 3, 1, 4 and 2 respectively, and records are stored in those
data block addresses.

SRMMCET / CSE / DBMS 12

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Types of Hashing

Fig. 4.18 Types of Hashing

Factors used to evaluate indexing and hashing techniques:

 One of the main decisions to be taken when developing an indexing technique is the order
in which indexed terms are traversed to generate p-strings. Some indexing techniques use
traversal orders that generate a single p-string from each indexed term, whereas others
generate multiple p-strings.

 Hashing is more appropriate for bigger databases that need to provide rapid and direct
access to records without the need for an index, while indexing is best suited for smaller
databases where quick read operations and ordered data retrieval are necessary.

4.7 Ordered Indices

The indices are usually sorted to make searching faster. The indices which are sorted are known
as ordered indices.
Example: Suppose we have an employee table with thousands of record and each of which is 10
bytes long. If their IDs start with 1, 2, 3....and so on and we have to search student with ID-543.

 In the case of a database with no index, we have to search the disk block from starting till
it reaches 543. The DBMS will read the record after reading 543*10=5430 bytes.
 In the case of an index, we will search using indexes and the DBMS will read the record
after reading 542*2= 1084 bytes which are very less compared to the previous case.

Primary Index
 If the index is created on the basis of the primary key of the table, then it is known as
primary indexing. These primary keys are unique to each record and contain 1:1 relation
between the records.
 As primary keys are stored in sorted order, the performance of the searching operation is
quite efficient.
 The primary index can be classified into two types: Dense index and Sparse index.

SRMMCET / CSE / DBMS 13

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Dense index
 The dense index contains an index record for every search key value in the data file. It
makes searching faster.
 In this, the number of records in the index table is same as the number of records in the
main table.
 It needs more space to store index record itself. The index records have the search key and
a pointer to the actual record on the disk.

Fig. 4.19 Dense Index

Sparse index
 In the data file, index record appears only for a few items. Each item points to a block.
 In this, instead of pointing to each record in the main table, the index points to the records
in the main table in a gap.

Fig. 4.20 Sparse Index

Clustering Index
 A clustered index can be defined as an ordered data file. Sometimes the index is created on
non-primary key columns which may not be unique for each record.
 In this case, to identify the record faster, we will group two or more columns to get the
unique value and create index out of them. This method is called a clustering index.
 The records which have similar characteristics are grouped, and indexes are created for
these group.

Secondary Index
 In the sparse indexing, as the size of the table grows, the size of mapping also grows.
These mappings are usually kept in the primary memory so that address fetch should be
faster.
 Then the secondary memory searches the actual data based on the address got from
mapping. If the mapping size grows then fetching the address itself becomes slower. In
this case, the sparse index will not be efficient. To overcome this problem, secondary
indexing is introduced.

SRMMCET / CSE / DBMS 14

 CS3492 – DATABASE MANAGEMENT SYSTEMS

4.8 B+ tree Index Files

A B+ tree is similar to a B-tree, the only difference is that their leaves are linked at the bottom.
Unlike B-tree, the nodes of the B+ tree do not store keys along with the pointers to the disk block. The
internal nodes contain only keys and the leaf nodes contain the keys along with the data pointers. All the
internal nodes are present at the leaves and are linked together and can be traversed just like a linked list.

B+ tree also removes one drawback of using B-tree. The internal nodes of the B-tree also store
keys along with data pointers which take more space and significantly reduce the number of keys that
can be stored at a node of the B-tree. Due to this the number of levels increases and searching time also
increases. Unlike this, the B+ tree can store more keys than the B-tree of the same level because of their
feature of storing pointers only at the leaf nodes. This contributes to storing more entries at fewer levels
in the B+ tree and lessens the searching time for a key.

Properties of B+ Tree
 The B+ tree is a balanced binary search tree. It follows a multi-level index format.
 In the B+ tree, leaf nodes denote actual data pointers. B+ tree ensures that all leaf nodes
remain at the same height.
 In the B+ tree, the leaf nodes are linked using a link list. Therefore, a B+ tree can support
random access as well as sequential access.

Structure of B+ Tree
In the B+ tree, every leaf node is at equal distance from the root node. The B+ tree is of the order
n where n is fixed for every B+ tree. It contains an internal node and leaf node.

Fig. 4.21 Structure of B+ Tree

Internal node
An internal node of the B+ tree can contain at least n/2 record pointers except the root node. At
most, an internal node of the tree contains n pointers.
Leaf node
The leaf node of the B+ tree can contain at least n/2 record pointers and n/2 key values. At most,
a leaf node contains n record pointer and n key values. Every leaf node of the B+ tree contains one block
pointer P to point to next leaf node.

SRMMCET / CSE / DBMS 15

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Example of B+ Tree

Fig. 4.22 Example for B+ tree

Searching a record in B+ Tree

Algorithm for Search in a B+ Tree:
Step 1: Start from the root node.
Step 2: If we find such a node, we move to the left child of that entry.
Step 3: We keep repeating this process until we reach a leaf node.
Step 4: We can do a linear search on the leaf node to find our key and the corresponding value.

Example:
Suppose we have to search 55 in the below B+ tree structure. First, we will fetch for the
intermediary node which will direct to the leaf node that can contain a record for 55.

So, in the intermediary node, we will find a branch between 50 and 75 nodes. Then at the end,
we will be redirected to the third leaf node. Here DBMS will perform a sequential search to find 55.

Fig. 4.23 searching a record in B+ Tree

B+ Tree Insertion
Suppose we want to insert a record 60 in the below structure. It will go to the 3rd leaf node after
55. It is a balanced tree, and a leaf node of this tree is already full, so we cannot insert 60 there. In this
case, we have to split the leaf node, so that it can be inserted into tree without affecting the fill factor,
balance and order.

The 3rd leaf node has the values (50, 55, 60, 65, 70) and its current root node is 50. We will split
the leaf node of the tree in the middle so that its balance is not altered. So we can group (50, 55) and
(60, 65, 70) into 2 leaf nodes.

SRMMCET / CSE / DBMS 16

 CS3492 – DATABASE MANAGEMENT SYSTEMS

If these two has to be leaf nodes, the intermediate node cannot branch from 50. It should have 60
added to it, and then we can have pointers to a new leaf node.

Fig 4.24 B+ tree insertion

B+ Tree Deletion
Suppose we want to delete 60 from the above example. In this case, we have to remove 60 from
the intermediate node as well as from the 4th leaf node too. If we remove it from the intermediate node,
then the tree will not satisfy the rule of the B+ tree. So we need to modify it to have a balanced tree.
After deleting node 60 from above B+ tree and re-arranging the nodes, it will show as follows:

Fig 4.25 B+ tree deletion

4.9 B tree Index Files

B Tree is a self-balancing tree data structure. It stores and maintains data in a sorted form where
the left children of the root are smaller than the root and the right children are larger than the root in
value. It makes searching efficient and allows all operations in logarithmic time. It allows nodes with
more than two children. B-tree is used for implementing multilevel indexing. Every node of the B-tree
stores the key-value along with the data pointer pointing to the block in the disk file containing that key.

Properties of B-Tree
 Every node has at most m children where m is the order of the B-Tree.
 A node with K children contains K-1 keys.
 Every non-leaf node except the root node must have at least ⌈m/2⌉ child nodes.
 The root must have at least 2 children if it is not the leaf node too.
 All leaves of a B-Tree stays at the same level.
 Unlike other trees, its height increases upwards towards the root, and insertion happens at the
leaf node.

SRMMCET / CSE / DBMS 17

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Example of a B-Tree of order 4

Fig. 4.26 Example for B Tree

Differences between B-Tree and B+ Tree

B-Tree B+ Tree
All internal nodes and leaf nodes contain Only leaf nodes contain data pointers along
data pointers along with keys. with keys, internal nodes contain keys only.
Duplicate keys are present in this, all internal
There are no duplicate keys.
nodes are also present at leaves.
Leaf nodes are not linked to each other. Leaf nodes are linked to each other.
All nodes are present at leaves, so sequential
Sequential access of nodes is not possible.
access is possible just like a linked list.
Searching for a key is slower. Searching is faster.
For a particular number of entries, the The height of the B+ tree is lesser than B-tree
height of the B-tree is larger. for the same number of entries.

4.10 Static Hashing

In static hashing, the resultant data bucket address will always be the same. That means if we
generate an address for EMP_ID =103 using the hash function mod (5) then it will always result in same
bucket address 3. Here, there will be no change in the bucket address. Hence in this static hashing, the
number of data buckets in memory remains constant throughout. In this example, we will have five data
buckets in the memory used to store the data.

Fig. 4.27 Static Hashing

SRMMCET / CSE / DBMS 18

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Operations of Static Hashing

 Searching a record - When a record needs to be searched, then the same hash function retrieves the
address of the bucket where the data is stored.
 Insert a Record - When a new record is inserted into the table, then we will generate an address for
a new record based on the hash key and record is stored in that location.
 Delete a Record - To delete a record, we will first fetch the record which is supposed to be
deleted. Then we will delete the records for that address in memory.
 Update a Record - To update a record, we will first search it using a hash function, and then the
data record is updated. If we want to insert some new record into the file but the address of a data
bucket generated by the hash function is not empty, or data already exists in that address. This
situation in the static hashing is known as bucket overflow. This is a critical situation in this
method.

To overcome this situation, there are various methods. Some commonly used methods are as
follows:
Open Hashing
When a hash function generates an address at which data is already stored, then the next bucket
will be allocated to it. This mechanism is called as Linear Probing.
For example: suppose R3 is a new address which needs to be inserted, the hash function
generates address as 112 for R3. But the generated address is already full. So the system searches next
available data bucket, 113 and assigns R3 to it.

Fig. 4.28 Open Hashing

2. Close Hashing
When buckets are full, then a new data bucket is allocated for the same hash result and is linked
after the previous one. This mechanism is known as Overflow chaining.
For example: Suppose R3 is a new address which needs to be inserted into the table, the hash
function generates address as 110 for it. But this bucket is full to store the new data. In this case, a new
bucket is inserted at the end of 110 buckets and is linked to it.

SRMMCET / CSE / DBMS 19

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Fig. 4 .29 Closing Hashing

4.11 Dynamic Hashing

The dynamic hashing method is used to overcome the problems of static hashing like bucket
overflow. In this method, data buckets grow or shrink as the records increases or decreases. This method
is also known as extendable hashing method. This method makes hashing dynamic, i.e., it allows
insertion or deletion without resulting in poor performance.
How to search a key
 First, calculate the hash address of the key.
 Check how many bits are used in the directory, and these bits are called as i.
 Take the least significant i bits of the hash address. This gives an index of the directory.
 Now using the index, go to the directory and find bucket address where the record might be.
How to insert a new record
 Firstly, you have to follow the same procedure for retrieval, ending up in some bucket.
 If there is still space in that bucket, then place the record in it.
 If the bucket is full, then we will split the bucket and redistribute the records.
For example:
Consider the following grouping of keys into buckets, depending on the prefix of their hash
address:

The last two bits of 2 and 4 are 00. So it will go into bucket B0. The last two bits of 5 and 6 are
01, so it will go into bucket B1. The last two bits of 1 and 3 are 10, so it will go into bucket B2. The last
two bits of 7 are 11, so it will go into B3.

SRMMCET / CSE / DBMS 20

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Fig. 4 .30 Dynamic Hashing

Insert key 9 with hash address 10001 into the above structure:
 Since key 9 has hash address 10001, it must go into the first bucket. But bucket B1 is full, so it
will get split.
 The splitting will separate 5, 9 from 6 since last three bits of 5, 9 are 001, so it will go into
bucket B1, and the last three bits of 6 are 101, so it will go into bucket B5.
 Keys 2 and 4 are still in B0. The record in B0 pointed by the 000 and 100 entry because last
two bits of both the entry are 00.
 Keys 1 and 3 are still in B2. The record in B2 pointed by the 010 and 110 entry because last
two bits of both the entry are 10.
 Key 7 are still in B3. The record in B3 pointed by the 111 and 011 entry because last two bits
of both the entry are 11.

Fig. 4.31 Insertion in Dynamic Hashing

Advantages of dynamic hashing

 In this method, the performance does not decrease as the data grows in the system. It simply
increases the size of memory to accommodate the data.
 In this method, memory is well utilized as it grows and shrinks with the data. There will not
be any unused memory lying.
 This method is good for the dynamic database where data grows and shrinks frequently.

SRMMCET / CSE / DBMS 21

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Disadvantages of dynamic hashing

 In this method, if the data size increases then the bucket size is also increased. These
addresses of data will be maintained in the bucket address table. This is because the data
address will keep changing as buckets grow and shrink. If there is a huge increase in data,
maintaining the bucket address table becomes tedious.
 In this case, the bucket overflow situation will also occur. But it might take little time to
reach this situation than static hashing.

4.12 Query Processing Overview

Query Processing is the activity performed in extracting data from the database. In query
processing, it takes various steps for fetching the data from the database.
The steps involved are:
1) Parsing and translation
2) Evaluation
3) Optimization
The query processing works in the following way:

Parsing and Translation

As query processing includes certain activities for data retrieval. Initially, the given user queries
get translated in high-level database languages such as SQL. It gets translated into expressions that can
be further used at the physical level of the file system. After this, the actual evaluation of the queries and
a variety of query -optimizing transformations and takes place. Thus before processing a query, a
computer system needs to translate the query into a human-readable and understandable language.

Consequently, SQL or Structured Query Language is the best suitable choice for humans. But, it
is not perfectly suitable for the internal representation of the query to the system. Relational algebra is
well suited for the internal representation of a query.

The translation process in query processing is similar to the parser of a query. When a user
executes any query, for generating the internal form of the query, the parser in the system checks the
syntax of the query, verifies the name of the relation in the database, the tuple, and finally the required
attribute value. The parser creates a tree of the query, known as 'parse-tree.' Further, translate it into the
form of relational algebra. With this, it evenly replaces all the use of the views when used in the query.
Thus, we can understand the working of a query processing in the below described diagram:

SRMMCET / CSE / DBMS 22

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Fig. 4.32 Steps in query processing system

Suppose a user executes a query. As we have learned that there are various methods of extracting
the data from the database. In SQL, a user wants to fetch the records of the employees whose salary is
greater than or equal to 10000. For doing this, the following query is undertaken:
select emp_name from Employee where salary>10000;

Thus, to make the system understand the user query, it needs to be translated in the form of
relational algebra. We can bring this query in the relational algebra form as:
σsalary>10000 (πsalary (Employee))
πsalary (σsalary>10000 (Employee))

After translating the given query, we can execute each relational algebra operation by using
different algorithms. So, in this way, a query processing begins its working.

Evaluation
For this, with addition to the relational algebra translation, it is required to annotate the translated
relational algebra expression with the instructions used for specifying and evaluating each operation.
Thus, after translating the user query, the system executes a query evaluation plan.

Query Evaluation Plan

 In order to fully evaluate a query, the system needs to construct a query evaluation plan.
 The annotations in the evaluation plan may refer to the algorithms to be used for the particular
index or the specific operations.
 Such relational algebra with annotations is referred to as Evaluation Primitives. The evaluation
primitives carry the instructions needed for the evaluation of the operation.
 Thus, a query evaluation plan defines a sequence of primitive operations used for evaluating a
query. The query evaluation plan is also referred to as the query execution plan.

SRMMCET / CSE / DBMS 23

 CS3492 – DATABASE MANAGEMENT SYSTEMS

 A query execution engine is responsible for generating the output of the given query. It takes the
query execution plan, executes it, and finally makes the output for the user query.

Optimization
 The cost of the query evaluation can vary for different types of queries. Although the system is
responsible for constructing the evaluation plan, the user does need not to write their query
efficiently.
 Usually, a database system generates an efficient query evaluation plan, which minimizes its cost.
This type of task performed by the database system and is known as Query Optimization.
 For optimizing a query, the query optimizer should have an estimated cost analysis of each
operation. It is because the overall operation cost depends on the memory allocations to several
operations, execution costs, and so on.
Finally, after selecting an evaluation plan, the system evaluates the query and produces the
output of the query.

4.13 Algorithms for Selection, Sorting and join operations

Algorithm for Selection Operation
For selection operation, the file scan is an important activity. Basically file scan is a based on
searching algorithms. These searching algorithms locate and retrieve the records that fulfills a selection
condition. Let us discuss various algorithms used for SELECT Operation based in file scan

Algorithm A1: Linear Search

• Scan each file block and test all records to see whether they satisfy the selection condition
Cost - br block transfers + 1 seek
Where, br denotes number of blocks containing records from relation r
• If selection is on a key attribute, can stop on finding record
Cost = (br/2) block transfers + 1 seek

Advantages of Linear Search

 Linear search works even-if there is no selection condition specified.
 For linear search, there is no need to have records in the file in ordered form.
 Linear search works regardless of indices.
Algorithm A2: Binary Search
Applicable if selection is an equality comparison on the attribute on which file is ordered.
Assume that the blocks of a relation are stored contiguously. Cost estimate is nothing but the number of
disk blocks to be scanned.

SRMMCET / CSE / DBMS 24

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Cost of locating the first tuple by a binary search on the blocks = [log2(br)] x (tT + tS)
Where, br denotes number of blocks containing records from relation r, tT is average time
required to transfer a block of data, in seconds and tS is average block access time, in second.
If there are multiple records satisfying the selection add transfer cost of the number of blocks
containing records that satisfy selection condition.

Algorithm for Sorting Operation

External sorting
In external sorting, the data stored on secondary memory is part by part loaded into main
memory, sorting can be done over there. The sorted data can be then stored in the intermediate files.
Finally these intermediate files can be merged repeatedly to get sorted data. Thus huge amount of data
can be sorted using this technique.

The external merge sort is a technique in which the data is loaded in intermediate files. Each
intermediate file is sorted independently and then combined or merged to get the sorted data.

For example:
Consider that there are 10,000 records that has to be sorted.
Clearly we need to apply external sorting method. Suppose main memory has a capacity to store
500 records in blocks, with each block size of 100 records. The sorted 5 blocks (i.e. 500 records) are
stored in intermediate file.
This process will be repeated 20 times to get all the records sorted in chunks. In the second step,
we start merging a pair of intermediate files in the main memory to get output file.

Algorithm for Join Operation

JOIN operation is the most time consuming operation to process. There are Several different
algorithms to implement joins. They are,
1. Nested-loop join
2. Block nested-loop join
3. Indexed nested- loop join
4. Merge-join
5. Hash-join
Choice of a particular algorithm is based on cost estimate.

SRMMCET / CSE / DBMS 25

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Algorithm For Nested Loop Join

This algorithm is for computing

Let, r is called the outer relation and s the inner relation of the join.

for each tuplet tr in r do begin

for each tuple ts in s do begin
test pair (tr,ts) to see if they satisfy the join condition
if 0 is satisfied, then, add (tr, ts) to the result,
end
end

This algorithm requires no indices and can be used with any kind of join condition. It is
expensive since it examines every pair of tuples in the two relations. In the worst case, if there is enough
memory only to hold one block of each relation, the estimated cost is
nr × bs + br block transfers, plus nr + br seeks

If the smaller relation fits entirely in memory, use that as the inner relation
Reduces cost to br + bs block transfers and 2 seeks

For example: Assume the query CUSTOMERS ORDERS (with join attribute only being CName)
Number of records of customer: 10000 order: 5000
Number of blocks of customer: 400 order: 100

Formula Used:
(1) nr × bs + br block transfers,
(2) nr + br seeks
r is outer relation and s is inner relation.

With order as outer relation:

nr= 5000 bs= 400, br = 100
5000 × 400 + 100 = 2000100 block transfers and
5000+100 5100 seeks
With customer as the outer relation:
nr =10000, bs =100, br =400
10000 x 100 + 400 = 1000400 block transfers and
10000+400=10400 seeks
If smaller relation (order) fits entirely in memory, the cost estimate will be:
br+bs =500 block transfers

SRMMCET / CSE / DBMS 26

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Algorithm For Block Nested Loop Join

Variant of nested-loop join in which every block of inner relation is paired with every block of
outer relation. This algorithm is for computing
Let, r is called the outer relation and s the inner relation of the join.
for each block Br of r do
for each block Bs of s do
for each tuple tr in Br do begin
for each tuple ts in Bs do begin
test pair (tr, ts) to see if they satisfy the join condition
if Ө is satisfied, then, add (tr, ts) to the result.
end
end
end
end

Worst case estimate: br × bs + br block transfers + 2 × br seeks

Each block in the inner relation s is read once for each block in the outer relation. Best case: br+ bs
block transfers + 2 seeks

Merge Join
In this operation, both the relations are sorted on their join attributes. Then merged these sorted
relations to join them. Only equijoin and natural joins are used. The cost of merge join is,
br + bs block transfers + [br /bb] + [bs/bb] seeks + the cost of sorting, if relations are unsorted.

Hash Join
In this operation, the hash function h is used to partition tuples of both the relations. h maps A
values to {0, 1, ..., n}, where A denotes the attributes of r and s used in the join.
Cost of hash join is:
3(br+bs) + 4 x n block transfers + 2([br /bb] + [bs/bb]) seeks
If the entire build input can be kept in main memory no partitioning is required Cost estimate goes down
to br+bs.

SRMMCET / CSE / DBMS 27

 CS3492 – DATABASE MANAGEMENT SYSTEMS

4.14 Query optimization using Heuristics

In DBMS, heuristic optimization is a procedure that is aimed at the rapid exploration of almost
all execution plans in a quick and efficient way. Unlike the exhaustive optimization methods that
postulate all the feasible plan alternatives and then apply them to the optimization process, the heuristic
optimization rules, with the use of derived empirical knowledge and approximate algorithms, tend to
speed up the process. Through the implementation of heuristics, the DBMS optimizers spur efficient
execution query plan convergence while also minimizing computational cost.

Key Components of Heuristic Optimization

1. Cost-Based Heuristics:
Cost estimation is a critical process in DBMS, where heuristic strategies are applied. We do this
in the future related to the query execution plan of the selection cost approach using statistical data
distribution, system parameters, and the physical machine characteristics. e.g., line-based cost
estimation and cardinality supposition are principally used to determine the cost correlated with each
plan next. Through utilization of service plans with lower estimated overheads by heuristic optimization,
budgets are managed judiciously, and the overall query is executed with the best performance.

2. Join Order Heuristics:

When there are queries involving multiple tables, deciding on what optimum join order to use
becomes necessary in order to decrease request execution time. Join ordering heuristics can include the
usage of greedy algorithms, dynamic programming approaches, and others. These methods assist the
system in exploring different join orders and selecting the most efficient one. Furthermore, in cardinality
estimation techniques the scale of output temporarily created by intermediate join results is predicted so
that better judgments can be made regarding the order of joins.

3. Index Selection Heuristics:

In addition to extraction and load of data, indexes have the ability to accelerate query processing
by making data retrieval fast. Index selection heuristics encompass the usage of the heuristic
optimization which can choose the most influential indexes for query execution among them. These
factors, like query predicates, selectivity, and the fuel activation cost, are considered in the optimal
indexing strategy. Running queries based on index selection heuristics, DBMS optimizers lead to better
performance and a timely response decreasing overall.

4. Query Rewriting Heuristics:

The use of heuristic optimization is focused on the rewriting and transformation of the
statements into semantically corresponding ones that are suitable for optimal query execution. The rule
of adaptation to domain knowledge is used for the purpose of query structure suggestion and

SRMMCET / CSE / DBMS 28

 CS3492 – DATABASE MANAGEMENT SYSTEMS

performance improvement. Methods like operation shift and query splitting allow for compressed data
flow and improved query running strategies in this manner.

Challenges and Considerations

While heuristic optimization offers significant advantages in terms of speed and efficiency, it is
not without its challenges and considerations:

 Sub-optimality: Although the heuristic approaches sometimes can yield plans with subpar
query execution compared to the exhaustive precision optimization, they again remove the
need to follow exact heuristic rules and use only heuristics that are broadly applicable, so
they can be useful for many queries.
 Cost Estimation Accuracy: The outcome of heuristic optimization, however, relies on the
accuracy of the cost estimations which in turn may be affected by the scalability of the data,
query complexity or system dynamics, among other factors.
 Trade-offs: Heuristic optimization introduces a balance between optimality and efficiency,
ramifying the problem of proportionality between speed and quality of the planning.

4.15 Cost Estimation

There are multiple possible evaluation plans for a query, and it is important to be able to
compare the alternatives in terms of their estimated cost and choose the best plan.
There are many factors that contribute to query cost are -
• Disk access
• CPU time to execute the query
• Cost of communication in distributed and parallel database system.
• The cost of access data from disk is an important cost.

Normally disk access is relatively slow as compared to in-memory operations. Moreover, the
CPU speed is much faster than the disk speed. Hence the time spent in disk is relatively dominant factor
in query execution. Computing CPU access time is comparatively harder, hence we will not consider the
CPU time to execute the query.
Similarly the cost of communication does not matter for simple large databases ero present in the
centralized database system.
Typically disk access is the predominant cost and is also relatively easy to estimate, taking into
account :
 Number of seeks × average-seek-cost
 Number of blocks read × average-block-read-cost
 Number of blocks written × average-block-write-cost

SRMMCET / CSE / DBMS 29

 CS3492 – DATABASE MANAGEMENT SYSTEMS

Cost to write a block is greater than cost to read a block because data is read back after being
written to ensure that the write was successful. We use number of block transfers from disk and number
of disk seeks to estimate the Query cost.
Let
- b be the number to blocks
- S be the number of Seeks
- tT is average time required to transfer a block of data, in seconds
- tS is average block access time in seconds.
Then query cost can be computed using following formula b*tr+S*ts

SRMMCET / CSE / DBMS 30