Module Title: Database Monitoring and
Administration
Module code: EIS WDDBA4 M05 1123
Nominal duration: 40 Hours
Prepared by: Haile Sitotaw
May, 2025
Learning outcome
1 Database startup
Database
2 management
Database Access
3 Management
Principles of Database, System
Configuration, and Monitoring
Principles of Databases
Definitions:
01
1.Fundamental guidelines that shape the logical structure and
operation of a database system.
2.Rules ensuring accurate data storage, retrieval, and integrity within
databases.
3.Best practices followed for designing relational schemas and
enforcing data consistency.
Examples:
•Designing a database where each customer record must have a
unique CustomerID.
•Using foreign keys to enforce referential integrity between Orders
and Customers.
Data Integrity
Term: Data Integrity
Definitions:
1. The accuracy and consistency of stored data over its lifecycle.
2. The assurance that data is not altered or tampered with in an unauthorized manner.
3. A condition in which the data in a database is valid, accurate, and reliable.
Examples:
A system prevents saving a record with an invalid date of birth.
Ensuring a student's grade is within an acceptable range (0–100).
Entity Integrity
Definitions:
1. Ensures that each table has a primary key and that the key is unique and
not null.
2. A rule that prohibits duplicate or missing primary keys in a relational
table.
3. A principle enforcing each record's uniqueness within a table.
Examples:
A Student_ID column defined as the primary key in the Students table.
Not allowing the insertion of a new record without a unique Product_ID.
Referential Integrity
Definitions:
1. A rule stating that foreign keys must reference existing primary key
values in another table.
2. Maintains logical relationships between tables by enforcing consistency.
3. Ensures child records do not exist without a corresponding parent record.
Examples:
A Sales table referencing a valid Employee_ID from the Employees table.
Deleting a customer is blocked if they have linked orders in the Orders table.
Normalization
Definitions:
1. The process of structuring a relational database to reduce redundancy.
2. Dividing large tables into smaller, related tables and defining
relationships.
3. A design technique to improve data integrity and simplify maintenance.
Examples:
Creating separate Customers, Orders, and Products tables.
Moving repeated address fields to a separate Addresses table linked by
Address_ID.
Consistency
Definitions:
1. The principle that data must always meet all defined rules and
constraints.
2. A database state that adheres to all integrity rules after a transaction.
3. Ensures data correctness and validity through constraints and logic.
Examples:
Ensuring Order_Total equals Quantity * Unit_Price.
A user can’t be registered without a valid email address format.
ACID Properties
Definitions:
Atomicity: A transaction must be fully completed or not at all.
Consistency: Transactions must leave the database in a valid state.
Isolation: Transactions must not interfere with each other.
Durability: Once committed, changes persist even after failure.
Examples:
Bank transfers update both accounts together or none at all.
A power outage doesn’t undo a committed database update.
Data Models
Definitions:
1. Conceptual frameworks that determine how data is stored and accessed.
2. Abstract representations of database structure and relationships.
3. Schemas defining tables, fields, types, and relationships.
Examples:
Relational model: Data stored in tables (SQL).
Document model: Data stored in JSON/BSON (MongoDB).
SQL (Structured Query Language)
Definitions:
1. A standard language for managing and manipulating relational
databases.
2. Used for performing CRUD operations: Create, Read, Update, Delete.
3. A declarative language to define database structure and interact with
data.
Examples:
SELECT * FROM Students WHERE Grade > 80;
INSERT INTO Orders (OrderID, CustomerID) VALUES (101, 12);
Security
Definitions:
1. Techniques to prevent unauthorized access, alteration, or destruction of data.
2. Includes authentication, authorization, encryption, and auditing.
3. A set of policies and controls to safeguard sensitive information.
Examples:
Role-based access allowing only managers to delete data.
Encrypted database connections using SSL/TLS.
Concurrency Control
Definitions:
1. Mechanism to ensure correct results for simultaneous database
operations.
2. Prevents conflicts when multiple users access the database concurrently.
3. Maintains transaction isolation and prevents dirty reads, lost updates.
Examples:
A row lock during an update to prevent two users editing the same
order.
Timestamp ordering to manage transaction order.
Scalability
Definitions:
1. The ability of a database to grow in capacity and performance with load.
2. How well a system handles increased users, data, or queries.
3. Flexibility to add resources like servers or storage to meet demand.
Examples:
Using sharding to distribute data across multiple databases.
Adding more memory to support larger cache and faster query processing.
Backup and Recovery
Definitions:
1. Backup: Creating data copies for safekeeping.
2. Recovery: Restoring data after corruption, deletion, or failure.
3. A business continuity strategy for database resilience.
Examples:
Taking a full backup of a sales database every night.
Recovering from backup after a server crash.
Data Independence
Definitions:
1. The ability to change the database schema without changing application
logic.
2. Logical and physical independence from application programs.
3. Flexibility to modify storage or structure without affecting users.
Examples:
Adding a new column to a table without rewriting application code.
Changing storage from HDD to SSD without impacting queries.
System Configuration
Definitions:
1. The process of preparing software, hardware, and parameters for
deployment.
2. Settings that define how a database system starts and operates.
3. Customizing system behavior according to workload needs.
Examples:
Tuning cache size to handle large data sets.
Setting up Oracle DB with optimized memory allocation.
Hardware and OS Configuration
Definitions:
1. Aligning database needs with CPU, memory, disk, and network specifications.
2. OS settings such as file system type, memory paging, and kernel parameters.
3. Adjustments that improve performance and availability.
Examples:
Using SSDs for better I/O speed.
Setting Linux kernel parameters for PostgreSQL optimization
Database Installation and Memory Allocation
Definitions:
1. Correct setup and deployment of database software.
2. Assigning memory to caches, buffers, and connection pools.
3. Ensures efficient database operation.
Examples:
Installing PostgreSQL 15 on Ubuntu with secure defaults.
Allocating 10GB to InnoDB buffer pool in MySQL.
Network and Authentication Configuration
Definitions:
1. Setup of protocols, ports, and security rules for access.
2. Authentication defines how users prove identity (password, certificate).
3. Ensures secure database communication.
Examples:
Enabling TCP/IP with SSL encryption.
Using Active Directory for user authentication.
Startup Parameters and Logging
Definitions:
1. Startup parameters include cache size, connection limit, recovery
options.
2. Logging helps in debugging and performance monitoring.
3. Defines database behavior during runtime.
Examples:
Setting max_connections=200 in PostgreSQL.
Reviewing logs for crash recovery analysis.
Backup and Monitoring Configuration
Definitions:
1. Scheduling automated backups and managing their retention.
2. Monitoring tools collect metrics and alert on issues.
3. Ensures ongoing system reliability.
Examples:
Setting up cron jobs for daily backups.
Using Zabbix to monitor DB server CPU and memory.
Documentation
Définitions:
1. Written records of system configuration, architecture, and changes.
2. Helps in troubleshooting, audits, and system upgrades.
3. Includes setup steps, parameter settings, and user roles.
Examples:
A Word document listing SQL Server installation settings.
A spreadsheet tracking schema changes.
Monitoring Irregularities - Overview
Definitions:
1. The process of identifying unexpected behavior or system faults.
2. Ensures early detection of issues before they cause failure.
3. Focuses on anomalies in performance or security.
Examples:
Noticing sudden spike in query execution time.
Detecting memory leaks via logs.
Log and Performance Monitoring
Definitions:
1. Analyzing system logs to trace warnings and errors.
2. Monitoring CPU, memory, disk usage to optimize performance.
3. Key for identifying bottlenecks and failures.
Examples:
Analyzing logs to find failed login attempts.
Using Grafana dashboard to view query performance trends.
Alerts, Health Checks, and Security Audits
Definitions:
1. Alerts: Notifications for predefined issues (CPU spike, failed backup).
2. Health Checks: Periodic review of system metrics.
3. Security Audits: Review of access control and logins.
Examples:
Email alert when disk usage > 90%.
Weekly script checking user permissions.
Query and User Activity Monitoring
Definitions:
1. Query monitoring identifies inefficient queries and delays.
2. User activity logs reveal access patterns and anomalies.
3. Helps in tuning performance and detecting misuse.
Examples:
Logging all queries that take > 5 seconds.
Tracking IP addresses used for logins.
Data Consistency and Backup Verification
Definitions:
1. Regular validation of data against expected rules.
2. Backup verification ensures backups can be restored correctly.
3. Prevents corruption and data loss.
Examples:
Running daily checks for NULLs in mandatory columns.
Restoring a test copy from backup every Sunday.
Data Consistency and Backup Verification
Definitions:
1. Regular validation of data against expected rules.
2. Backup verification ensures backups can be restored correctly.
3. Prevents corruption and data loss.
Examples:
Running daily checks for NULLs in mandatory columns.
Restoring a test copy from backup every Sunday.
Database
2 management
2.1. Data Dictionary Compilation and Structure Verification
What is a Data Dictionary?
Definition :
A data dictionary is a centralized repository providing metadata about data within a
database.
A collection of descriptions of data objects/items in a data model for programmers and
users.
Information describing database contents, format, structure, and relationships to control
access and manipulation.
Purpose and Contents of Data Dictionary
Definitions of schema objects
Space allocation and usage
Default column values
Integrity constraints
Auditing info (access, updates)
User roles and privileges
Descriptions of users and responsibilities
Example of Data Dictionary Entry
Table: Customers
Fields: CustomerID (int, PK), Name (varchar), Email
(varchar), Phone (varchar)
Constraints: CustomerID is primary key, Email unique
Usage: Helps developers understand data types and
constraints
Advantages of a Data Dictionary
Simplifies understanding of table structures
Helps new users/admins quickly learn schema
Centralized documentation of database design
Improves communication among developers
What is Structure Verification?
Definition
Process of verifying actual database schema against the
documented data dictionary.
Ensures database structure aligns with intended design.
Confirms all changes in database are updated in the data
dictionary for consistency.
Why Structure Verification is Important
Maintains database design integrity
Detects discrepancies between documentation and actual schema
Prevents errors due to outdated or incorrect documentation
Ensures smooth database maintenance and upgrades
Example of Structure Verification
Data dictionary says "Orders" table has FK to
"Customers" table
Verify FK relationship exists in the database schema
Check data types and constraints match documentation
Example of Structure Verification
Data dictionary says "Orders" table has FK to
"Customers" table
Verify FK relationship exists in the database schema
Check data types and constraints match documentation
Structure Verification Types
Consistency Checks: Verify documentation matches database
Data Type Verification: Check field types match dictionary
Constraint Verification: Verify PK, FK, Unique, Check constraints
Documentation Updates: Reflect changes in data dictionary
Version Control: Track changes over time
Types of Data Integrity Constraints
1. Entity Integrity
2. Referential Integrity
3. Domain Integrity
4. User Defined Integrity
Entity Integrity
Primary key must be unique and not null
Ensures every record can be uniquely identified
Example 1: CustomerID in Customers table is PK, unique and not null
Example 2: ProductID in Products table serves as unique identifier
Referential Integrity
Foreign key must reference a valid primary key or be null
Prevents orphaned records and inconsistent relationships
Example 1: Order.CustomerID FK references Customers.CustomerID
Example 2: Invoice.CustomerID FK references Customers.CustomerID or is null if
no customer linked
Domain Integrity
All columns must contain values within a defined domain (valid data type and
range)
Example 1: Age column must be integer between 0 and 120
Example 2: Email column must follow proper email format.
Referential Integrity
Foreign key must reference a valid primary key or be null
Prevents orphaned records and inconsistent relationships
Example 1: Order.CustomerID FK references Customers.CustomerID
Example 2: Invoice.CustomerID FK references Customers.CustomerID or is null if
no customer linked
Domain Integrity
All columns must contain values within a defined domain (valid data type and
range)
Example 1: Age column must be integer between 0 and 120
Example 2: Email column must follow proper email format.
User Defined Integrity
Business-specific rules not covered by other integrity types
Implemented via constraints, stored procedures, triggers
Example 1: On deletion of a sales record, write to audit table automatically
Example 2: Discount applied only if customer is premium member
Indexes: What and Why?
Definition
Index is a separate physical data structure to speed up data retrieval.
Used to quickly locate data without scanning entire table.
Improves query performance at the cost of additional storage and write overhead.
Why Use Indexes?
Enforce uniqueness on columns
Speed up query response time
Support quick lookups, joins, and sorting
Optimizes large database operations
Deciding Which Columns to Index
Frequently used in WHERE clauses
Frequently used in JOIN operations
Frequently used in ORDER BY clauses
Columns with mostly unique values
Avoid small tables or columns with many duplicates
Creating Indexes
Use SQL command: CREATE INDEX index_name ON table_name (column1 ASC|DESC, ...)
Can be single or multi-column
Improves ordered data access
Deciding Which Columns to Index
Frequently used in WHERE clauses
Frequently used in JOIN operations
Frequently used in ORDER BY clauses
Columns with mostly unique values
Avoid small tables or columns with many duplicates
Creating Indexes
Use SQL command: CREATE INDEX index_name ON table_name (column1 ASC|DESC, ...)
Can be single or multi-column
Improves ordered data access
Deleting Indexes
Use SQL command: DROP INDEX index_name ON table_name
Can be deleted via GUI tools like Object Explorer
Keep database optimized by removing unused indexes
Viewing and Editing Indexes
Use database tools (Object Explorer) to view indexes
Modify indexes via properties or SQL commands
Adjust columns and index options as needed
Multiple-Field Keys (Composite Keys)
Definition
A key made up of two or more columns to uniquely identify a record.
Used when no single column is unique by itself.
Helps maintain entity integrity with combined uniqueness.
Example of Composite Key
Table: OrderDetails
Composite PK: (OrderID, ProductID)
Ensures uniqueness of product in each order
Example 2: Enrollment table with (StudentID, CourseID) composite key
Summary
Data dictionary documents database schema and metadata
Structure verification ensures consistency between dictionary
and actual database
Integrity constraints maintain data accuracy and business rules
Indexes improve performance by speeding up data retrieval
Composite keys ensure unique identification via multiple fields.
2.4. Lock options for the database monitoring
Contents
Lock Options for Database Monitoring
Backup Verification & Retrieval
Storage Monitoring & Resizing
Data Updates per Guidelines
2.4. Lock options for the database monitoring
Types of Locks
1. Shared Locks
Definition: Allows concurrent read access but blocks writes.
Example 1: Multiple users querying a product catalog simultaneously.
Example 2: Reporting tools reading financial data without blocking transactions.
2. Exclusive Locks
Definition: Grants exclusive write access; blocks other reads/writes.
Example 1: Admin updating a user’s salary record.
Example 2: Batch process modifying inventory levels.
Monitoring Locks
Definition: Tracks active locks to prevent contention.
Example 1: SQL Server’s sys.dm_tran_locks view.
Deadlock Detection & Timeout
1. Deadlock Detection
Definition: Identifies cyclic dependencies where transactions wait indefinitely.
Example 1: Transaction A locks Table 1; Transaction B locks Table 2; both wait for the other.
Example 2: DBMS automatically kills one transaction to resolve deadlock.
Isolation Levels & Lock Escalation
1. Isolation Levels
Definition: Controls transaction visibility (e.g., dirty reads, phantom reads).
Example 1: READ COMMITTED prevents dirty reads but allows non-repeatable reads.
Example 2: SERIALIZABLE ensures strict isolation but reduces concurrency.
2. Lock Escalation
Definition: Converts many row-level locks to a single table-level lock.
Example 1: SQL Server escalates locks to reduce memory usage.
Example 2: Disabling escalation for high-concurrency OLTP systems.
Backup Retrieval Process
1. Backup Logs & Timestamps
Definition: Logs confirm successful backups with timestamps.
Example 1: AWS RDS logs backup completion times.
Example 2: Nagios alerts for missed backups.
2. Retention Policy Compliance
Definition: Ensures backups align with legal/organizational rules.
Example 1: 7-year retention for financial data (SOX compliance).
Example 2: Auto-deletion of backups after 30 days.
Storage Monitoring & Resizing
Storage Monitoring
1. Auto-Growth Settings
Definition: Automatically expands database files when space runs low.
Example 1: SQL Server file groups set to grow by 10% increments.
Example 2: Monitoring alerts for frequent auto-growth events.
2. Partitioning
Definition: Splits large tables into manageable chunks.
Example 1: Partitioning sales data by year.
Example 2: Faster queries on recent partitions while archiving old data.
Storage Optimization
1. Data Archiving
Definition: Moves old data to cheaper storage.
Example 1: Moving 5-year-old records to cold storage (e.g., AWS Glacier).
Example 2: Compliance archiving for audit trails.
2. Index Optimization
Definition: Rebuilds fragmented indexes to save space.
Example 1: Weekly REINDEX jobs in PostgreSQL.
Example 2: Removing unused indexes to free 20% storage.
Compliance & Testing
1. Staging Environment Tests
Definition: Validates updates in a non-production clone.
Example 1: Testing a price increase in staging before production.
Example 2: Data migration validation.
2. Regulatory Compliance
Definition: Ensures updates follow laws (e.g., GDPR, CCPA).
Example 1: Anonymizing data before testing.
Example 2: Approval workflows for sensitive data changes.
Database Access
3 Management
Access Privilege Management (3.1)
Definition :
Access privilege management refers to the process of assigning, modifying,
or revoking access rights to database resources based on user roles and
organizational policies.
It ensures that users have appropriate permissions to perform their tasks
while protecting sensitive data.
Database Access
3 Management
Allocate Access Privileges for New Users
Definition: Granting appropriate access rights to new employees
based on their job roles.
Example 1:
GRANT SELECT ON database.table TO 'new_user'@'localhost';
Example 2:
GRANT INSERT, UPDATE ON database.table2 TO 'new_user'@'localhost';
Database Access
3 Management
Remove Access Privileges for Departing Users
Definition: Revoking access from users who have left the
organization.
Example 1:
REVOKE ALL PRIVILEGES ON database.* FROM 'departing_user'@'localhost';
Example 2:
DROP USER 'departing_user'@'localhost';
Database Access
3 Management
Adjust Access Privileges for Role Changes
Definition: Modifying access rights when a user changes
roles within the organization.
Example 1:
REVOKE SELECT ON database.table FROM 'user'@'localhost';
Example 2:
GRANT UPDATE ON database.table TO 'user'@'localhost';
Database Access
3 Management
Periodic Access Review and Adjustment
Definition: Reviewing user access periodically to align with
job responsibilities.
Example 1:
REVOKE DELETE ON database.table FROM
'user_to_review'@'localhost';
Example 2: Remove privileges after audit reveals over-access.
Access Privileges Based on User Status
Definition: Granting or restricting access depending on
whether a user is active, on probation, or inactive.
Example 1: IF user_status = 'active' THEN
GRANT SELECT ON database.table TO 'user';
END IF;
Example 2:
REVOKE ALL PRIVILEGES ON database.table FROM 'inactive_user';
User Access Termination
Definition: Ensuring all access is removed when a user exits.
Example 1:
REVOKE ALL PRIVILEGES ON *.* FROM 'former_user'@'localhost';
Example 2:
DROP USER 'former_user'@'localhost';
Access Privileges for Temporary Roles
Definition: Granting time-limited access to users in project
roles.
Example 1:
GRANT SELECT, INSERT ON project_db.* TO 'temp_user';
Example 2: Schedule script to revoke access after a deadline.
Access Based on User Approval
Definition: Implementing a system where privilege changes
need managerial approval.
Example 1:
GRANT SELECT ON database.table TO 'approved_user';
Example 2: Use approval workflows in IAM tools.
Access Audit and Logging
Definition: Access Audit and Logging refers to the systematic process of
recording and reviewing all user activities related to access control within a system
or database.
Access Audit
• Definition: An access audit is the process of reviewing records of who accessed
what data, when, and what actions they performed.
• Purpose:
• Identify unauthorized access.
Access Audit and Logging
Access Logging
• Definition: The act of recording access events (login, data read/write, privilege
changes) into a secure log file or database.
• Key Log Information:
• Timestamp
• Username or process ID
• Accessed resource
• Action type (e.g., SELECT, UPDATE, DELETE)
• Result (Success/Failure)
• Benefits:
• Enables traceability.
• Supports incident investigation.
• Helps in forensic analysis after security breaches.
Access Audit and Logging
Access Logging
• Definition: The act of recording access events (login, data read/write, privilege
changes) into a secure log file or database.
• Key Log Information:
• Timestamp
• Username or process ID
• Accessed resource
• Action type (e.g., SELECT, UPDATE, DELETE)
• Result (Success/Failure)
• Benefits:
• Enables traceability.
• Supports incident investigation.
• Helps in forensic analysis after security breaches.
Focus on Security-Relevant Logs
Definition: Logs showing authentication attempts and access
to sensitive resources.
Example 1: /var/log/auth.log
Example 2: Cloud-based security logs (AWS CloudTrail).
Identify Suspicious Patterns
Definition: Recognizing abnormal behaviors such as failed login
bursts.
Means analyzing server logs or system activity to detect unusual or abnormal
behavior that may indicate a potential security threat or unauthorized access
attempt.
Example 1: 5 failed logins within a minute.
Example 2: Login from foreign country IPs.
Automate Log Analysis
Definition: Using tools to analyze logs and generate alerts
automatically.
Example 1: Use Splunk for real-time alerting.
Example 2: Employ ELK stack (Elasticsearch, Logstash,
Kibana).
Access Audit and Logging
Access Logging
• Definition: The act of recording access events (login, data read/write, privilege
changes) into a secure log file or database.
• Key Log Information:
• Timestamp
• Username or process ID
• Accessed resource
• Action type (e.g., SELECT, UPDATE, DELETE)
• Result (Success/Failure)
• Benefits:
• Enables traceability.
• Supports incident investigation.
• Helps in forensic analysis after security breaches.
Set Thresholds for Alerts
Definition: Defining limits to flag abnormal login behavior.
Example 1: Alert if more than 3 failed logins in 10 minutes.
Example 2: Alert if login from blacklisted IPs.
Real-Time Monitoring & Geolocation
Definition: Monitor logs live & detect logins from suspicious
locations.
Example 1: Alert for logins from Asia if users are based in
Africa.
Example 2: Flag midnight access.
Review Failed Logins & Admin Logins
Definition: Prioritizing attention to failed attempts and
privileged accounts.
Example 1: Excessive failed root logins.
Example 2: Admin login during holiday.
Incident Response Planning
Definition: Predefined steps for responding to suspicious
logins.
Example 1: Lock account after multiple failures.
Example 2: Notify IT admin instantly.
Manage System Resources (3.3)
Introduction to System Resource Management
Definition 1: Monitoring and optimizing CPU, memory,
storage, and bandwidth.
Definition 2: Ensures optimal performance and reliability.
Definition 3: Aligns system capacity with usage demand.
Monitor CPU, Memory, Disk I/O
Definition: Use tools to assess how resources are consumed.
Example 1: Use top or htop on Linux.
Example 2: Use Azure Monitor for cloud systems.
Set Thresholds & Automate Alerts
Definition: Define max acceptable values; trigger alerts when
exceeded.
Example 1: Alert at 80% disk use.
Example 2: Email when CPU > 90%.
Capacity Planning & Scaling
Definition: Forecast future needs and scale
hardware/software.
Example 1: Add 32GB RAM based on projection.
Example 2: Move to cloud auto-scaling.
: Prioritize Critical Processes
Definition: Ensure essential tasks get resources.
Example 1: Allocate more memory to DB backup than
logging.
Example 2: Lower priority for analytics during peak hours.
Optimize Indexing and Queries
Definition: Reduce load with efficient queries and indexes.
Example 1: Add index to customer_id.
Example 2: Use query plans to optimize joins
Caching & Connection Management
Definition: Reduce redundant DB access and handle
connections smartly.
Example 1: Use Redis to cache frequent queries.
Example 2: Implement connection pooling in app server.
Disk & Network Optimization
Definition: Efficient use of disk space and bandwidth.
Example 1: Compress archived logs.
Example 2: Minimize JSON response sizes.
.
Backup Optimization
Definition: Reduce backup impact on system.
Example 1: Use incremental backups.
Example 2: Schedule backups during off-peak.
.
Virtualization & Load Balancing
Definition: Distribute resources smartly in virtual environments.
Example 1: Adjust VM RAM allocation.
Example 2: Use HAProxy to balance load.
Regular Updates & Security Patches
Definition: Maintain software for performance & security.
Example 1: Update MySQL monthly.
Example 2: Apply Linux kernel patches.
.
Virtualization & Load Balancing
Definition: Distribute resources smartly in virtual environments.
Example 1: Adjust VM RAM allocation.
Example 2: Use HAProxy to balance load.
Regular Updates & Security Patches
Definition: Maintain software for performance & security.
Example 1: Update MySQL monthly.
Example 2: Apply Linux kernel patches.
.
THANKS!
Do you have any questions?
[email protected]
+251 9xx xx xx xx
yourwebsite.com