Custom Database Engine

Session 2023-2027
Submitted by:
Umme Aymen 2023-CS-112
Rafia Sattar 2023-CS-120
Khushbakht Noor 2023-CS-129
Samar Riaz 2023-CS-137
Supervised by:
Sir Atif Hussain
Course:
Advance Database Management System
Department of Computer Science
University of Engineering and Technology,
Lahore Pakistan
1. Project Description
This project is a custom-built educational Database Management System
(DBMS) designed to help users understand and experiment with core database concepts,
especially transaction management and ACID properties. Through a modern web interface, users
can start, monitor, and control database transactions with selectable isolation levels, execute SQL
operations (SELECT, INSERT, UPDATE, DELETE) within transactions, and observe real-time
transaction logs. The platform visually demonstrates how transactions interact with the database,
ensuring atomicity, consistency, isolation, and durability. It also provides tools for schema
management, query execution, and relationship visualization, making it an ideal environment for
learning and exploring database internals in a hands-on, interactive way.

2. Objectives
 Provide an interactive platform to demonstrate and manage database transactions with
full ACID property support.
 Allow users to start, monitor, and control multiple concurrent transactions with selectable
isolation levels.
 Enable execution of core SQL operations (SELECT, INSERT, UPDATE, DELETE)
within transactions for hands-on learning.
 Visualize active transactions, transaction logs, and operation histories in real time.
 Offer a user-friendly web interface for managing database schema, data, and
relationships.
 Facilitate understanding of transaction states (active, committed, aborted) and their
effects on database consistency.
 Support educational exploration of transaction isolation and concurrency control
mechanisms.
 Enhance learning by providing immediate feedback and visual cues for transaction events
and outcomes.

3. Business Case
Effective transaction management is critical for any organization that relies on databases to
store and process business data. This project addresses the need for robust, transparent, and
educational transaction handling by providing a platform where users can visualize, control,
and experiment with database transactions and their ACID properties. By simulating real-
world scenarios—such as concurrent data access, transaction isolation, and error recovery—
this system helps organizations and educational institutions train staff, test business logic,
and ensure data integrity in a risk-free environment. Ultimately, it supports better decision-
making, reduces the risk of costly data inconsistencies, and enhances the reliability of
business operations that depend on transactional data processing.
4. Motivation
The motivation behind this project stems from the need to make complex database transaction
concepts more accessible and interactive for students and professionals alike. Traditional
textbooks and static tutorials often fail to convey the dynamic nature of transaction management
and ACID properties in real-world scenarios. By providing a hands-on, visual platform, this
project bridges the gap between theory and practice, allowing users to experiment with
transactions, observe their effects, and gain a deeper understanding of how databases ensure data
integrity and consistency. This interactive approach not only enhances learning but also builds
confidence in managing and troubleshooting transactional systems in practical environments.

5. Audience
 Students learning database management systems and transaction concepts.
 Educators and instructors teaching database theory and practical implementation.
 Developers and software engineers interested in understanding ACID properties and
transaction handling.
 Database administrators seeking a safe environment to experiment with transaction
isolation and concurrency.
 Researchers exploring database internals and transaction processing mechanisms.
 Anyone preparing for interviews or certifications related to databases and transaction
management.

6. Features

Core Database Engine

 Schema Management: Easily create and manage database tables with customizable
columns and data types through a user-friendly web interface.
 Data Manipulation: Perform essential SQL operations such as INSERT, UPDATE,
DELETE, and SELECT on your tables within managed transactions.
 Constraint Enforcement: Automatically enforce key constraints including PRIMARY
KEY, FOREIGN KEY, UNIQUE, NOT NULL, and CHECK to maintain data integrity.
 Custom Data Structures: Utilizes efficient data structures such as B-Trees for indexing,
hash tables for fast lookups, and linked lists/graphs for internal transaction and
relationship management.
 Persistence Layer: Ensures all schema definitions and table data are securely stored and
retrieved using JSON files, providing durability and easy backup.
 Transaction Support: Manage multiple concurrent transactions with full ACID property
support and selectable isolation levels for realistic database behavior.
 REST API (Flask-based)
  Designed with a modular backend architecture for maintainability and scalability.
 Provides dedicated API endpoints for managing database schemas, executing data
operations, and handling transactions.
 Supports transaction control (begin, commit, abort) and operation execution (SELECT,
INSERT, UPDATE, DELETE) via RESTful endpoints.
 Returns consistent and structured JSON responses for seamless integration with the
frontend and external tools.
 Enables real-time updates and interaction between the web interface and the underlying
database engine.

 Web Frontend
 Transaction Manager: Intuitive interface to start, monitor, and control database
transactions with selectable isolation levels and real-time status updates.
 Database Browser: Easily view and explore existing tables, inspect their data, and
review schema details.
 Query Editor: User-friendly tool to execute SQL-like queries and view results instantly
within the browser.
 Schema Designer: Visual interface for creating, editing, and managing table schemas,
columns, and constraints.
 Data Structure Visualizer: Interactive visualizations that demonstrate internal data
structures (such as B-Trees and hash tables) and show how operations affect them in real
time.
 Transaction Log Viewer: Real-time display of transaction events, operations, and state
changes for transparency and learning.

7. Architecture

The architecture of this project follows a modular, layered approach, combining a Flask-based
REST API backend with a dynamic, responsive web frontend. The backend is responsible for
core database logic, including schema management, transaction processing, ACID enforcement,
and data persistence using JSON files. It exposes RESTful endpoints for all major operations,
enabling seamless communication with the frontend. The frontend, built with HTML, Bootstrap,
and JavaScript (jQuery), provides intuitive interfaces for managing transactions, executing SQL-
like operations, and visualizing transaction logs and database states in real time. This separation
of concerns ensures scalability, maintainability, and a clear flow of data between user actions and
backend processing, making the system both robust and user-friendly for educational and
experimental use.

8. Technology Stacks
Component Technology
Platform Desktop/Web (Browser-based UI)

Language Python 3.x

Backend Flask

Frontend HTML, CSS (Bootstrap), JavaScript (jQuery)

Storage JSON

Tools/Env Visual Studio Code, Git

Table.01

9. Database Features

General Features
 Lightweight, custom-built relational database engine designed for educational and
experimental use.
 Full support for reading and writing data, with persistent storage using JSON files.
 Transaction management with ACID properties and multiple isolation levels.

Schema Features
 Create, modify, and delete tables with user-defined columns and data types through an
intuitive web interface.
 Flexible schema management to support evolving data models.

Constraint Features
 Enforces essential constraints including PRIMARY KEY, UNIQUE, FOREIGN KEY,
NOT NULL, and CHECK to ensure data integrity.

Indexing Features
 Implements B-tree indexing for optimized SELECT queries and efficient data access.

Hash Map Features

 Utilizes hash maps for rapid key-value data retrieval and internal lookups.

Visualization Features
  Interactive frontend visualizations of data structures such as B-trees and real-time display
of query results and transaction logs.
 Visual tools to monitor transaction states and database operations for enhanced learning
and transparency.

10. Data Structures Used

 Stack:
Utilized for parsing SQL-like expressions and managing the execution state during
transaction processing.
 Queue:
Handles background tasks, such as scheduling transaction operations and managing batch
execution requests.
 Linked List:
Supports dynamic record management and efficient chaining of data entries within tables.
 Hash Map:
Forms the core of the storage engine, enabling fast key-value access to rows and internal
metadata.
 Graphs:
Models relationships and dependencies between tables, especially for enforcing and
visualizing foreign key constraints.
 B-Trees:
Implements indexing for numeric and string keys, significantly improving the efficiency
of search and SELECT operations.

 Summary of Data Structure Usage

Data Structure Usage
Stack Query parsing and transaction tracking

Queue Asynchronous tasks

Linked List Record chains

Hash Map Primary data storage

Graph Table relationships (foreign keys)

B-Tree Indexing and range queries

Table.02
11. Technical Details
 UI Components

Frontend Technologies Purpose

HTML Define structural layout of the UI

CSS Handles styling and visualization

presentation

JavaScript Adds interactivity and dynamic behavior

Table.03

 Backend Modules
Flask-based REST API that manages
 Flask-based REST API that manages all core database functionalities.
 Handles CRUD operations (Create, Read, Update, Delete) within transaction contexts.
 Supports dynamic schema creation, modification, and deletion through API endpoints.
 Implements validation for data integrity, constraint enforcement, and transaction
isolation.
 Provides endpoints for transaction management, including begin, commit, and abort
actions.
 Ensures real-time synchronization between the backend database engine and the web
frontend.

 Storage

All database schemas, table data, and transaction logs are persistently stored as structured JSON
files within the designated data directory, ensuring durability and easy data management.

12. Classes
 Main Class (main.py):
Responsible for launching the server and initializing all core components of the database
system.
 App Class (app.py):
Defines REST API routes, manages request handling, and delegates business logic to
specialized modules.
 DataHandler Class:
Handles all reading, writing, and updating of persistent data stored in JSON files,
ensuring data durability.
  ConstraintHandler Class:
Validates data against schema constraints (e.g., PRIMARY KEY, FOREIGN KEY,
UNIQUE, NOT NULL, CHECK) before any write operation to maintain integrity.
 BTree Class (btree.py):
Implements a custom B-tree structure for efficient indexing and fast search operations on
table data.
 HashTable Class (hashtable.py):
Provides hash-based key-value lookups for rapid data access and internal metadata
management.
 Graph Class (graph.py):
Models and manages inter-table relationships, especially for enforcing and visualizing
foreign key dependencies.

13. Workflow
 Server Initialization
Launch the application by running main.py, which starts the Flask server and initializes
all backend modules.
 Schema Creation
Users define new table schemas through the web interface; the backend validates input
and stores schema definitions as JSON files.
 Record Insertion
Insert operations are submitted via the frontend, validated against all relevant constraints,
and persisted to the appropriate data files if valid.
 Query Execution
SQL-like operations (SELECT, INSERT, UPDATE, DELETE) are parsed and executed
within the context of active transactions, utilizing indexes for efficient access when
available.
 Constraint Handling
The ConstraintHandler module checks all operations for violations (e.g., PRIMARY
KEY, FOREIGN KEY, NOT NULL) and returns clear error messages if constraints are
not met.
 Index Updates
Whenever records are added, updated, or deleted, associated B-tree indexes are
automatically updated to maintain fast query performance.
 Data Retrieval
The backend uses B-tree indexes or hash maps for rapid data lookups, returning results to
the frontend for display.
  Transaction Management
All operations are executed within transactions, supporting ACID properties and isolation
levels; users can commit or abort transactions as needed.
 Shutdown
On server shutdown, all in-memory data and transaction logs are safely persisted to disk,
ensuring durability and consistency.

 Flowchart

Figure.01

14. Project Structure

ADBMS Final Project/
├── backend/ # Backend logic and modules
│ ├── app.py # Flask app and API routes
│ ├── data_handler.py # Handles reading/writing JSON data
│ ├── constraint_handler.py # Enforces schema constraints
│ ├── transaction_manager.py # Manages transactions and ACID logic
│ └── ... # Other backend modules
├── data/ # Persistent JSON files for schemas and tables
│ ├── schemas.json
│ ├── table_data.json
│ └── transaction_log.json
├── frontend/ # Static frontend assets
│ ├── css/
│ │ └── styles.css # Custom styles
│ └── js/
│ └── scripts.js # Custom JavaScript (if any)
├── templates/ # Jinja2 HTML templates for Flask
│ ├── index.html # Home page
│ ├── transactions.html # Transaction management UI
│ ├── schema.html # Schema management UI
│ ├── query.html # Query tool UI
│ ├── relationships.html # Table relationships visualization
│ └── tableview.html # Table data viewer
├── utils/ # Custom data structure implementations
│ ├── btree.py # B-Tree indexing
│ ├── hashtable.py # Hash table for fast access
│ └── graph.py # Graph for relationships
├── main.py # Entry point, starts Flask server
├── pyproject.toml # Project metadata and dependencies
├── requirements.txt # Python dependencies
├── README.md # Project overview and usage instructions
 ├── .gitignore # Git ignore file

15. System Requirements

Hardware Requirements
Processor: Dual-core 2.0 GHz or higher (Intel/AMD)

RAM: Minimum 4 GB (8 GB recommended for optimal performance)

Storage: At least 200 MB of free disk space

Display: 1024x768 resolution or higher

Software Requirements
Operating System:

Windows 10 or later

macOS 10.14 (Mojave) or later

Linux (Ubuntu 18.04+ or equivalent)

Python Version: Python 3.8 or above

Web Browser: Any modern browser (Google Chrome, Mozilla Firefox, Microsoft Edge)

Package Dependencies:

 Flask
 Standard Python libraries (os, sys, json, threading, logging, uuid, datetime, enum, shutil)
 No external database required (uses JSON files for storage)
 Development Tools (Optional for Contributors)
 Code Editor: Visual Studio Code, PyCharm, or any Python-compatible IDE
 Git: For version control and collaboration
 Virtual Environment Tool: venv or virtualenv for dependency isolation

16. Installation Guide

 Prerequisite
 Python 3.8
 Pip (python package installer)
 Setup Instructions
 # Clone the repository
$ git clone <repository_url>
$ cd ADBMS final project

# Create a virtual environment (optional but recommended)

$ python -m venv venv
$ source venv/bin/activate # On Windows: venv\Scripts\activate

# Install dependencies
$ pip install -r requirements.txt

# Run the app

$ python main.py

17. Usage
 Running Application
$ python main.py
Visit http://localhost:5000 in your browser to access the frontend interface.

 API Endpoints
Method Endpoint Description

POST /api/create_table Creates a new table

GET /api/get_tables Lists all tables

POST /api/insert_data Inserts data into a table

GET /api/select_data Queries data

PUT /api/update_data Updates existing records

DELETE /api/delete_data Deletes data from a table

 Frontend Interface
 Navigate to different modules via top navbar
 Design schema, run queries, and view data
 Visualize data structures live as operations are performed

18. Wireframes
 Create Table

Figure.02

 Table Visualization

Figure.03
Figure.04

Figure.05
Figure.06
 Add Record

Figure.07

 Query Processing

Figure.08
Figure.09
 Schema Creation

Figure.10

Figure.11
 Figure.12

 Data Structure Visualizer

Figure.13
Figure.14

Figure.15
Figure.16

Figure.17
Figure.18

Figure.19
TRANSACTION MANAGEMENT:

Figure.20

Figure.21
Figure.22

Figure.23