AI-Based Online Spam Detection

Table of Contents

1. Abstract
2. Chapter 1: Introduction
○ 1.1 Background
○ 1.2 Problem Statement
○ 1.3 Objectives
○ 1.4 Scope of the Project
3. Chapter 2: Literature Survey
○ 2.1 Overview of Spam and Its Impact
○ 2.2 Traditional Filtering Techniques
○ 2.3 Machine Learning in Spam Detection
○ 2.4 Related Work and Studies
4. Chapter 3: System Analysis
○ 3.1 Existing System
○ 3.2 Proposed System
○ 3.3 Feasibility Study
○ 3.4 Requirements Analysis
■ 3.4.1 Functional Requirements
■ 3.4.2 Non-Functional Requirements
5. Chapter 4: System Design
○ 4.1 System Architecture
○ 4.2 Use Case Diagram
○ 4.3 Data Flow Diagram
○ 4.4 Database Schema
○ 4.5 UI Design
6. Chapter 5: Implementation
○ 5.1 Technology Stack
○ 5.2 Django Model and Views
○ 5.3 ML Model Integration
○ 5.4 Input/Output Flow
7. Chapter 6: Results and Discussion
○ 6.1 Evaluation Metrics
○ 6.2 Model Accuracy
○ 6.3 Sample Predictions
○ 6.4 Advantages and limitation

8. Chapter 7: Conclusion and Future Work

○ 7.1 Conclusion
○ 7.2 Scope for Enhancement
9. References
Chapter 1: Introduction

1.1 Background
In today’s digital communication age, spam messages and phishing emails have become a
significant threat, especially with the rapid growth of online platforms. These malicious contents
aim to trick users into sharing sensitive data or clicking on harmful links. Efficient spam detection
has thus become critical for ensuring user safety, privacy, and secure communication.

To counter these threats, machine learning (ML) models have emerged as powerful tools for
classifying messages as spam or not based on content, structure, and keywords. This project
explores a web-based solution to detect spam in emails and online messages using a Django
framework integrated with a pre-trained machine learning model.

1.2 Problem Statement

Manual detection of spam is inefficient and error-prone, especially when the volume of messages
is high. Traditional filtering techniques, such as keyword matching, often fail against
sophisticated spam that uses obfuscated content. Hence, the need arises for an intelligent, real-
time spam detection system.

Problem Definition: To design and develop a web-based AI-powered spam detection system
that classifies incoming messages or emails into spam or not spam using natural language
processing (NLP) and machine learning models.

1.3 Objectives
● To implement a machine learning model capable of accurately classifying spam
messages.
● To develop a Django-based web interface for users to submit messages and receive
predictions.
● To integrate a simple and extensible model-view-controller (MVC) structure for spam
classification.
● To store and view classified messages with their labels for review and analysis.

1.4 Scope of the Project

This project focuses on building a lightweight yet effective spam detection platform using Django
and machine learning. It enables users to:

● Input or upload messages and receive classification.

● Automatically classify messages as "Spam" or "Not Spam."
● View stored classified records in a simple database table.
The current system will not include:

● Real-time email scanning or API-level integration with email services.

 ● Multi-language support or deep learning-based models.

Chapter 2: Literature Survey

2.1 Overview of Spam and Its Impact

Spam is any unsolicited message that aims to deceive or defraud the recipient. The rapid
proliferation of spam can overwhelm systems, reduce productivity, and expose users to security
threats. According to research, over 45% of email traffic globally is considered spam.

2.2 Traditional Filtering Techniques

Earlier methods for spam detection relied on:

● Keyword-based filtering
● Blacklisting known sources
● Rule-based systems
While these methods are simple, they lack adaptability and are prone to high false-positive rates.

2.3 Machine Learning in Spam Detection

Modern systems employ ML models trained on labeled datasets to learn the difference between
spam and non-spam. Techniques used include:

● Naive Bayes Classifiers

● Support Vector Machines (SVM)
● Logistic Regression
● Random Forest
● Neural Networks (for advanced detection)
Feature extraction plays a crucial role, typically using techniques such as:

● Bag of Words (BoW)

● TF-IDF (Term Frequency–Inverse Document Frequency)
● Word embeddings (in more advanced systems)

2.4 Related Work and Studies

Several open-source and academic systems have explored spam detection:

● The Enron Spam Dataset has been a benchmark dataset for training models.
● Researchers have achieved accuracy upwards of 95% using ensemble learning.
● Gmail, Outlook, and other platforms use hybrid models combining rule-based and AI
filters.
This project builds upon these foundations, implementing a logistic regression classifier
integrated into a Django web app for real-world testing.

Chapter 3: System Analysis

3.1 Existing System

The [[project name] is a full-stack web application designed to provide intelligent
recommendations based on real-time environmental data. It combines a modern web framework
(Django) with machine learning techniques to [[project name], administrators, and agribusiness
customers with insights that can improve planning, resource utilization, and yield outcomes.

This chapter outlines the major components of the system, describing how each module
contributes to the overall functionality. The system architecture is modular, allowing for future
expansion and easy maintenance. The four primary components are: Frontend, Backend,
Database, and Machine Learning Service.

Frontend

The user interface is built using Django templates and styled with Tailwind CSS, a utility-first
CSS framework that allows for rapid design and consistent aesthetics across pages. The
frontend is designed to be clean, responsive, and accessible, ensuring usability on both
desktop and mobile devices.

Key frontend features include:

The frontend interacts with the backend using standard form submissions and can be extended
to include AJAX or API-based data submission in future versions. Mobile responsiveness is
achieved using Tailwind's responsive breakpoints, ensuring optimal user experience on smaller
screens.

Backend

The backend is developed in Django 4.x, a high-level Python web framework known for its
scalability, security, and built-in ORM. It handles:

● Routing and Views: URL mappings are created for all major functionalities such as data
entry, prediction, and dashboard access. Views control the logic for rendering templates
or processing prediction requests.
 ● Authentication and Authorization: Built-in Django auth is used for login, logout, and
password protection. Users are grouped into roles using Django's Group model, enabling
role-based access control.
● Model Integration: The machine learning model is integrated into Django as a Python
module. It is invoked during form processing to generate real-time predictions.
● Data Validation and Security: The backend ensures input validation, uses CSRF
protection on forms, and handles exceptions gracefully to prevent system crashes.

The architecture follows the Model-View-Template (MVT) design pattern, which cleanly
separates data, logic, and presentation layers.

Database

The system uses PostgreSQL as its primary relational database management system.
PostgreSQL was selected due to its robustness, ACID compliance, and support for advanced
features such as JSON storage, indexing, and role management.

Database tables and their roles include:

● User Accounts: Stores information about registered users, their roles, and credentials.
● Prediction Logs: Captures environmental input values, recommended s, timestamps,
and user ID for traceability.
● Training Dataset Management (Admin Only): A provision to upload and manage
datasets for retraining the ML model in the future.
● Feedback Records: Planned for future implementation where can submit feedback
on the recommendation quality.

The Django ORM abstracts SQL queries and handles migrations seamlessly, which simplifies
database operations and schema evolution.

Machine Learning Service

The core intelligence of the system lies in the Machine Learning service, implemented as a
standalone Python module integrated into the Django backend. The service uses a Random
Forest Classifier trained on a structured dataset containing labels and environmental features.

Key components of the ML service include:

● Model Training Script: Written in Python using pandas, scikit-learn, and NumPy.
The model is trained offline and validated using test data before deployment.
● Model Serialization: The trained model is serialized using joblib for efficient storage
and fast loading at runtime.
 ● Runtime Inference: When a user submits environmental data, the Django view loads the
serialized model and passes the input to generate predictions in real-time.
● Result Output: The prediction result is returned to the user interface along with optional
metadata like confidence score (planned in future).

The design ensures that the model can be updated independently of the web app. Admins can
retrain the model offline and replace the serialized .pkl file without needing to redeploy the
entire system.

Integration Workflow

Here’s how the components interact in practice:

1. User Interaction: The user fills out the data to be used for prediction form via the
frontend interface.
2. Form Submission: The input is sent to a Django view through a secure POST request.
3. Model Prediction: The backend view invokes the ML service, loads the model, and
performs inference.
4. Result Storage and Display: The prediction is stored in the database and returned to
the frontend for display.
5. Admin Access (Optional): Admins can view all prediction logs and performance
analytics.

This modular and loosely coupled architecture ensures that each part of the system is
independently testable, replaceable, and scalable.

3.2 Proposed System

The proposed system addresses these limitations by offering a custom, content-

based detection tool using supervised learning. The application uses TF-IDF
vectorization to extract features from the text and Logistic Regression to classify.

Key features of the proposed system:

● Accepts content (subject + body) through a web interface.

● Uses a trained machine learning model to classify the input.
● Offers a result page showing the prediction.
● Allows users to register and log in before accessing the tool.
● Designed with a clean UI and scalable backend using Django.
This system is intended as an educational prototype to demonstrate real-world AI integration
in web applications.

3.3 Feasibility Study and Requirements Analysis

A feasibility study was conducted to assess the practicality of the project in terms of
technology, cost, and effort. The results are as follows:

Technical Feasibility: The system is built using widely available open-source tools (Python,
Django, Scikit-learn). No proprietary software or specialized hardware is required. The ML
model is trained offline and loaded into memory during runtime using joblib.

Operational Feasibility: The system is easy to use and deploy. Once trained, the model
can serve multiple prediction requests in real time without needing re-training unless
explicitly required.

Economic Feasibility: Since the system uses open-source tools and publicly available
datasets, there are no direct costs involved. It is feasible for individual developers and
academic institutions.

3.4 Requirements Analysis

3.4.1 Functional Requirements

● User Registration and Login System.

● submission form with subject and body fields.
● ML-based prediction of or not .
● Result page displaying the classification output.
● Admin interface to monitor users (optional).

3.4.2 Non-Functional Requirements

● Usability: Clean, simple UI for ease of use.

● Performance: Fast prediction response (<1s).
● Scalability: Model can be retrained on larger datasets.
● Security: User data stored securely; authenticated access enforced.
● Maintainability: Modular code structure, following MVC architecture.
Chapter 4: System Design
4.1 System Architecture Overview
The system follows a three-tier architecture:
1. Presentation Layer (Frontend)

2. Application Layer (Backend with Django & ML Model)

3. Data Layer (Database & ML Model Storage)

1. Presentation Layer (Frontend)

Components:
 Web-based form for user input (HTML templates using Django's templating engine)

 Result page displaying prediction output

Responsibilities:
 Collect data from users (age, cholesterol, etc.)

 Display prediction results to users

 Provide navigation between pages (form, result, admin)

2. Application Layer (Backend)

Components:
 Django views and URLs

 Trained Machine Learning model (e.g., .pkl file)

 Business logic for prediction

Responsibilities:
 Handle HTTP requests and form submissions

 Validate and preprocess form data

 Load the ML model into memory

 Pass preprocessed data to the model for prediction

 Save results to the database

 Render appropriate templates with data (e.g., prediction result)

3. Data Layer
Components:
 Database: SQLite (db.sqlite3)

 Machine Learning Model File: Serialized model file (_model.pkl)

Responsibilities:
 Store input and prediction results
  Provide data persistence for admin view and record-keeping

 Store and load the trained ML model used for real-time predictions

Workflow Summary
1. User Accesses the Form
User navigates to the form page and enters required information.

2. Data Submitted to Django View

Django receives the data, validates it, and prepares it for prediction.

3. Data Passed to ML Model

The structured input is sent to the pre-trained model, which returns a prediction (0 or
1).

4. Prediction Stored and Displayed

The result is saved to the database and shown on a result page.

5. Admin Panel Access

Admin users can view all stored records and predictions through Django’s built-in
admin interface.

Optional Enhancements (Future-Proofing)

 Authentication Layer: Add login for /admin access.

 API Layer: REST API using Django REST Framework for external integration.

 Model Retraining: Periodic retraining with new data entries to improve prediction
accuracy.

4.2 Use case Diagram

4.3 Data Flow Diagram

1. User Input
 The user submits a message through a web form.
 Inputs include:
o Subject (e.g., “Win a free iPhone”)
o Body text (e.g., “Click here to claim your prize!”)

2. Django Web Application

 The Django backend receives the input.
 It processes the form using ScamRecordForm.
 A temporary database object (ScamRecord) is created but not saved yet.
 This data is prepared for analysis.

3. Phishing Link Detection

 Before ML prediction, the system runs utils.py to scan the message body.
 It checks for:
o URLs from blacklisted domains (like bit.ly, fakebank.ru)
o Suspicious patterns (.ru, “free”, “claim”, etc.)
 Detected phishing links can be optionally flagged (future enhancement).

4. Machine Learning Model

  The combined subject + body is passed to a pre-trained Logistic Regression
model with TF-IDF vectorization.
 The model predicts whether the message is:
o Spam
o Not Spam

5. Prediction Output
 The prediction (Spam or Not Spam) is saved in the database with the
message.
 The user is redirected to a result page showing the prediction.

4.4 UI Design

Place screen shot and write minimal explanation about the screenshot you can UI you
feel good

Chapter 5: Implementation

5.1 Technology Stack

1. Programming Languages
 Python
Used for data processing, model training (machine learning), and back-end
development (Django framework).
  HTML/CSS
Used for designing the web front-end interface (forms, templates).

2. Machine Learning and Data Science

 Pandas – For data loading and manipulation.

 NumPy – For numerical operations.

 Scikit-learn – For building and training the Logistic Regression model.

 Joblib – For saving the trained model ( _ .pkl) for reuse in the web app.

3. Web Framework
 Django (Python-based Web Framework)

o Handles URL routing, form submissions, rendering HTML templates,

and database interaction.

o Django Admin panel is used for managing data models through a web
UI.

4. Database
 SQLite

o Lightweight, serverless relational database used for storing form

submissions and managing admin data.

o Default database engine used in Django during development.

5. Front-End
 Django Templates (HTML with template tags)
Used to render web forms, display results, and interact with users dynamically.

6. Deployment Tools (if applicable)

 No specific deployment tools were provided in the project, but typical options
for a Django project would include:

o Gunicorn + Nginx for production servers

o Heroku or PythonAnywhere for quick cloud deployment

7. Development Tools
 Jupyter Notebook / Python scripts (for model development)

 VS Code / PyCharm (likely used for coding)

 Command line / Terminal for running Django commands (runserver,

makemigrations, etc.)

5.2 User Authentication

Place screenshot of log in page once you get login page
Second after filling login info

5.3 Dataset and Preprocessing

This project is a web-based application built with Django that enables users to detect spam
or scam content in messages or emails. It leverages machine learning to classify text as
spam or not, and includes phishing link detection through rule-based logic.

Web Application Structure (Django)

 Apps:
o home: Landing page.
o scam: Core logic for spam detection, ML integration, and user input.
 URLs:
o '/': Homepage.
o '/upload/': Upload dataset CSV.
o '/classify/': Start bulk classification.
o '/result/<id>/': View classification result.
o '/scam/': Submit message for spam check.
 Forms:
o ScamRecordForm: Collects subject and body from the user.
 Models:
o ScamRecord: Stores the subject, body, and predicted label.

Machine Learning Model

 Training Script: train_model.py
 Model:
o Pipeline with TfidfVectorizer and LogisticRegression.
o Trained on scam_dataset.csv.
o Predicts based on a combination of subject and body.
 Usage:
o Trained model saved as scam_model.pkl.
o Loaded in views to make predictions.

Spam Detection Logic

 Real-time prediction:
o User inputs a message → Form submitted.
o Message is vectorized → Model predicts label.
o Result stored in DB and shown to user.
 Phishing Detection:
o utils.py: Uses regex to extract URLs.
o Checks against known blacklisted domains.
o Also flags suspicious patterns like .ru, free, bit.ly, etc.

Dataset and Storage

 Dataset: scam_dataset.csv (used to train the model).
  Database: db.sqlite3 (stores user inputs and predictions).
Dependencies
Defined in requirements.txt, includes:
 Django, scikit-learn, pandas, joblib, matplotlib, etc.

Additional Features (Planned or Partial)

 CSV Upload and Batch Classification (seen in views but implementation may need
review).
 Static templates for forms and result display (scam/templates/).

Key Features
1. Email or Message Analysis:
o Takes subject and body text as input.
o Combines them for spam detection processing.
2. Phishing Link Detection:
o utils.py contains a rule-based phishing detection using:
 Blacklisted domains (e.g., bit.ly, fakebank.ru).
 Pattern-based detection (e.g., presence of .ru, free, win, etc.).
3. Spam Classification:
o Uses a trained machine learning model to classify whether a message is
spam or not.
4. Web Interface:
o Built with Django (home and scam apps).
o Forms to submit messages and receive predictions.
o Uses scam_model.pkl for real-time prediction.
Machine Learning Model
 Algorithm: Logistic Regression
 Vectorization: TF-IDF (TfidfVectorizer)
 Pipeline:
 Training Data: scam_dataset.csv which contains:
 subject
 body
 label (target)
 Storage: Trained model saved as scam_model.pkl using joblib.

Admin Panel:

Step 1: Register the Model in scam/admin.py

Your model is defined in scam/models.py as ScamRecord.
from django.contrib import admin
from .models import ScamRecord

@admin.register(ScamRecord)
class ScamRecordAdmin(admin.ModelAdmin):
 list_display = ('id', 'subject', 'body', 'label', 'created_at')
search_fields = ('subject', 'body', 'label')

from django.contrib import admin

 Imports Django’s admin module, which allows us to customize how models
are displayed and managed in the admin panel.
from .models import ScamRecord
 Imports the ScamRecord model from the models.py file in the same app (i.e.,
scam app).
 This is the model you want to manage in the admin panel.

@admin.register(ScamRecord)
 A decorator that registers the ScamRecord model with the Django admin
site.
 Equivalent to:
admin.site.register(ScamRecord, ScamRecordAdmin)
 Cleaner and more modern approach.

class ScamRecordAdmin(admin.ModelAdmin):
 A custom admin class that lets you control how ScamRecord appears in the
admin interface.
 Inherits from admin.ModelAdmin.
list_display = (...)
 Specifies which fields to show in the list view of the admin dashboard.
 You’ll see a table with columns like:
o id: Primary key

o subject: Email/message subject

o body: Message content

o label: Prediction (e.g., "Spam", "Not Spam")

o created_at: Timestamp

search_fields = (...)
 Enables search functionality in the admin panel.
 Admins can search across subject, body, or label text.

Step 2: Create Superuser and Run Server

python manage.py makemigrations
python manage.py migrate

Create Superuser:
python manage.py createsuperuser

Step 3: Access Admin Panel

 Open your browser and go to
 Log in using the superuser credentials.

Output
In the admin panel, under the Scam section, you’ll now see a model
named Scam Records
ID Subject Body Text Label Created At
“Win a prize 2025-05-10
1 “Click this link to claim...” Spam
today” 12:00
“Meeting “Attached is the file for Not 2025-05-10
2
Agenda” review” Spam 12:05

Explanation
 The admin.py file connects the backend model to Django's default

admin interface.
 list_display shows fields in a table.
 search_fields enables keyword search.
 This allows admins to view, edit, or delete spam predictions
directly from the dashboard.
Django web application in AI-Based Spam Detection
project
Django is a Python web framework used here to:
1. Render the web interface for inputting message data.
2. Handle user submissions through forms.
3. Call the ML model to make predictions.
4. Store results in a database.
 5. Display results and admin panel views.
Working Flow of Django in This Project
1. User Enters Message
 Route: /scam/
 Form: ScamRecordForm
 App: scam/views.py
2. Form is Processed
 Django collects subject and body.
 Combines them and sends them to scam_model.pkl for prediction.
3. ML Model Predicts Spam or Not
 Using joblib.load(...), the trained model makes a prediction.
4. Database Stores Input & Prediction
 Django ORM saves subject, body, and predicted label in the ScamRecord model.
5. Admin Views or Result Page Shows Output
 Admin panel or user result route shows whether the message is spam.
scam/models.py – Defines Data Structure
from django.db import models

class ScamRecord(models.Model):

subject = models.TextField()

body = models.TextField()

label = models.CharField(max_length=20)

created_at = models.DateTimeField(auto_now_add=True)

Purpose: Holds user input and prediction.

scam/forms.py – Form for User Input
from django import forms
from .models import ScamRecord

class ScamRecordForm(forms.ModelForm):
class Meta:
model = ScamRecord
fields = ['subject', 'body']
Purpose: Renders an input form on the web page.
scam/views.py – Web Logic and Prediction

from django.shortcuts import render, redirect

from .forms import ScamRecordForm

from .models import ScamRecord

import joblib

model = joblib.load('scam_model.pkl') # Load the ML model

def scam_view(request):

if request.method == 'POST':

form = ScamRecordForm(request.POST)

if form.is_valid():

instance = form.save(commit=False)

text = f"{instance.subject} {instance.body}"

prediction = model.predict([text])[0]

instance.label = prediction

instance.save()

return render(request, 'scam/result.html', {'result': prediction})

else:

form = ScamRecordForm()

return render(request, 'scam/form.html', {'form': form})

This function:
 Displays a form to the user.
 Processes the form when submitted.
 Uses a trained ML model to predict if the message is spam.
 Displays the result to the user.

 Imports render to display HTML templates.

  redirect could be used to redirect after POST (not used here but good to
have).
 ScamRecordForm: Django form to collect subject and body from the user.
 ScamRecord: Model that stores form input and prediction.
 joblib.load(...) loads the trained model stored in scam_model.pkl.
 This model is used to make predictions on user input.
 Defines the Django view function that handles both GET and POST requests
on the /scam/ URL.
 Checks if the form is submitted.
 request.POST holds the user-submitted data.
 Validates the form.
 commit=False prevents immediate saving to the DB — lets us modify before saving.
 Combines the subject and body into one string for the ML model.
 Model prediction happens here.
 predict() returns a list — [0] gives the first (and only) result.
 Stores the prediction result (label) in the model instance.
 Saves the record to the database.
 Displays the result to the user using the result.html template.
 If no form is submitted, display an empty form to the user.
 Render the HTML template form.html with the form.

Feature What It Does

form.is_valid() Checks if all required fields are filled correctly.
model.predict() Makes the spam/ham prediction using trained ML model.
instance.save() Saves the subject, body, and prediction to the database.
render() Displays either the form or the result page.

scam/urls.py – Route to Scam View

from django.urls import path

from .views import scam_view

urlpatterns = [

path('scam/', scam_view, name='scam_check'),

Purpose: Adds URL endpoint /scam/.

templates/scam/form.html – User Interface (simplified)

<h2>Spam Detection Form</h2>

<form method="post">

{% csrf_token %}

{{ form.as_p }}

<button type="submit">Check</button>

</form>

Purpose: Adds URL endpoint /scam/.

templates/scam/result.html – Show Prediction

<h2>Prediction Result</h2>

<p>This message is: <strong>{{ result }}</strong></p>

Summary: How Django Is Used

Component Role in Project

Models Defines DB schema for messages
Forms Renders HTML forms for input
Views Handles logic (form + ML prediction)
Templates Displays form and result in the browser
URL Dispatcher Maps web URLs to views
Admin Panel Lets admin manage predictions and data visually

ML Workflow in This Project

This project uses a supervised learning algorithm (Logistic Regression) trained
on a dataset of labeled messages (spam or not spam). The key ML components are:

Step Description
Training Done offline in a Python script (e.g., Jupyter Notebook or Python file).
Step Description
Saving The trained model is stored using joblib as scam_model.pkl.
Loading Django loads this .pkl file and uses it to make predictions.
Prediction New messages are passed to the model and classified as Spam or Not Spam.

Project File Structure Summary

File Purpose
scam_model.pkl Trained ML model used in production
views.py Loads model and calls predict()
utils.py (optional) Link/URL pattern detection (basic logic)

2. Model Training Code

Usually done in a script like train_model.py or Jupyter
notebook:

import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
import joblib

# Load dataset
df = pd.read_csv('spam_dataset.csv') # Contains 'subject', 'body', 'label'

# Combine subject and body

df['text'] = df['subject'] + " " + df['body']

# Prepare features and labels

X = df['text']
y = df['label']

# Vectorize text
vectorizer = TfidfVectorizer()
X_vect = vectorizer.fit_transform(X)

# Train model
model = LogisticRegression()
model.fit(X_vect, y)

# Save model and vectorizer

joblib.dump((vectorizer, model), 'scam_model.pkl')

Output from Training

A file named scam_model.pkl is created. It contains:
 The trained model
 The vectorizer (TF-IDF)

3. Integration With Django

In views.py:
import joblib

# Load vectorizer and model together

vectorizer, model = joblib.load('scam_model.pkl')

def scam_view(request):
...
if form.is_valid():
instance = form.save(commit=False)
text = f"{instance.subject} {instance.body}"
vect_text = vectorizer.transform([text]) # Convert input to numerical form
prediction = model.predict(vect_text)[0] # Predict

Explanation:
 The model is loaded once when Django starts.
 The user's input is transformed using the same TF-IDF logic used during
training.
 The model gives a prediction ("Spam" or "Not Spam").
Complete Web-to-ML Flow Diagram

5.4 Image Upload and Live Prediction

Put screen shots of 2-4 examples

Chapter 6: Results and Discussion

6.1 Model Accuracy

Model accuracy is the percentage of correct predictions made by a machine learning model on
unseen data (test set):

Accuracy = Correct Predictions/ Total Prediction

 The model was trained on a dataset of spam and non-spam messages.

 When tested, it predicted correctly 80% of the time.

Interpretation of 80% Accuracy

 This means the model performs no better than random guessing.
 For a binary classification (Spam vs Not Spam), 80% accuracy is poor and indicates:
o Possibly imbalanced data (e.g., mostly non-spam).
o Weak features: The model doesn’t have enough useful signals to distinguish
spam from real messages.
o Underfitting: The model is too simple or not trained well.

Advantages:

1. Automated Spam Filtering:

o Efficiently identifies spam messages or content in real-time using
trained ML models.
2. Improved Accuracy:
o Machine learning models (like Naive Bayes, SVM, or deep learning
models) can achieve high precision and recall when trained on quality
datasets.
3. Scalability:
o Django allows the project to scale easily as the web application grows
in traffic or complexity.
4. Customizability:
o Models can be retrained with new data to adapt to evolving spam
tactics, improving long-term performance.
5. Integration Friendly:
o Django REST framework makes it easy to expose the model via APIs,
which can be integrated with other applications or platforms.
6. Real-time Processing:
o Spam detection can happen in real-time upon content submission,
improving user experience.
 7. User Interface and Admin Panel:
o Django provides a robust admin interface for monitoring flagged
content and managing training data.
Limitations
1. Model Drift:
o Over time, the effectiveness of the ML model may decline if not
retrained with updated spam examples.
2. False Positives/Negatives:
o Some legitimate messages may be marked as spam (false positives),
or spam may go undetected (false negatives).
3. Data Dependency:
o The quality and quantity of labeled training data significantly affect
model performance.
4. Performance Overhead:
o Real-time inference might introduce latency depending on model
complexity and server performance.
5. Maintenance Complexity:
o Regular updates and monitoring are needed to ensure the model
remains effective and secure.
6. Security Concerns:
o If not properly sandboxed, attackers might exploit ML APIs or send
adversarial inputs to bypass detection.
7. Bias in Training Data:
o If the dataset is biased, the model may unfairly target certain content
types or language patterns.

Chapter 7: Conclusion and Future Work

Conclusion
 The AI-Based Online Spam Detection System developed in this project is a
web-based application that integrates machine learning techniques with the Django
web framework to identify and classify potentially harmful or unwanted messages.
The primary goal of this project was to develop a robust and user-friendly interface
where users can input text content — specifically email subjects and bodies — and
receive real-time feedback on whether the content is classified as spam or not.
At the heart of this application lies a machine learning model trained on a
labeled dataset containing spam and non-spam messages. The project employed a
logistic regression classifier in conjunction with TF-IDF vectorization to extract and
quantify text features. These features were then used to train the model to
distinguish between spam and legitimate messages. The trained model was
serialized and deployed within the Django web application using the joblib library,
enabling seamless interaction between the front-end user interface and the back-end
predictive engine.
Despite its simplicity and successful integration into the Django framework,
the current version of the model achieved an accuracy of approximately 50%. This
highlights several important considerations and limitations. First, the training data
may not have been diverse or balanced enough, limiting the model's ability to
generalize effectively. Second, the logistic regression model, while efficient and
interpretable, may not be powerful enough to capture complex spam patterns that
more advanced models like Random Forests or deep learning techniques could
handle. Lastly, the feature set — which only includes raw text from the subject and
body — may be insufficient. More sophisticated features such as the presence of
URLs, common spam keywords, or metadata like sender reputation could
significantly improve prediction accuracy.
From a software development perspective, the Django framework proved to
be a reliable and scalable choice for building the application. It facilitated rapid
development of the user interface, form handling, database integration, and dynamic
content rendering. By coupling Django with the trained machine learning model, the
application was able to process user input in real time and return actionable results,
demonstrating a practical example of ML deployment in a web environment.
In conclusion, while the project successfully demonstrates the integration of
machine learning with a web application for spam detection, there is considerable
scope for improvement. Enhancing the dataset, refining the preprocessing pipeline,
and experimenting with more advanced algorithms are the next logical steps. With
these improvements, the system can evolve into a highly reliable spam detection
platform suitable for production environments or enterprise deployment.

Future Enhancements
As technology advances and spam detection methods become increasingly
sophisticated, several future enhancements can be applied to improve the accuracy,
usability, and scalability of this project. These enhancements focus on both the
machine learning model and the web application components:
1. Use of Advanced Machine Learning Models
 Replace the logistic regression model with more powerful algorithms like:
o Random Forest
o Support Vector Machine (SVM)
o XGBoost
o Deep Learning (LSTM, BERT) for contextual understanding.
 These models can better capture complex spam patterns and contextual
nuances.

2. Model Optimization and Hyperparameter Tuning

 Implement GridSearchCV or RandomizedSearchCV to find the best
hyperparameters.
 Use cross-validation to ensure model generalization and reduce overfitting.

3. Feature Engineering Enhancements

 Introduce additional features such as:
o Number of links or attachments.
o Presence of suspicious keywords or phrases.
o Sender email domain analysis.
o Email formatting patterns (e.g., excessive use of uppercase, emojis).
 Use word embeddings (like Word2Vec or BERT) instead of TF-IDF for better
semantic understanding.

4. Larger and More Diverse Dataset

 Train the model on a larger, more diverse, and balanced dataset.
 Incorporate publicly available datasets like the Enron Email Dataset or
SpamAssassin.

5. Real-Time Email Integration

 Integrate the system with email clients (e.g., Gmail, Outlook) using APIs.
 Scan incoming messages in real time and alert users about potential spam.

6. Improved User Interface and Feedback Mechanism

 Enhance the UI for better user experience and accessibility.
  Add a feedback option where users can correct false positives or false
negatives — enabling online learning.

7. Mobile and API Support

 Build a mobile version of the application for on-the-go usage.
 Develop a RESTful API to allow external apps or services to access the spam
detection engine.

8. Live Model Retraining

 Implement a model retraining pipeline that learns from new user feedback.
 Use tools like MLflow or DVC for model tracking and versioning.

9. Security and Privacy Measures

 Ensure secure handling of user-submitted content using encryption and
input sanitization.
 Implement authentication and authorization in the admin panel.

10. Cloud Deployment and Scalability

 Deploy the application to cloud platforms like AWS, Azure, or Heroku.
 Use containers (Docker) and CI/CD pipelines for production-ready
deployment.

References
1. Metsis, V., Androutsopoulos, I., & Paliouras, G. (2006). Spam filtering with Naive

Bayes – Which Naive Bayes? CEAS Conference.

2. Zhang, L., Zhu, J., & Yao, T. (2004). An evaluation of statistical spam filtering

techniques. ACM Transactions on Asian Language Information Processing.

3. Sahami, M., Dumais, S., Heckerman, D., & Horvitz, E. (1998). A Bayesian

approach to filtering junk e-mail. Learning for Text Categorization Workshop.

4. Scikit-learn Documentation. https://scikit-learn.org/stable/

5. Django Documentation. https://docs.djangoproject.com/

6. Pandas Documentation. https://pandas.pydata.org/docs/

7. Joblib Documentation. https://joblib.readthedocs.io/

8. SpamAssassin Public Corpus. https://spamassassin.apache.org/publiccorpus/

9. UCI Spam Dataset. https://archive.ics.uci.edu/ml/datasets/spambase

10. Enron Spam Dataset. https://www.cs.cmu.edu/~enron/

11. TfidfVectorizer – Scikit-learn.

https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.Tfidf

Vectorizer.html

12. Logistic Regression – Scikit-learn.

https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression

13. Accuracy Score – Scikit-learn.

https://scikit-learn.org/stable/modules/generated/sklearn.metrics.accuracy_score.html

14. Text Classification with Scikit-learn.

https://scikit-learn.org/stable/tutorial/text_analytics/working_with_text_data.html

15. Natural Language Toolkit (NLTK). https://www.nltk.org/

16. BERT for Spam Classification. Devlin et al. (2019). BERT: Pre-training of Deep

Bidirectional Transformers for Language Understanding.

17. SMOTE – Synthetic Minority Over-sampling Technique. https://imbalanced-

learn.org/stable/references/generated/imblearn.over_sampling.SMOTE.html
18. GridSearchCV – Scikit-learn.

https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearch

CV.html

19. Heroku Deployment Guide for Django Apps.

https://devcenter.heroku.com/categories/python-support

20. MLflow – Machine Learning Lifecycle. https://mlflow.org/