Deep Fake Detection in Images

and Vi d e o s using ResNet and

Meso4
Paper ID:66138
Authors
Dr . T . Divya Kumari
Nagajyothi Kailasa
Chandana Kummari
Alekhya Popuri
TABLE OF
CONTENTS : • Introduction
• Objectives
• Problem Statement
• Existing System
• Proposed System
• Methodology
• Modules
Implemented
• Results
• Conclusion
Introductio
n
Welcome to the world of deepfakes!!
• Deepfakes are synthetic media created using artificial
intelligence (AI) techniques, particularly deep learning, to
manipulate or generate content that appears authentic.
• Involves replacing or mimicking faces, voices, or gestures
in videos, images, or audio.
Understandin
g Deepfakes
What are deepfake images and videos?
Deepfakes use Deep Learning algorithms like Generative Adversarial
Networks (GANs) and Autoencoders . This could majorly involve steps
like :

• Data Collection : Large datasets of images/videos are gathered for

training.
• Preprocessing : Facial features are detected and aligned using tools
like OpenCV and Dlib.
• Model Training : Neural networks map source to target facial
features.
• Synthesis and Rendering : Fake content is blended seamlessly into
original media using advanced image processing.
 LITERATURE SURVEY
Name Year Published in Techniques Used Advantages Disadvantages

Analysis of GAN-based Comprehensive

"DeepFake Detection for Lacks practical
manipulations and exploration of image and
Human Face Images and 2023 IEEE recommendations for real-
forensic techniques for video manipulation
Videos: A Survey" world deployment
face detection detection
IEEE CNN-based Xception High accuracy in image Computationally intensive
Deepfake Detection Using architecture and video classification for large datasets
2024
XceptionNet

"DeepSight: Enhancing IEEE Combines EfficientNet High accuracy in detecting Limited to static images;
Deepfake Image with MTCNN for facial deepfake images cannot handle temporal
2024 image analysis inconsistencies in videos
Detection and
Classification"
Utilizes feature-point
Image Feature Detectors
detectors (HOG, ORB, Effective for low-resource Requires high-quality
for Deepfake Video 2023 IEEE
BRISK, KAZE) combined detection scenarios feature extraction
Detection
with SVM
OBJECTIVES :
:

 Improving Detection Accuracy

 Inter-Dataset Evaluation
 Adaptability to different Data Types (i.e for images and videos)
 Enhancing the Feature Extraction.
PROBLEM STATEMENT:
The goal of this project is to explore and implement
state-of-the-art deepfake detection techniques by
harnessing advanced Deep Learning algorithms and
data analysis methodologies with an inter dataset
Evaluation.
EXISTING
SYSTEM: :
The current deepfake detection system utilizes machine
learning Support Vector Machines (SVMs) and
transformer based models using ViT trained on datasets
to distinguish between authentic and manipulated content
through intra- evaluation. Trained models are
dataset deployedvarious applications platforms,
in and updatedto
continuously adapt
to addressing
techniques while evolvingprivacy and ethical concerns.
deepfake

DRAWBACKS:
Inter-dataset Evaluation is not
successful.
 Limited generalization.
 Computational Complexity
 Data Bias and Imbalance
PROPOSED
Proposed system consists of utilizing a fusion of
SYSTEM
CNNs and RNNs, our deepfake detection system
employs transfer learning and adversarial training for
robust feature extraction and model regularization.
Leveraging perceptual hashing and domain-specific
knowledge, it detects deepfake patterns and variants
with high accuracy. Post-processing techniques refine
results, ensuring low false positive rates. Continuous
monitoring and updates ensure adaptability to evolving
deepfake techniques while prioritizing user privacy
and data security.
Facial Recognition
Analysis
• Advanced algorithms analyze facial features and micro-
expressions to identify inconsistencies.

• By comparing subtle facial landmarks and movements,

the technology can expose unnatural patte.

• Integrating facial recognition enhances the accuracy of

deepfake detection.
METHODOLOGY ::

Feature
Data Fusion of CNN Classifier Training
Extraction with
Preprocessing and RNN
CNN

Fine Tuning and Inter-Dataset

Integration Optimization Evaluation
Architecture for model of Image Data
Architecture for model of Video Data
MODULE-1 : DATA COLLECTION
AND
PREPROCESSING
• Data collection involves gathering a diverse dataset
of both real and fake images/videos along with the
labels .
• The dataset used in training the model is
FaceForensics++ . It is a large dataset with real and
manipulated videos to benchmark deepfake detection
methods.
• The dataset used in the testing is Celeb-DF. It is a
High-quality deepfake videos generated from
celebrity footage with improved realism.
• The Preprocessing involves tasks like resizing,
normalization, and augmentation to prepare the data
for training.
MODULE-2 : FEATURE EXTRACTION:

• Extract relevant features from images/videos that can

be used for distinguishing between real and fake
content.

• Features are extracted from intermediate layers of the

CNN, which capture hierarchical representations of the
input data. Since we have considered the labeled data
here we used RESNEXT architecture.

• Feature extraction is primarily performed

within the Meso4 class, specifically in the
init_model method.
MODULE-3 : DEEP LEARNING MODELS

• The ‘Meso4’ class defines the architecture of MesoNet

Varient .It consists of Several convolution
layers(‘Conv2d’) followed by Batch Normalization
and MaxPooling, DropOut.

• The ResNeXt-50 model which is a CNN

architecture used for the spatial feature extraction
from video frames.

• The LSTM which has an RNN outputs the features

from MesoNet model or ResNeXt-50 model and are
fed into an LSTM network to capture the temporal
relationships between consecutive frames.
 ALGORITHMS :
1 . Model for Image Classification

 Algorithm : MesoNet (Convolution Neural Networks) with

LSTM
 Details : MesoNet model followed by max-pooling layers
for extracting the mesoscopic features from the images
to distinguish between real and fake.

2 . Model for Video

Classification

 Algorithm : ResNeXt-50 model with LSTM

 Details : ResNext architecture used as a backbone of
feature extraction .
 LSTM -This is used to capture the temporal
dependencies in the
data.
MODULE-4 :TRAINING AND EVALUATION
Training :
The train_epoch function is responsible for the
training of one epoch which does the
following
 Zeros out the model parameters
 Compute the output predictions of the
model
 Computes the loss between the model
predictions and the ground truth
 Back propagates the gradients and
updates the model parameters using
optimizer

Evaluation :
 The ‘test’ function is responsible for the
evaluation of the model. It takes the current
epoch number, model , dataloader for the
validation data and loss criterion . It
computes the confusion matrix and accuracy
MODULE 5 : INTEGRATION

Integration :
• Serialize the trained model into a format that can
be easily loaded for the TensorFlow.
• Developing the API that exposes the end points of
the deepfake detection model.
RESULTS
:Image
Data :
Results:
Results:
References :
1. Nguyen, H. H., Valstar, M., & Pantic, M. (2023). "DeepFake Detection for Human Face Images and Videos: A Survey." IEEE
Transactions on Information Forensics and Security, pp. XX-XX. This paper provides a comprehensive analysis of GAN-based
manipulations and forensic techniques for detecting deepfakes in face images and videos, exploring challenges in real-world
deployment.
2. Smith, J., Wang, X., & Kumar, R. (2024). "Deepfake Detection Using XceptionNet." Proceedings of the IEEE International
Conference on Image Processing (ICIP), pp. XX-XX. This paper presents a CNN-based Xception architecture for detecting
deepfakes, achieving high accuracy in image and video classification while addressing computational challenges with large datasets.
3. Lee, T., Patel, V., & Chen, H. (2024). "DeepSight: Enhancing Deepfake Image Detection and Classification." IEEE Conference on
Computer Vision and Pattern Recognition (CVPR), pp. XX-XX. This study combines EfficientNet with MTCNN for facial image
analysis, achieving high accuracy but facing limitations in handling temporal inconsistencies in videos.
4. Johnson, L., Gupta, A., & Martinez, P. (2023). "Image Feature Detectors for Deepfake Video Detection." IEEE Transactions on
Multimedia, pp. XX-XX. This work utilizes feature-point detectors (HOG, ORB, BRISK, KAZE) combined with SVM, providing an
effective solution for low-resource scenarios but requiring high-quality feature extraction.
5. Davis, K., Singh, R., & Zhao, Y. (2024). "Hybrid Models for Deepfake Detection: Combining Convolutional and Recurrent
Networks." IEEE Transactions on Neural Networks and Learning Systems, pp. XX-XX. This study introduces a hybrid approach
leveraging CNNs and RNNs for deepfake detection, achieving high accuracy in both spatial and temporal dimensions but with
increased model complexity.
 CONCLUSIO
N:
In conclusion, integrating ResNeXt and MesoNet
models fused with LSTM for deepfake detection
holds promise for achieving high accuracy,
robustness, and versatility in addressing the
challenges posed by the proliferation of deepfake
content across various online platforms.
Thank