BERT-Based Cyberbullying Detection
BERT-Based Cyberbullying Detection
e ISSN: 2584-2137
Vol. 03 Issue: 04 April 2025
Page No: 2007-2015
https://irjaeh.com
https://doi.org/10.47392/IRJAEH.2025.0293
Abstract
Cyberbullying on social media has been on the rise lately-and with it, the serious psychological effects it leaves
in its wake: anxiety and sadness. That's why early detection and intervention are so crucial. Those traditional
methods of tackling online abuse often fall short when it comes to slang and ever-changing language. That's
because they just can't pick up on the intent behind the words. Our project tackles that problem head-on by
combining text and visual elements in a way that deep learning can really understand. We use a refined BERT
model to put language in context, Demoji to decipher the meanings of emojis and Pytesseract to extract text
from images. That hybrid approach ensures even the most hidden or indirect bullying messages are identified.
We deliver that analysis-and the tools to visualize it-in real-time through a mobile app. That means non-
technical users-parents, teachers and moderators-can easily use it to spot and stop cyberbullying. By
harnessing the latest AI technologies to safeguard vulnerable people, we create a safer online environment.
Keywords: Cyberbullying, Real-time Analysis, BERT Model, Demoji, Pytesseract.
1. Introduction
The world has evolved through different dimensions learning-based models for better accuracy. The deep
through internet in various fields like education, learning-based models used GloVe and SSWE for
sports, entertainment etc,. As there are ups and downs different word embedding techniques. The results
in life, the internet also has its own downsides. The conclude that more than those generated from
biggest problem in this digital world is cyberbullying. traditional machine learning models, deep learning-
In recent surveys it has been shown that about 36.5% based models work better consistently. A new
of the respondents have gone through cyberbullying, approach for cyberbullying detection is developed by
by facing harassment through digital media. Increase using a pre-trained BERT that outperform on
in internet usage lead to increase in cyberbullying numerous NLP tasks. Such models are applicable in
activities at an alarming rate. 87% of the young social catching contextual meanings of words and phrases,
media users have accepted that they have been thereby able to dig deeper into the complexities of
through these kinds of online harassments. It’s a online communication. Research says that the deep
tough task to deal with cyberbullying as it can happen learning-based models would beat the traditional
in different ways like toxic comments, photographs, models when it comes to detecting cyberbullying
videos etc,. Advanced technologies have been used to tasks. Using this advancement, there came BERT-
detect and remove cyberbullying activities on social based detection, which is actually an innovation
platforms to prevent these kinds of harmful developed by Google AI that marks an enormous leap
happenings. But cyberbullying detection is a very in the world of natural language processing. Also,
tough task. Sometimes when we are having a normal BERT can be fine-tuned over specific tasks, which
conversation with friends, they might sound like makes it fit perfectly to handle the task of harmful
bullying but if we examine properly it might not be social media content identification. Though
the same. Many studies based on the Cyberbullying Cyberbullying is a serious and growing problem but
detection suggested the application of traditional the development of deep learning models,
machine learning-based models and advanced deep specifically BERT, might bring some hope for
system take place via a web or mobile application, proposed overall structure of the plan to show the
using React Native, that allows the user to submit interactions of the actors and the subcomponents of
text, images, or emojis for processing. The the use case which highlights how user interaction
Processing Layer uses a highly tuned BERT model to will provide the necessary action to generate
perform deep processing of the text, Pytesseract will proactive or intervention strategies with regard to
extract text from any images submitted by the user, cyberbullying while contributing to inclusive and
and Demoji will interpret emojis, allowing for the safe online experiences, shown in Figure 2.
detection of bullying communicated in all three
formats. The Backend & Database Layer uses Python
with Flask for processing of the user request and
stores the information captured from the users into a
structured SQL/NoSQL tied to further processing.
The proposed architecture allows for user data to be
processed efficiently, securely and accurately, which
will support a reliable and efficient method for
detecting instances of cyberbullying, Figure 1.
5. Sequence Diagram
processing. After pre-processing, the system sends flows and between these components and the various
the cleaned text to a BERT model for feature layers of the architecture all designed to assist the
extraction, which generates numeric values understanding and generative capabilities of the Bert
representing the content in the context of meaning. decision framework, shown in Figure 4.
After my BERT model features, the model is then 7. Proposed System
trained to classify the user submission as bullying or The cyberbullying detection system has been
non-bullying. In the end, the system returns the enhanced using deep learning and natural language
classification back to the users along with alerts or processing (NLP) methods to evaluate text, emojis,
actions if the content contained any harmful material. and images. It takes advantage of a fine-tuned BERT
Overall, this sequence of observations highlights the model to enhance the understanding of language by
structure and flow of data and processing within the determining the contextual meaning of a sentence,
system and highlights the organizations process flow rather than of single words. Pytesseract is to be
aligns to user engagement, shown in Figure 3. utilized to extract text that could be hidden in the
6. Architecture Diagram of The Bert Model image for a greater detection of textual content
capacity so that it can evaluate images that are often
used to bully others, such as memes and screenshots.
Demoji will be used to interpret the meaning of
emojis so that any meaning or sarcasm conveyed in
the symbols are not ignored. Overall, the system is
using these technologies to improve the ability to
detect cyberbullying both more accurately while at
the same time decreasing its risk of a false negative
and improving the reliability overall [6-8].
7.1 Data Collection
Data collection is a vital stage in building a more
precise cyberbullying detection system, involving the
gathering of different datasets from a variety of
digital spaces, including text, images, and video. The
Figure 4 BERT Architecture Diagram goal of the data collection is to create a holistic
dataset that carries representations of multiple forms
BERT (Bidirectional Encoder Representations from of online bullying, so that the model can recognize
Transformers) model is based on the architecture of harmful interactions across different contexts. The
transformers which is particularly constructed for datasets we are collecting contain chat conversations,
language understanding. BERT model is composed comments, Twitter posts, and images that are
of numerous layers of transformers which use self- typically platforms where cyberbullying occurs. One
attention to identify the relevance or relationship or more forms of multimodal content sample will also
between words in a sentence, irrespective of their be collected. Cyberbullying occurs primarily in text,
location in the sentence. The model utilizes this self- however, often includes images, memes, screenshots,
attention model framework in which context is and emojis multi-modal content, which if we only use
provided bidirectionally from the left and right side text we will inadvertently miss. This will allow us to
of each token within a sentence to produce a more identify indirect, covert, or sarcastic forms of
natural understanding of language. When fine-tuned bullying, that are not identifiable using typical text-
to a specific task such as bullying language detection, based approaches. All personally identifiable
BERT is able to apply its wide-ranging context-based information (PII) will be removed or anonymized for
knowledge in predicting based on textual input. In compliance with data protection frameworks like
summary, the architecture diagram depicts the GDPR and CCPA. The identified sample will be
interconnected components, including how data
International Research Journal on Advanced Engineering Hub (IRJAEH) 2010
International Research Journal on Advanced Engineering Hub (IRJAEH)
e ISSN: 2584-2137
Vol. 03 Issue: 04 April 2025
Page No: 2007-2015
https://irjaeh.com
https://doi.org/10.47392/IRJAEH.2025.0293
auto-filtered, then manually filtered to remove features facilitate the ability of the model to identify
irrelevant samples that may be biased to provide, as a hostile language, insults and threats of violence.
goal, a more balanced dataset that represents a sample Sentiment analysis captures negative emotions such
of real-world cyberbullying incidents. This provides as anger, hatred or fear, all of which are often
the detection system with the ability to recognize indicators of cyberbullying. For image-based
different types of online harassment, and improve detection, feature extraction extracts text and patterns
detection, intervention, and prevention strategies in a in images (utilizing OCR) that may suggest malicious
more effective way. intent. The integration of both linguistic and image-
7.2 Pre-processing based information creates better accuracy for
Pre-processing utilizes raw data for use in a data classification and richness of context increasing the
analysis technique by standardizing and cleaning it. ability to detect subtle, indirect or multimodal forms
For example, for text data pre-processing will entail of cyberbullying.
removing stopwords, punctuation, special characters, 7.4 OCR Module
and other irrelevant aspects which do not provide Pytesseract is an OCR tool that can pull text from
substantive analysis. Pre-processing may also entail images such as screenshots, memes, and social media
what we refer to the process of cleaning up the posts. Cyberbullying may not always be
variations of the words, as you will see later on. This communicated through descriptive text, and therefore
entails converting the words to lowercase, stemming, analyzing content that utilizes images is important for
and lemmatizing (or normalizing) the variations of complete detection. Pytesseract converts the text in
words to make sure they have the same general an image to a machine-readable format and allows the
representation. This cleaning up process is system to analyze and process visual media, in
particularly relevant for slang, abbreviations, and conjunction with typical text-based information.
other informal expressions which occur very After text extraction, the text is then processed using
frequently in online communication. For image data NLP and eventually sentiment analysis so that
pre-processing takes place to improve the accuracy of threats, insults, and harmful messaging can be
Optical Character Recognition (OCR) by resizing and classified correctly. The use of OCR for detecting
adjusting the contrast of images with text (e.g., cyberbullying has the added benefit of conducting
memes or screenshots). This is due to the fact that multimodal analysis for the system, which may
changes will improve OCR efficiency for extracting improve the detection of abusive content regardless
the embedded text and make it easier for machine- of the format, providing a safer and more equitable
learning models to analyze. Pre-processing of images online space across diverse digital formats.
may also include emojis which may be converted to 7.5 Emoji Transcription
text descriptions using Demoji, for example, and thus In cyberbullying, emojis can heighten and reinforce a
maintain the emotional context that emojis invariably negative intent, or disguise an abusive message that
alter in meaning of usages of the words. is often difficult to detect with legacy text-based
Consequently, pre-processing develops a structured approaches. Demoji transcribes emojis to text
and standardization process by which the descriptions to allow the system to interpret meaning
cyberbullying detection model can process the data and intent accurately. For example, an insult followed
used for learning analytical submission. by a laughing emoji is representative of mockery of
7.3 Feature Extraction the target, whereas another "stable" message,
Feature extraction detects patterns related to combined with an angry emoji, may represent
cyberbullying through the analysis of textual as well aggression. Systematically converting emojis to text
as visual information. Text analysis involves key enables the system to analyze sentiment, intent, and
features such as word frequency, n-grams (bigrams, emotional signals in a conversational exchange with
trigrams), sentiment scores, and TF-IDF (Term greater precision.
Frequency-Inverse Document Frequency). These 7.6 Text Analysis
A finetuned BERT model improves cyberbullying and emojis to have a multimodal monitoring
detection by evaluating text in a bidirectional manner, approach for detection. In contrast to traditional
as compared to typically unidirectional models. This methods that only analyze the text, this new
means it evaluates each word based on the words that integrated system allows for a more whole
came before it and the words that will come after it. encompassing assessment of online interactions.
Because it is bidirectional, the model can evaluate More specifically, the BERT model is used as a way
context, nuance, and sentiment, meaning it can detect to identify the text's context, the Pytesseract
instances of bullying that are more complex such as recognizes the text from the images, and the Demoji
indirect, sarcastic, or ambiguous types or forms of tools receives emojis in order to identify a potential
bullying. A unidirectional model would evaluate the bullying intent. Based on combining these models,
words and relationships one at a time, whereas, the aim is to ensure higher detection accuracy, even
BERT evaluates relationships within the individual if the data contains sarcasm, slang, or multimedia.
sentence at a deeper level. Which will lead to The real-time prediction promises to allow
detecting examples like, "Great job, genius!" One moderators, educators, and other relevant actors to be
could easily interpret, "Great job!" as positive, proactive and take action, aiming to create a safer
however, the "genius!" gives the impression that the online space.
person was being sarcastic. This is apparent due to 8. Result and Discussion
the context of how the sentence is constructed. By The updated cyberbullying detection system has a
finetuning BERT on a dataset with cyberbullying superior level of accuracy and reliability, thanks to
content, BERT is highly accurate and identifies new deep learning and natural language processing
patterns of abuse with very good recall. (NLP) tools. While existing text-based methods have
7.7 Test Data benefited from support tools like Benenson's 6-form
Test data is a distinct set of data that is leveraged to text categorization system relying on algorithms and
evaluate the performance of a machine learning keywords, this system can analyze information across
model and the model's ability to generalize. While the various formats since it is multimodal. The ability to
training data is the data that the model learns from, analyze images, emojis, and text improves the
the test data evaluates whether a model can perform detection of subtle and indirect forms of
in an area the model has not seen. Test data is an cyberbullying because cyberbullying does not have to
important evaluation for performance metrics such as be straightforward and direct. The modified BERT
accuracy, precision, recall, and F1 score as it ensures model considers contextual meanings about language
the model punctually identifies instances of to vastly improve comprehension, providing
cyberbullying as efficiently as possible minimizing additional support for the detection of sarcasm, slang,
false positives and false negatives. Testing different or indirect threats, which reduces the number of false
examples in the real-world will also allow for the negatives in text-based detection analysis. The OCR
evaluation of bias, weaknesses, and overfitting which program Pytesseract will allow us to extract text from
improves the model before and over time deploying memes and screens, ensuring that any bullying
the model. Future evaluations and new test data will language hidden in the text-as images can be
be analyzed to allow the model to continue to adapt analyzed. The system will use Demoji to allow
the model to changes and patterns of behavior in hidden meanings associated with emojis that may
cyberbullying and maintain the model's relevance include mockery, sarcasm, or emotional cues. This
and usefulness into the future. evidence demonstrates the opportunity provided
7.8 Prediction through the project to promote a safer digital
In this stage, the model that has been previously environment through improved detection of
trained is applied to new data to subsequently predict cyberbullying in real-world online communications.
possible instances of cyberbullying. The system uses 8.1 Precision
deep learning techniques to scope the text, images, In the context of detecting cyberbullying using our
BERT model, precision matters for evaluating how comments - a critical factor in terms of prompt
well the model detected incidents of abusive response and preventing further abuse. Therefore,
language (or abusive behavior) in online language. recall is often analysed in conjunction with precision
More specifically, since the BERT model detects text - to consider both factors to understand model
as bullying vs. non-bullying, precision is calculated performance (high recall means inclusion of cases
as the ratio of bullying incidents detected (true that matter, and minimizing false positive/identifying
positives, TP) to the total of detected bullying (true abusive conversation is high precision).
positives, TP) and non-bullying that were incorrectly 8.3 F1 Score
detected as bullying (false positives (FP)). The F1 score evaluates a BERT model's performance
in cyberbullying detection because it consolidates
precision and recall to assess a model's effectiveness
in a single evaluation metric. In cyberbullying
detection, precision is how many instances the model
flagged as bullying were actually abusive, while
recall refers to the number of actual cases of
cyberbullying the model detected. F1 score is a
balanced evaluation of a model's performance
A high precision metric, in this case, indicates that the especially when there is a trade-off like in precision
model correctly identifies language indicating and recall.
cyberbullying, or in other words, most of the content
it flags as being bullying is in fact bullying. Thus, we
reduce false positive content since most of what is
flagged is highly true. The representation of a
precision metric reinforces the integrity of the
detection system we are measuring, allowing us to
feel confident that the system is effectively
functioning to identify harmful interactions in online
environments.
A model can have a high precision by identifying
8.2 Recall
only a few clear cases of cyberbullying, but missing
Recall is a key metric for measuring the performance
completely. This would reflect a very low recall rate.
of a model and tells us whether the model identified
This would yield a lower F1 score, which would
every applicable incident of abusive language or
imply that the model is precise but that there is little
behavior in online content for cyberbullying
overall contribution to the detection of cyberbullying.
classification. It is calculated using true positive (TP)
A similar comment would relate to a case with a high
divided by true positive (TP) plus false negative
recall for detecting a cyberbullying, but that would
(FN), where bullying was present and was not
mean that the model would detect too many items,
identified by the model.
and the items might be those that are non-relevant or
benign commentary; this would yield a lower
precision, and lower F1 score. Ultimately, a high F1
score, in the context of cyberbullying detection,
would indicate that the BERT model is capturing a
strong proportion of true bullying cases while
filtering out non-relevant material, Figure 5.
For instance, if BERT is used to classify text
messages or social media posts, a high recall score
means the model identified a lot of actual abusive