Real time detection of
fraudulent telephone calls
Problem Definition
Mobile phone scams have become a widespread problem.
Scammers use convincing tactics to steal money, often through calls
from real people or automated systems. Common methods include:
● Investment Opportunities: Offering high-return investments that
seem too good to pass up.
● Free Offers: Promising free trials, grants, student loans, or lottery
winnings, often requiring personal information or a small fee to
claim.
● Threats: Using scare tactics like threats of lawsuits or jail time
over supposed unpaid loans or taxes.
● These scams exploit trust and fear to manipulate victims into
giving away their money or personal information.
2
3
4
Previous Approaches
● Current solutions employed for the detection of fraudulent
mobile phone calls are by simply whitelisting and blacklisting
numbers.
● Other solutions are there which include a trained model for
detecting fraud using information such as caller's mobile phone
number, duration of call, country of call it originated from etc.
● Lack of fraudulent conversational sample data has limited the
use of Machine Learning in this area.
5
My Approach
Testing
Collecting
Data
Making
the AI
Model
6
Collecting Data
Twitter Data SBC Data Test Data
Tweets describing In order for the AI Model Audio Recordings from
fraudulent calls are to develop the ability to different websites and
collected and differentiate between social media are collected
to make the Test Data. Due
processed using fraudulent calls and non
to scarce availability of
various NLP techniques fraudulent ones, SBC, a
such data, only 10 samples
to extract the relevant public dataset containing
were found. Likewise, 10
information from it. This call recordings and samples of non fraudulent
helps in overcoming transcripts of various conversations were also
the lack of fraudulent everyday conversations added to the Test Data.
conversational data. is used. 7
Making the AI Model
Dataset containing Twitter and
TWITTER AND SBC DATA SBC Data are given to the Model
Text Normalization is done using
DATA PREPROCESSING various NLP techniques
XLNet Transformer model is given
XLNET TRANSFORMER
the tokenized data
Hyperparameters of the Model are
OPTIMIZING
tuned to give best performance
The Model is trained for 5 epochs
TRAINING
which takes 10 minutes to complete
8
Testing
● At the end of the Model training phase, it had achieved a training
accuracy rate of 98% and a cross validation accuracy rate of 93%.
● The call recordings of the Test Data are converted from speech to
text using IBM Watson's Speech To Text with speaker diarization so
that the speech of different speakers are separated. This proved to
be better than converting the whole conversation into a single block
of text.
● The AI Model then predicts the type of each call recording of the Test
Data. This approach gave an accuracy rate of 90% on the Test Data.
9
Result
Metric Score
Accuracy 90%
Precision 100%
Recall 80%
F1 Score 89%
10
Conclusion and Applications
● Test results show that such frauds can be detected effectively and
could help a large community of mobile phone users.
● Accuracy rate of 90% is significantly better when compared to the
existing approaches for this problem.
● This solution can be implemented via a mobile app to alert the
user in realtime of fraudulent calls.
● In order to ensure Data Privacy, recordings of the call should be
deleted after they are analysed.
● This Model will be continuously updated using newer twitter
messages as well as confirmed fraud calls from the app users
11
Thanks!
Any questions?
12
