Introduction to Big

Data Management
Engineer Merna Magdy
Engineer Omar Ayman
 Confusion Matrix

A confusion matrix is a matrix that summarizes the

performance of a machine learning model on a set of test data.
It is often used to measure the performance of classification
models, which aim to predict a categorical label for each input
instance

It is a 2-D machine learning validation measure that compares

predicted and actual classifications of a classifier, evaluating
its performance using the data in the matrix
 Confusion Matrix

Actual
Class +ve -ve
Predicted

+ve TP FP
True Positive (TP): Actual +ve predicted +ve

-ve FN TN True Negative (TN): Actual –ve predicted –ve

False Positive (FP): Actual –ve predicted +ve

False Negative (FN): Actual +ve predicted –ve

 Key Concepts and Terms

Accuracy Precision Recall F1-Measure

Accuracy is the share of Precision is the share of Recall shows the share of The harmonic mean of
correctly classified true positive predictions true positive predictions precision and recall,
objects in the total in all positive made by the model out of providing a balanced
number of objects. In predictions. In other all positive samples in measure. It combines
other words, it shows words, it shows how the dataset. In other precision and recall into
how often the model is often the model is right words, the recall shows a single metric, useful
right overall when it predicts the how many instances of when there is an
target class the target class the imbalance between
𝑡𝑜𝑡𝑎𝑙 # 𝑜𝑓 𝑐𝑜𝑟𝑟𝑒𝑐𝑡𝑙𝑦 𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑒𝑑 model can find positive and negative
𝑐𝑜𝑟𝑟𝑒𝑐𝑡𝑙𝑦 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑 classes
𝑡𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑖𝑛𝑠𝑡𝑎𝑛𝑐𝑒𝑠 𝑐𝑜𝑟𝑟𝑒𝑐𝑡𝑙𝑦 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑
𝑡𝑜𝑡𝑎𝑙 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑
𝑡𝑜𝑡𝑎𝑙 𝑎𝑐𝑡𝑢𝑎𝑙 2 ∗ 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 ∗ 𝑟𝑒𝑐𝑎𝑙𝑙
𝑇𝑃 + 𝑇𝑁
𝑇𝑃 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑟𝑒𝑐𝑎𝑙𝑙
𝑇𝑃 + 𝑇𝑁 + 𝐹𝑃 + 𝐹𝑁 𝑇𝑃
𝑇𝑃 + 𝐹𝑃
𝑇𝑃 + 𝐹𝑁
 Predicted
Class a b c d Total

a 5 23 17 17 62

b 10 540 21 14 585
Actual

c 166 96 436 110 808

d 1 2 5 87 95

Total 182 661 479 228 1550

Class a Class a b c d Total
!" %
Precision =
!"#$"
=
&'(
a 5 TP 23 FN 17 FN 17 FN 62

Recall =
!"
=
% b 10 FP 540 21 14 585
!"#$) *(

"+,-./.01 ∗ 4,-566 %
c 166 FP 96 436 110 808
F1-Score = 2 ∗ =
"+,-./.01#4,-566 &((
d 1 FP
2 5 87 95
%
Accuracy (class a) = 𝑅5 =
*( Total 182 661 479 228 1550
% # %78 # 79* # ':
Overall Accuracy = ∗ 100 = 68%
&%%8

%
Weighted Precision = 𝐴𝑐𝑡𝑢𝑎𝑙5 ∗ 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛5 = 62 ∗
&'(

;-<=56! *( %
Weighted Average Precision = ∗ 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛5 = ∗
<0<56 &%%8 &'(
Class b Class a b c d Total
!" %78
Precision =
!"#$"
=
**&
a 5 23 FP 17 17 62

Recall =
!"
=
%78 b 10 FN 540 TP 21 FN 14 FN 585
!"#$) %'%

"+,-./.01 ∗ 4,-566 %78

c 166 96 FP 436 110 808
F1-Score = 2 ∗ =
"+,-./.01#4,-566 *(9
d 1 2 FP 5 87 95
%78
Accuracy (class b) = 𝑅> =
%'% Total 182 661 479 228 1550
% # %78 # 79* # ':
Overall Accuracy = ∗ 100 = 68%
&%%8

%78
Weighted Precision = 𝐴𝑐𝑡𝑢𝑎𝑙> ∗ 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛> = 585 ∗
**&

;-<=56" %'% %78

Weighted Average Precision = ∗ 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛> = ∗
<0<56 &%%8 **&
Class c Class a b c d Total
!" 79*
Precision =
!"#$"
=
7:?
a 5 23 17 FP 17 62

Recall =
!"
=
79* b 10 540 21 FP 14 585
!"#$) '8'

"+,-./.01 ∗ 4,-566 ':(

c 166 FN 96 FN 436 TP 110 FN 808
F1-Score = 2 ∗ =
"+,-./.01#4,-566 &(':
d 1 2 5 FP 87 95
79*
Accuracy (class c) = 𝑅- = '8'
Total 182 661 479 228 1550
% # %78 # 79* # ':
Overall Accuracy = ∗ 100 = 68%
&%%8

79*
Weighted Recall = 𝐴𝑐𝑡𝑢𝑎𝑙- ∗ 𝑅𝑒𝑐𝑎𝑙𝑙- = 808 ∗
'8'

;-<=56# '8' 79*

Weighted Average Recall = ∗ 𝑅𝑒𝑐𝑎𝑙𝑙- = ∗
<0<56 &%%8 '8'
Class d Class a b c d Total
!" ':
Precision =
!"#$"
=
(('
a 5 23 17 17 FP 62

Recall =
!"
=
': b 10 540 21 14 FP 585
!"#$) ?%

"+,-./.01 ∗ 4,-566 &:7

c 166 96 436 110 FP 808
F1-Score = 2 ∗ =
"+,-./.01#4,-566 9(9
d 1 FN 2 FN 5 FN 87 TP 95
':
Accuracy (class d) = 𝑅@ = ?%
Total 182 661 479 228 1550
% # %78 # 79* # ':
Overall Accuracy = ∗ 100 = 68%
&%%8

':
Weighted Recall = 𝐴𝑐𝑡𝑢𝑎𝑙@ ∗ 𝑅𝑒𝑐𝑎𝑙𝑙@ = 95 ∗
?%

;-<=56$ ?% ':
Weighted Average Recall = ∗ 𝑅𝑒𝑐𝑎𝑙𝑙@ = ∗
<0<56 &%%8 ?%
Thank you!J
[email protected]
[email protected]