The Journal of Supercomputing

https://doi.org/10.1007/s11227-021-03622-w

Recognition of food type and calorie estimation using

neural network

R. Dinesh Kumar1 · E. Golden Julie2 · Y. Harold Robinson3 · S. Vimal4 ·

Sanghyun Seo5 

Accepted: 4 January 2021

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC part of Springer Nature 2021

Abstract
Across the globe, health cognizant among the people is increasing and everyone
wants to maintain a healthy and normal life. But due to the fast moving world, obe-
sity and other related issue becomes the major health problem among the human
beings. According to medical experts, a person is defined as obese when their BMI
is greater than 30 kg/m2. Obesity leads to many diseases like high cholesterol, liver
failure, breathing issues, heart problems, diabetes and sometimes cancer. By eating
healthy foods with high nutrition and low calorie values, we can control the obesity
among the people. Human cannot control their appetite and have the nature of eating
food which they like the most which leads to obesity. Many people have the diffi-
culty in choosing the food items that have good nutrient and low calorific values. If a
system can help the people and give them suggestions about the food and its calo-
rific values, we can find a solution for this obesity problem. In this paper, identifying
the food type and its calorific value estimation is done using multilayer perceptron
model and the results are discussed. From the mixed food items, region of interest
is selected from which the features are extracted. Extracted features are fed as the
input to the MLP. Based on the food volume, the calories present in the food are
calculated. Implementation of the algorithm is done in MATLAB environment for
fruits and food items. The results showed that the level of detection of food item and
accuracy of estimation of calorific level was acceptable.

Keywords  BMI · ROI · Feature extraction · MLP

* Sanghyun Seo
[email protected]
Extended author information available on the last page of the article

13
Vol.:(0123456789)
 R. D. Kumar et al.

1 Introduction

Many people are interested in eating junk food and soft drink which have more
sugar content and high calorific value. Due to less exercise and lack of knowledge
about the dietary food and uncontrollable eating habits among the people, there is
an increase in obesity level. There are several issues related to obesity like hyper-
tension, diabetics, cardiac issues, breathing problems etc. Obesity causes people
to have ligament damages in their knee or other joints due to over body weight.
They also have breathing issues while walking or climbing staircase and heart
gets excess strain in pumping the blood all over the body. A diabetic is a condi-
tion in which the insulin production in the human body gets reduced and leads to
increase in the sugar content in the blood.
Imbalanced dietary food habit and consuming food with less nutrient and more
calorific values are the main reason for obesity. Proper dietary and balanced food
with regular exercise can reduce the obesity level and also helps to have a healthy
life with normal body mass index (BMI) level. By having measured food volumes
which have high nutrient fibre and less calories can help in losing weight. It is
very difficult for a person to measure the food volume and have knowledge about
the nutrient content in each food. Hence, everyone needs an assistant or system
to give information about the food volume and calories and guide them. It is very
difficult to have a dietician every time with everyone to guide about the food.
They need an automated system which can assist anyone anytime just by using
an image of the food and give detail information about its volume and nutrients.
Visual perception of an image is mainly based on the colour and its texture. In
this proposed image processing system, image resizing, feature extraction, seg-
mentation, and classification are performed. Multilayer perceptron (MLP) is used
for classification and based on the food volume the calorific value is calculated.
Digital imaging’s promisingly have better results in recognition food items and
calculating food calories over other traditional methods. Recognizing food items
and calorie estimation to maintain proper dietary information are still a research
challenging task and problem. We proposed an algorithm an improved MLP for
recognizing food items with high performance and accuracy. The main objective
of the proposed work is to provide computer-based solution to maintain proper
dietary intake and BMI.

2 Related work

During the adolescence period, everyone will undergo many changes in both
physiological and psychological aspects. Eating habits get modified as they start
to decide what they want to eat. It is very difficult for them to maintain proper
balanced diet with regular physical activity. Both physical activity and nutrition
are linked and helps in maintaining a normal health with reducing health hazards
[1].

13
Recognition of food type and calorie estimation using neural…

For any expert dietician, it is very difficult to give the full information about
the nutrition value from seeing a plate of food. This is because salt, sugar, fruits,
vegetables, meat, oil contents cannot be examined without tasting. But in case
of natural foods like vegetables and fruits, it is easy for them to give the nutrient
values. Based on the proteins, carbohydrates and fats contents present in the food,
its energy or calories are calculated. A balanced diet with required calories will
help in maintaining proper BMI. Table 1 shows sample of fruits, vegetables, nuts
and their corresponding calories values based on the international food standards
[2, 3].
Generally, a clinical technician or dietician used to monitor the inpatient food and
drinks intake and note the dietary information and make a record of them. In case
of out-patients, it is not easy to get all the details. Since patient cannot remember all
the food and drinks he/she has taken. From this, it is clear that manual approach of
calculating the dietary information is very difficult one. Hence, an automated system
giving information about the nutrient and calories contents in a food is required.
To overcome the drawbacks in the clinical methods, improved methods with fast
response are developed by several researchers. Simple image of the food is given
to the system which automatically calculates the amount of calories present in that
[2–4].
Artificial intelligence field is gaining more interest among the researches and
used in many real-time practical application. Like humans, the AI system is must
take instantaneous solution for a problem. This is achieved by giving required
knowledge, data and trains them to solve the problem effectively. Many research-
ers use ANN, SVM, CNN, KNN, decision tree and other methods to classifying the
fruits and vegetables [5–8].
Amazon Recognition, Vision AI, Computer Vision and Clarifai are some of the
deep learning platforms which are developed for identifying or detecting logos,
celebrity, emotions, objects, texts, foods, vegetables, places etc. [9]. Convolutional
neural network model is used for food classification by providing information such
as name of the food, calories and nutrition value [10, 11].

Table 1  Calories, protein and Fruits/vegetables/nuts Approximate values of

fats present in fruits, vegetables
and nuts samples Calories Protein (g) Fats (g)

Almonds 164 6 14
Banana 105 1.5 0.5
Potato 168 5 0.2
Apple 95 0.9 0.3
White bread 70 2 1
Wheat bread 70 3 1
Egg 72 13 7
Guava 112 4.5 1.6
Pizza 285 13 10
Pomegranate 128 3 2

13
 R. D. Kumar et al.

Pro Trip uses the internet technology and helps users in getting personalised
recommendation about the demographic information, actions, motivations, cli-
mate attributes. Based on the climate attributes, it suggests about the food avail-
ability based on the nutritive value [12].
Ancestral traditional food knowledge is very important for maintaining proper
health. Food images are captured under proper environment for better quality.
The traditional food images are divided into testing, training and validation sets.
CNN model is used for performing the automatic recognition and analysis with
high accuracy [13].
Machine learning-based automatic classification of food image and feature
extraction was performed using convolutional neural network. Here, the food
images are sent to the server by the client for decision-making and the feedback
information are given to the client [14]. Fast RCNN classifier was used for deter-
mining the category labels for region of interest. Later, YOLO framework is used
for computation and predicting. Food images are addresses as binary dataset by
the neural network model to extract the information for food recognition [15, 16].
Food image and calories recognition is a challenging task. By using the cross-
modal alignment and transfer network, food images are synchronised with the
base images and visual embedding is implemented to perform the food recogni-
tion [17]. Ten class of food images in small scale are used as input for the combi-
national model of conventional a bag of feature and linear support vector machine
for food image recognition [18].
Images of Korean food are resized and divided into training and testing groups.
Deep convolutional neural network is used for the food recognition, and the
results are tested with other models like AlexNet, GoogLeNet, VGG and ResNet
[19].
To segment the food item, affinity propagation and unsupervised clustering
method have been adopted. The affinity propagation with agglomerative hierarchi-
cal clustering (AHC) obtains 95% of accuracy. The monitoring of ingestive behav-
iour model has built to monitor and estimate the calories. Food calorie measure-
ment is done with the help of personal digital assistive device in which the food
intake details are recorded and estimation of calories is calculated. It is not easy for
a patient to take these PDA devices always [2, 20].
The input food images have been taken from Smartphone with single and mixed
food items and fed into training and testing. The pre-processing steps are carried
first and followed by vision-based segmentation is done and deep learning algorithm
applied to estimate calories. In the input food image, chopstick is used as a reference
for measurement. The density-based database of the food is considered to evaluate
the food volume, weight and calorie estimation. The estimated weight and calories,
relative average error rate is 6.65% and 6.70% [21, 22].
The two dataset have been collected and trained with single task and multitask
CNN. The above multitask CNN classifier achieved the good result in food identifi-
cation and calorie estimation than single task CNN. The top 10 Thai curries are con-
sidered. The segmented image is fed into the fuzzy logic to identify the ingredients
based on their intensities and boundaries. The calories are calculated by sum of all
the ingredient calories [23–25].

13
Recognition of food type and calorie estimation using neural…

Food shape, size, colour and texture characteristics are used for measuring the
nutritional level from the image [26]. The extracted features are given to a support
vector machine which calculates the calories by using the nutrition table [27]. Few
systems were unable to calculate the nutrient value in case semi-solid food or liq-
uid foods. Mobile phones integrated with the diet data recorder system was capable
of calculating the food volume and gave nutrient and dietary information about the
food [28].
Calculating the food nutrient value from an image is a tedious process since food
actual contents like salt, sugar, oil and other items cannot be measured in that. Based
on the food volume portion, the calories and nutrient are matched with the tables.
Rotten or spoiled food must be informed to the patient so that they can avoid them.
System must give a proper information to the user based on their BMI, that what
type of food, quantity they must intake so that they have a balanced diet [29].
The input is given by mixed food items. The input food item applied to multiple-
hypotheses image segmentation, followed by feature extraction to detect the local
and global features. After extracting features, feed-forward neural network classi-
fier applied to recognize the food items. The output of the experimentation reached
0.947 (MAA) and 0.9599 (SA) accuracy [30].
The food images are collected from the web pages. The dataset with 92,000
images is considered and divided into 23 class foods. The fixed 150 * 150 resolution
of the image has been used. The DCCN model is used to perform food item recog-
nition and compared the output results with other networks for better performance
[31].
The work is proposed with multi-scale multi-view feature aggregation model for
food item recognition. Various features are extracted from supervised CNN like mid
level, high level and deep visual features for class representation. MSMVFA scheme
with various deep networks achieves the high performance and accuracy [32].
The RNN classification is used with multiple hyperparameters to detect malware.
The three feature vector is taken into consideration to perform the implementation.
The feature vector RNN with Word2Vec model provides high-performance malware
detection and stability [33]. The experimentation is carried out with various machine
learning algorithms with 8200 java classes with 14 software systems. The metrics
supported are sensitivity and specificity. Based on the comparative study, multilayer
perceptron proves the best predictor on change proneness using code smell [34].
Recommendation system has been used mainly in E-commerce applications to find
the good product. Here, hybrid action-related K-nearest neighbour similarity (HAR-
KNN) method has been used to classify features from quality aspects. The imple-
mentation is carried with various metrics like MAE, MSE and RMSE with less error
rate and high predictive rate [35].
Several authors with multiple researches work on different pre-processing meth-
ods, segmentation, feature selection and various food item recognition techniques
such as Bayesian classifier, SVM, neural networks, HMM and SOM for improving
the recognition rate and complexity for recognizing food items and calorie estima-
tion is presented with right dietary intake. Research works for recognizing food
items, the provided computer-based solution are not completely convincing and
accuracy is not appropriate to retain dietary intake.

13
 R. D. Kumar et al.

3 Proposed work

Block diagram of our proposed system is shown in Fig. 1. In our system, image of
the fruits / vegetables / food is used as input for the system. The system has two
stages first is training and second is testing. In training phase, from the input images
required features are extracted and classifier is trained to obtain the datasets. Input
image is resized and segmented to obtain the region of interest for which the nutrient
value has to be determined. Features like size, shape, texture are extracted from the
segmented image by using the multilayer perceptron (MLP), these information are
stored as datasets which are used for the real testing phase.
In the testing phase, the images of the food items are given as input and resized.
Images are resized to the size of 128 × 128 and they are segmented. Without per-
forming segmentation of the image, it is not possible to process and extract the
required features. Entire images with plates containing the food items cannot be
used for calculating the nutrient values. So the region of interest has to be segmented
using a proper segmentation techniques. Segmented image feature are extracted, and
the data obtained from the training phase are used to calculating the calorific value.
Images of the food samples are shown in Fig. 2. Food like apple, banana, bread,
guava, pizza and pomegranate are taken as samples for our work.

3.1 Resizing

The first step is resizing the input food image into the size of 128 × 128 pixels
image. The region of interest is obtained, i.e. the edges of the food samples are
determined so that the required feature information can be obtained from them.
The original image, resized image, edge detected region of interest are shown in

Fig. 1  Proposed system architecture

13
Recognition of food type and calorie estimation using neural…

Fig. 2  Sample input images of the food items

Fig. 3  Resized and ROI for different samples of banana

Figs. 3, 4, 5, 6, 7, and 8 for the different samples of banana, apple, bread, pome-
granate, pizza and guava.

3.2 Feature extraction

The ROI is used here for extracting various features from the image. Feature that
are required for calculating the calorific values are size, shape, colour and texture.
Here, three different methods are used to extract features of the image.

13
 R. D. Kumar et al.

Fig. 4  Resized and ROI for different samples of apple

3.2.1 Scale invariant feature transform

SIFT technique is used for identifying the local features of the image. Here, from
the ROI image, the key point and feature vectors are extracted using this SIFT
algorithm.
The entire ROI is used here for obtaining the key points. Scale Space extreme
detection identifies the key point by using the Gaussian function. Gaussian function
is given in Eq. (1). Key point localization method is used to eliminate the low con-
trast key points from the extracted features,
L(x, y, σ) = G(x, y, σ) ∗ I(x, y) (1)
where L(x, y, σ) is the convolution of the original image I(x, y) with the Gaussian
blur G(x, y, σ) at scale. By using the gradient direction the orientation of the image
is modified if required. Image scaling is performed on the oriented image. The Gra-
dient transformation is done using invariance transformation technique as given in
Eq. (2)

𝜃(x, y) = tan−1 ((L(x, y + 1) − L((x, y − 1))∕(L(x + 1, y) − L(x − 1, y))) (2)

13
Recognition of food type and calorie estimation using neural…

Fig. 5  Resized and ROI for different samples of bread

Key point descriptor generates feature vector from the key points. Here, all the
regions are supported with 8 orientation points.

3.2.2 Gabor filter method

This method helps to haul out feature vector from the input resized image. A Gabor
filter will return an array of Gabor objects which contain the pixels/cycle, i.e. wave-
length and orientation. The energy of filtered image is obtained by using Eq. (3).
1 [ { } ]
G(x, y, 𝜃, u, 𝜎) = 2
exp −x2 + y2 ∕2𝜎 2 exp {2𝜋i(ux cos 𝜃 + uy sin 𝜃)} (3)
2𝜋𝜎
𝜃 represents the orientation of the normal to the parallel stripes of a Gabor func-
tion,u is the phase offset, 𝜎 is the sigma/standard deviation of the Gaussian envelope.

3.2.3 Colour histogram

Image Histogram is graphical method which shows the distribution of pixel in an

image. Histogram of a digital image with grey levels in the range [0, L−1] repre-
sented as a discrete function as given in Eq. (4).

13
 R. D. Kumar et al.

Fig. 6  Resized and ROI for different samples of pomegranate

( )
h rk = nk (4)

where, rk is the kth grey level, nk is the number of pixels in the image having
grey level rk.
In colour histogram process, colour distribution of the image is obtained.
Pixel is grouped based on the colour zone and shown in the histogram image.
In grey level, the images pixels are grouped based on the intensities level and
shown in the histogram.

3.3 Segmentation

Dividing an image into equal parts is called as segmentation. This process

makes the computation easy. Here, segmentation process divides the image into
4 blocks of equal size such that 4 × 4 segmented image is obtained as a result.
Gabor filter in each block gives 6 orientation and 5 scales.

13
Recognition of food type and calorie estimation using neural…

Fig. 7  Resized and ROI for different samples of pizza

4 Classification

The output of Gabor filter is used for image feature extraction for which decision-
making is performed using the classification algorithms. Classification methods are
independent of each other, i.e. a change in one training class will not affect the other
one. Classification process can be either supervised or unsupervised. Classification
contains two phases training phase and testing phase. In training phase, based on the
training datasets the reliable class is generated which is used in the testing phase for
classification.
Basic algorithm for classification is given as follows:

1. Acquiring multiple images of each fruits / vegetables / food items for training and
testing
2. Pre-processing and segmentation of the image for obtaining the ROI
3. Based on the boundaries, extracting the appropriate features
4. Measuring the actual volume of the fruits
5. Training the network by using the real volume as target and obtaining training
data sets.
6. Performing the volume estimation on other images and calculating the calories

13
 R. D. Kumar et al.

Fig. 8  Resized and ROI for different samples of guava

4.1 Multilayer perceptron

In supervised learning, data sets are already provided and the results are already
known. But in case of unsupervised learning, no initial data are provided and
based on trial and error methods with several iteration expected result accuracy is
obtained. In semi-supervised learning, sample data are provided. The algorithm
must learn to use these data and update its parameters while learning process.
Human brain can process a complicated task very easily by storing the infor-
mation as the pattern and gains the knowledge to solve those complex problems
by experience when required. Similarly, multilayer perceptron neural network
works like a human brain with feed forward network. MLP uses supervised train-
ing algorithm with feed forward network as shown in Fig. 6 having input, hidden
and output layers.
In Fig. 9, each circle represents a network unit. Except the output layer unit, all
the units are termed as hidden units of hidden layers for which the input or output
is not directly accessible. MLP has the capability to approximate any arbitrary
function when proper number of hidden units, activation function and weights are
provided. Activation function is computed as given in Eq. (5).

13
Recognition of food type and calorie estimation using neural…

Recognized

Unrecognized

Output
Layers
Input Hidden Layers
Layers

Fig. 9  Multilayer perceptron model

[ M
]
∑
xim+1 (n) =f Wijm xj (n) (5)
j=1…N m

The weightage factor is changed by using error rate as given in Eq. (6).

T
∑
new.wm
ji
= wm−1
ij
+𝛾 𝛿im (n)xjm−1 (n) (6)
t=1

Algorithm for MLP.

1. Initialize the neurons and weights

2. Initializing the activation function for hidden and output layers in Eqs. (7) and
(8).
( d )
∑
zh = sigmoid whj xjt (7)
j=0

(H )
∑
yi = softmax vih zh (8)
h=0

3. Weightage function is defined in Eq. (9)

(K )
∑( ) ( )
ri − yi vih zh 1 − zh xjt (9)
t t
Δwih = 𝜂
i=1

4. Error calculation in Eq. (10)

13
 R. D. Kumar et al.

(K )2
1 ∑ ∑( )
err(epoch) = rit − yti (10)
2 x∈X i=1
Validating

5. Modified Weightage function in Eq. (11)

T
∑
new.wm
ji
= wm−1
ij
+𝛾 𝛿im (n)xjm−1 (n) (11)
t=1

4.2 SVM

SVM classification also involves training and testing like all supervised networks.
Single data in a training set contains several features. SVM feature vectors con-
tain the information like colour, shape, size and texture of the food item. Features
extracted during the training phase will be used while performing the testing.
After the identification of region of interest, i.e. the food portion, volume meas-
urement is performed. Both top and side view of the food images are used for cal-
culating the food volume. From the image length, width, height and depth are meas-
ured. Area of the food is calculated by using Eq. (12)
∑
A= Ti (12)
i=1

The entire area and depth of food determine the food volume and it is given in
Eq. (13).
V=A ∗T (13)
Based on the food type, mass of the food is calculated by the system. Mass of the
food is obtained by multiplying the food volume (V) and density (ρ). Equation (14)
represents the food mass calculation
M = 𝜌V (14)
Finally, calorie value of the food image is obtained by multiplying the calculated
mass of the food with the normalized calorific level.

5 Results

Apple, banana, bread, guava, pizza and pomegranate are the 6 food classes which
are considered here. SVM- and MLP-based classifiers are used here and the results
are compared. Based on the precision, recall, F-measure the efficiency and accuracy
of SVM and MLP methods are analysed and the results are tabulated.
Accuracy represents how close the measurements are to the specific value.
Whereas, precision represents how close the measurements values are related to
each other. Here, precision is obtained using Eq. (15).

13
Recognition of food type and calorie estimation using neural…

| true positive|
Precision = (15)
| true positive | + |false positive|

Table 2 shows the precision values of MLP and SVM. It is clear that, MLP’s pre-
cision rate is higher compared to the SVM. Recall is a measure which clearly refers
to how many relevant results are correctly obtained. Recall is obtained from Eq. (16)
|true positive|
Recall = (16)
| true positive| + |false negative|

From Table 3, it is very clear that the MLP has the best recall values when com-
pare with SVM. The F-measure is calculated by taking the precision and recall val-
ues of each class and is shown in Table 4.
F-measure values calculated with respect to different class values using MLP pro-
vides the appropriate results when compared with the various traditional classifiers.

Table 2  Precision rate Class SVM MLP

Apple 0.7 0.8

Banana 0.8 0.9
Bread 0.8 1
Guava 0.6 0.7
Pizza 0.7 0.9
Pomegranate 0.6 0.8

Table 3  Recall rate Class SVM MLP

Apple 0.7 0.8

Banana 0.8 0.9
Bread 0.9 1
Guava 0.6 0.6
Pizza 0.9 0.9
Pomegranate 0.6 0.7

Table 4  F-measure
Class Precision SVM Precision MLP Recall SVM Recall MLP SVM = 2*(p * r/p + r) MLP = 2 * 
(p * r/p + r)

Apple 0.7 0.8 0.7 0.8 0.70 0.80

Banana 0.8 0.9 0.8 0.9 0.80 0.90
Bread 0.8 1 0.9 1 0.85 1
Guava 0.6 0.7 0.6 0.6 0.60 0.65
Pizza 0.7 0.9 0.9 0.9 0.79 0.90
Pomegranate 0.6 0.8 0.6 0.7 0.60 0.75

13
 R. D. Kumar et al.

Table 5  Mean square error, Classifiers Mean square error Sensitivity Specificity

sensitivity and specificity of
SVM and MLP SVM 0.78 79 80
MLP 0.689 86 87

Table 6  Comparison results of Class Sample calo- SVM based calo- MLP based
calorie estimation ries (100 g) rie estimation calorie estima-
tion

Apple 52 61 57
Banana 23.75 29 25
Bread 75 83 80
Guava 79 78 82
Pizza 81.3 88 83
Pomegranate 21.6 27 23

Mean square error helps to detect the number of errors present in the recognized
food item. MSE is evaluated as,
M N 2
1 ∑∑( )
(17)
�
MSE = f (i, j) − f (i, j)
MN i=1 j=1

In Eq. (17) f (i, j) is represented as the original image and f (i, j) is denoted as the
�

recognized character image. M is the height of the image, and N is the width of the
image.
Sensitivity allows how the proposed system classifies the food items. Sensitivity
is evaluated as,
True Positive
Sensitivity =
(True positive + False Negative) (18)

Specificity classify the negative classifiers during the food item recognition pro-
cess and it is evaluated as,
True Negative
Specificity = (19)
(True negative + False positive)

Table  5 shows the mean square error, sensitivity, specificity performance met-
rics of the proposed system. Calories estimation of SVM and MLP are shown in
Table 6. Calorific values are calculated based on the reference calorific value, cal-
culated mass and original mass of the food items. From Table 6, it is clear that the
MLP results are very close to the actual calories values of the food type. SVM calo-
rie value of are very much higher than the actual calories of the samples. When
compared to SVM, MLP method was able to get results which are close to the actual
calories of the samples.

13
Recognition of food type and calorie estimation using neural…

Fig. 10  Classification result

Fig. 11  Overall accuracy

Mean absolute error is measured through the dataset that enhances the computa-
tion of the magnitude in the dataset. These values are combined to identify the error
variations. The class accuracy is related with the precision, recall and the F-meas-
ure values that the incorrectly classified instances are discovered. The classification
result according to the anomaly and weighted average through the normal values is
demonstrated in Fig. 10.
Figure  11 illustrates the overall accuracy generated with the training and the
validation parameters into the training process. The proposed framework with false
quantization values that simulate the error, the result illustrates that the full preci-
sion state of training is combined to identify the high-quality models and the process
of generating the false values for computing the classification parameters.
Figure 12 demonstrates the confusion matrix of the proposed classification tech-
nique that the input class implements on the background process that has been com-
puted using the non-classical components. The generated class with the confusion

13
 R. D. Kumar et al.

Fig. 12  Confusion matrix

matrix identifies the other classes through the highest level of spectrum parameters
that may not be implemented into the direct classification process.
The computation time is the important parameter which has the minimized time
that the efficiency of the proposed technique is measured and compared with the
relevant technique in Fig. 13.
Precision value is calculated with the exactly predicted values through the total
amount of values, recall is measured through the positive values of the whole abso-
lute precision values. F1-score is computed through the average weighted values of
precision and recall that the false positive values are used to distribute the class val-
ues. The accuracy value is measured through the ratio of exactly predicted values
according to the total amount of values which is demonstrated in Fig. 14.
With respect to detailed survey on food item recognition, quality results and bet-
ter performance are obtained from SVM. So, SVM classifier is taken as the base
and the results are compared with improved MLP. From the results attained, it is
observed that proposed MLP algorithm provides better performance and classifica-
tion accuracy when compared to other existing food item recognition techniques.

6 Conclusion

In this research work, food item recognition and calorie estimation are carried out
using SVM and improved MLP models. The proposed work is carried out for single
food item with various pre-processing techniques, segmentation, and feature extrac-
tion. The extracted features are fed into SVM and MLP classifier for recognition.
This application helps the diabetic patient to maintain the proper diet and BMI by
automatically calculating food item calorie values. Implementation of the proposed

13
Recognition of food type and calorie estimation using neural…

Fig. 13  Computation time

Fig. 14  Performance comparison

method is carried out using MATLAB. Food samples of 6 classes are considered
here for calorific value estimation. MLP was capable of obtaining the calories val-
ues close to the actual sample values when compared with the SVM. The perfor-
mance metrics are evaluated in terms of precision, recall, true positive rate, false
positive rate, true negative rate and false negative rate measures. Thus, the proposed

13
 R. D. Kumar et al.

method achieves the high accuracy and efficient results while recognizing the food
items. The future of this work deals with complex food items that are analysed with
different directions, features are extracted with optimized techniques and it can be
integrated with external devices, clustering techniques and various machine learn-
ing models can be used to perform real-time food item recognition to monitor the
food intake consumed by the person and report from time to time which leads to free
from various diseases.

Acknowledgements This work has supported by the National Research Foundation of Korea (NRF)
Grant funded by the Korea government (MSIT) (No. NRF-2019R1F1A1058715) and supported by the
Chung-Ang University Research Scholarship Grants in 2020.

Compliance with ethical standards 

Conflict of interest  The authors declare that there is no conflict of interest.

References
1. Blunt J, Morris J, Trigg J (2020) Diet and physical activity practices of South Australian adolescents. Heli-
yon 6:e04326
2. Pouladzadeh P, Shirmohammadi S, Almaghrabi R (2014) Measuring calorie and nutrition from food
image. IEEE Trans InstrumMeas 63:1947–1956
3. Rajeswari M, Satheesh Kumar R, Subramanian C, Xavi A, Golden Julie E, Harold Robinson Y (2020) Per-
son identification with aerial imaginary using SegNet based semantic segmentation. Earth Sci Inform
13:1293–1304
4. Ahmed Subhi M, Hamid Ali S, Abulameer Mohammed M (2019) Vision-based approaches for automatic
food recognition and dietary assessment: a survey. IEEE Access 7:35370–35381
5. Torres JN, Mora M, García RH, Barrientos RJ, Fredes C, Valenzuela A (2020) A review of convolutional
neural network applied to fruit image processing. MPDI J ApplSci 10:3443
6. Gama S, Himashree BN, Nagashree DB, Hegde M (2019) Precisional detection of calorie information in
Indian food types using image recognition to address Annorexia Nervosa. Int J Eng Sci Comput 9(3)
7. Akhi AB, Akter F, Khatun T, Shorif Uddin M (2018) Recognition and classification of fast food images.
Glob J ComputSciTechnol 18(1):6–13
8. Santhana Krishnan R, Golden Julie E, Harold Robinson Y, Kumar R, Hoang Son L, Anh Tuan T, Viet
Long H (2020) Modified zone based intrusion detection system for security enhancement in mobile ad-
hoc networks. WirelNetw 26:1275–1289
9. Kiourt C, Pavlidis G, Markantonatou S (2020) Deep learning approaches in food recognition, machine
learning paradigms—advances in theory and applications of deep learning. Springer, Berlin
10. Chaudhari A, More S, Khane S, Mane H, Kamble P (2019) Object detection using convolutional neural
network in the application of supplementary nutrition value of fruits. Int J Innov Technol Explor Eng
8(11)
11. Chung DTP, Tai DV (2019) A fruits recognition system based on a modern deep learning technique. In:
IOP Conference Series: Journal of Physics
12. Subramaniyaswamy V, Manogaran G, Logesh R, Vijayakumar V, Chilamkurti N, Malathiand D, Sen-
thilselvan N (2019) An ontology-driven personalized food recommendation in IoT-based healthcare
system. J Supercomput 75(6):3184–3216
13. Wibisono A, Wisesa HA, Rahmadhani ZP, Fahira PK, Mursanto P, Jatmiko W (2020) Traditional food
knowledge of Indonesia: a new high-quality food dataset and automatic recognition system. J Big Data
6(7):1–19
14. Shen Z, Shehzad A, Chen S, Sun H, Liu J (2019) Machine learning based approach on food recognition
and nutrition estimation. In: International Conference on Identification, Information and Knowledge in
the Internet of Things
15. Sun J, Radecka K, Zilic Z (2019) Exploring better food detection via transfer learning. In: International
Conference on Machine Vision Applications (MVA)

13
Recognition of food type and calorie estimation using neural…

16. Wasif SMd, Thakery S, Nagauri A, Pereira SI (2019) Food calorie estimation using machine learning
and image processing. Int J Adv Res Ideas Innov Technol 5(2)
17. Meng L, Chen L, Yang X, Tao D, Zhang H, Miao C, Chua T-S (2019) Learning using privileged infor-
mation for food recognition. In: Knowledge Processing and action analysis MM’19
18. Lu Y (2019) Food image recognition by using convolutional neural networks. arXiv​:1612.00983​v2 [cs.
CV]
19. Park S-J, Palvanov A, Lee C-H, Jeong N, Cho Y-I, Lee H-J (2019) The development of food image
detection and recognition model of Korean food for mobile dietary management. Nutr Res Pract
13(6):521–528
20. Burke LE et al (2005) Self-monitoring dietary intake: current and future practices. J Renal NutrOff J
Council Renal NutrNatl Kidney Found 15(3):281–290
21. Lopez-Meyer P, Schuckers S, Makeyev O, Fontanaand JM, Sazonov E (2012) Automatic identification
of the number of food items in a meal using clustering techniques based on the monitoring of swallow-
ing and chewing. Biomed Signal Process Control 7(5):474–480
22. Pouladzadeh P, Yassine A (2015) FooDD: food detection dataset for calorie measurement using food
images. Lecture notes in Computer science
23. Akpa EAH, Suwa H, Arakawa Y, Yasumoto K (2017) Smartphone-based food weight and calorie esti-
mation method for effective food journaling. SICE J Control MeasSystIntegr 10:360–369
24. Ege T, Yanai K (2017) Image-based food calorie estimation using knowledge on food categories, ingre-
dients and cooking directions. In: Association for computing machinery
25. Nguyen BT, Dang-Nguyen D-T, Dang TX, Phat T, Gurrin C (2018) A deep learning based food recog-
nition system for lifelog images. In: International Conference on Pattern Recognition Applications and
Methods
26. Turmchokkasam S, Chamnongthai K (2018) The design and implementation of an ingredient based
food calorie estimation system using nutrition knowledge and fusion of brightness and heat. Inform
IEEE Access 6:46863–46876
27. Mezgec S, Seljak BK, Net NA (2017) Deep learning food and drink image recognition system for die-
tary assessment. Nutrients 9:E657
28. Bruno V, Resende S, Juan C (2017) A survey on automated food monitoring and dietary management
systems. J Health Med Inform 8(3):272
29. Zhou L, Zhang C, Liu F, Qiu Z, He Y (2019) Application of deep learning in food: a review. Compreh
Rev Food Sci Food Safety 18:1793–1811
30. Jasmine MS, Emmanuel WRS (2020) Food recognition using neural network classifier and multiple
hypotheses image segmentation. ImagSci J 68(2):100–113
31. Wibisono A, Wisesa HA, Rahmadhani ZP, Fahira PK, Mursanto P, Jatmiko W (2020) Traditional food
knowledge of Indonesia: a new high-quality food dataset and automatic recognition system. J Big Data
7(69):1–19
32. Jiang S, Min W, Liu L, Luo Z (2020) Multi-scale multi-view deep feature aggregation for food recogni-
tion. IEEE Trans Image Process 29:265–276
33. Jha S, Prashar D, Viet Long H, Taniar D (2020) Recurrent neural network for detecting malware. Com-
put Secur 99:102037
34. Pritam N, Khari M, Son LH, Kumar R, Jha S, Priyadarshini I, Basset MA (2019) Assessment of code
smell for predicting class change proneness using machine learning. IEEE Access 7:37414–37425
35. Patro SGK, Mishra BK, Panda SK, Kumar R, Long HV, Taniar D, Priyadarshini I (2020) A hybrid
action-related K-nearest neighbor(HAR-KNN) approach for recommendation systems. IEEE Access
63(6):90978–90991

Publisher’s Note  Springer Nature remains neutral with regard to jurisdictional claims in published maps
and institutional affiliations.

13
 R. D. Kumar et al.

Authors and Affiliations

R. Dinesh Kumar1 · E. Golden Julie2 · Y. Harold Robinson3 · S. Vimal4 ·

Sanghyun Seo5 
R. Dinesh Kumar
[email protected]
E. Golden Julie
[email protected]
Y. Harold Robinson
[email protected]
S. Vimal
[email protected]
1
CSE Department, Siddhartha Institute of Technology and Science, Hyderabad, India
2
Department of Computer Science and Engineering, Anna University Regional Campus,
Tirunelveli, India
3
School of Information Technology and Engineering, Vellore Institute of Technology, Vellore,
India
4
Department of Information Technology, National Engineering College, Kovilpatti, India
5
School of Computer Art, College of Art and Technology, Chung-Ang University, 4726
Seodongdae‑ro, Daedeok‑myon, Anseong‑si, Kyunggi‑do 17546, South Korea

13