1 s2.0 S2666285X21000893 Main
1 s2.0 S2666285X21000893 Main
a r t i c l e i n f o a b s t r a c t
Keywords: Animal intrusion is a major threat to the productivity of the crops, which affects food security and reduces the
Internet of things (IoT) profit to the farmers. This proposed model presents the development of the Internet of Things and Machine
Raspberry Pi learning technique-based solutions to overcome this problem. Raspberry Pi runs the machine algorithm, which is
Region-based convolutional neural network
interfaced with the ESP8266 Wireless Fidelity module, Pi Camera, Buzzer, and LED. Machine learning algorithms
(R-CNN)
like Region-based Convolutional Neural Network and Single Shot Detection technology plays an important role
Single shot detector (SSD)
TensorFlow to detect the object in the images and classify the animals. The experimentation reveals that the Single Shot
Twilio Detection algorithm outperforms than Region-based Convolutional Neural Network algorithm. Finally, the Twilio
API interfaced software decimates the information to the farmers to take decisive action in their farm field.
1. Introduction Pi Camera, LED, and Buzzer interacting with the cloud a new service in
the domain. The peripheral part adopted wireless technologies such as
Agriculture is the backbone of the Indian economy, where more than WiFi for cooperating with the data center by an advanced IoT gateway.
60% of the country’s population is directly or indirectly depends on this Pi Camera is used to capture real-time images in a farm field 24 × 7
sector. Where they need to feed this huge increasing population year i.e., day and night. The low cost and ease of programming controller
by year with the decreasing land cultivating size. In near future, it is Raspberry Pi for coordination of hardware part and data transferred
expected to have around 15-20% of food commodities to get increased through ESP8266 WiFi module, which uses TCP/IP protocol [3].
within 5 years [1]. Even though a huge number of the population depen- Machine learning is a branch of artificial intelligence used for
dent on this sector, they are still in uncertainty to lead their life in this data analysis to automate the analytical model building, which iden-
sector. The reason for this may be inter and intra farm field variability’s tifies the pattern and objects to make a decision [4]. In this work, a
such due environment, seed selection, fertilization inputs, irrigation, etc deep machine learning algorithm for object detection and classification
[1]. Nowadays, one more important factor causing crop loss is an ani- model is trained and tested. Twilio communication is API interface soft-
mal intrusion into the farm field. The conflict between the animal and ware which used to communicate throughout globally by creating its
farmers is becoming common all over the region. That too in hill station own/private network [5]. It is used to forward the decisive information
area and adjacent to the forest area have major issues and the farmers to the framers, which has control through the Raspberry Pi controller.
suffer a huge loss. To date, they use some traditional and current meth- Here from all this motivated to propose an effective model to protect
ods to overcome this issue like Hellikites, Shot Gun, String and Stone, the crop from animal intrusion through modern technology.
use of electrified welding mesh fence etc, but not up to the expecta-
tion of protecting their crops. Also, few attempts were tried to solve this
2. Related works
conflict by using technology such as IoT and Machine learning, which
is called AIoT (Artificial Intelligence for the Infrastructure of Internet of
S. Giordano et al., [6] worked and developed an IoT application for
Things). Our proposed model uses IoT and Machine learning concepts
the protection of crops from animal intrusion in the crop field. To collect
based solution to it.
or monitor the crop field, authors used the wireless technologies such as
IoT (Internet of Things) controls the Things that are connected to it
6LoWPAN, WiFi, and ZigBee with the advanced IoT gateway. An ultra-
and transfers the data over the network. The IoT technology enables the
sound repeller device was developed to work even in partial and total
collection of real-time data from the farm field using Sensors and various
darkness using the solar panel along with LiPo batteries. To improve
electronic components [2]. In this work, we present the coordination of
the efficiency of this device used PIR (Passive Infrared) sensor, which
takes care of the frequency transmission and networking operation by
∗
Corresponding author.
E-mail address: [email protected] (K. Balakrishna).
https://doi.org/10.1016/j.gltp.2021.08.061
Received 31 May 2021; Accepted 28 June 2021
Available online 12 August 2021
2666-285X/© 2021 The Authors. Publishing Services by Elsevier B.V. on behalf of KeAi Communications Co. Ltd. This is an open access article under the CC
BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
K. Balakrishna, F. Mohammed, C.R. Ullas et al. Global Transitions Proceedings 2 (2021) 169–174
transmitting a small size frame at a distance of 50m. This communica- is used to run a machine learning algorithm. Pi cam is used to capture
tion happens using the RIOT-OS software, when the animal is detected the intruder images entering the agricultural field. The images are then
it produces a sound of 120dB. The performance of this device reduces analyzed by the machine learning algorithm running on the Raspberry
below 90% if the distance from the gateway is above 60m and it will pi board and conclusions are drawn as an output of machine learning
never work if the distance crosses 100m. algorithm. If any danger is sensed the Raspberry pi generates sounds of
Mukesh Mahajan et al., [7] worked on protecting the crop in the different frequencies with the help of a buzzer and an input signal is sent
farm field from animals such as buffaloes, cows, goats, and birds using to ESP8266 nodemcu which is in communication with the firebase IoT
a PIC microcontroller. The developed model uses the motion sensor to cloud which sends messages to the farmer through the android applica-
detect the animals that are near to farm field and the sensor signals the tion [12]. If the intruder is detected at night, the LED lights and buzzer
microcontroller to take appropriate action by farmers. The PIR-based are triggered simultaneously to divert the intruder away from the field.
motion sensor is used to detect the animals and the buzzer is used to no- The proposed model consists of two parts as Hardware and Software.
tify the farmers based on the microcontroller instruction. Here authors Raspberry pi acts like a heart for the hardware part and it is interfaced
claim that this model avoids the farmers staying for 24 hours in the farm with components like a voltage regulator, Pi Camera, LED lights, WiFi,
field and barricade their crops. and Buzzer. The software part for the hardware is done through embed-
Iniyaa K K et al., [8] worked on protecting crops by animals using ded C and for computer vision prediction used machine learning models
deep machine learning and Convolutional neural network algorithm. like R-CNN and SSD for object detection and predicts the animals.
The author aims to protect the crops from animals and not harm both
a Hardware
animals as well as humans in the conflict. Due to this, the authors de-
veloped a model to divert the animals nearer to the crop fields. The The hardware section discusses the associated factors in the selection
machine-learning algorithm-based model is developed to detect the an- of the components for farm field regular monitoring to crop protection.
imals coming nearer to the farm field using the neural network concept Processor
through the computer vision technique. In this model, the farm field is Raspberry Pi 4 controller unit is integrated with Broadcom BCM2711
monitored using a camera placed at the farm field at a regular interval of and Quad-core Cortex-72 processor which works for 64-bit SoC (System-
time. The algorithm detects the animal’s movement through the camera on-Chip). This controller works for a voltage of 5V and communicates
frames using various libraries function and concepts of neural networks easily with other connectivity such as ESP8266, Pi camera, etc. It pro-
and plays appropriate sound to divert the animals away from the farm vides a set of general-purpose input/output pins to have control over
field. electronic components for physical computing for IoT technology, also
D. Kalra et al., [9] worked and developed a model to protect the crop the programming is easy and the cost is low.
from insects and small animals through the sensor and also for control Pi Camera
irrigation using the IoT technologies. The Arduino UNO microcontroller Pi Camera V2 module is 8-megapixel custom-designed board that is
works like a heart for the proposed model in managing proper irrigation equipped with a flexible ribbon cable, making it compatible with Rasp-
and crop protection. The irrigation is managed automatically on/off wa- berry Pi. The camera module takes pictures in two versions i.e. Standard
ter siphons depending on the dampness parameters of the farm field. The version and the NoIR version [13]. The Standard version; is designed to
crops are protected by insects, animals, etc through the use of deliberate capture images in normal light and the NoIR version; is designed to cap-
sensors connected in the farm field; sensors estimate the motion of in- ture images in dark light using the infrared light source.
sects and animals nearer to the crop and sent the signal to the Arduino WiFi Module
Uno microcontroller for calculation of distance and all. Based on the ESP8266 WiFi module allows the Raspberry Pi controller to con-
calculated distance values, the microcontroller enables high-frequency nect with WiFi network using TCP/IP (Transmission Control Proto-
sound. col/Internet Protocol) connection. It operates for a frequency of 2.4GHz
Raksha R and Surekha P, [10] worked and developed a prototype to using the serial communication UART to transmit and receive data. Two
monitor the crops and warning the wild animals based on two emerg- general-purpose pins such as GPIO0 and GPIO2 are used to decide to
ing technologies such as IoT and Machine learning. The IoT components transmit/receive for serial input/output purposes, which are connected
used are like PTZ (Pan-Tilt-Zoom) camera, GSM module, Sensors, and to ground and VCC.
Arduino UNO microcontroller and Machine learning algorithm for clas- Power
sification of the animals are done using KNN (K-Nearest Neighbor) Al- A 12V Lead-acid battery is used for the power supply of electronic
gorithm, Logistic Regression, and SVM (Support Vector Machine) Algo- components connected in the hardware part, which consists of 6 cells
rithm. Datasets of elephants, horses, Zebra, etc are taken in total in some connected in series each producing 2.1V. The lead-acid battery can’t
605 images. SVM provides an accuracy of 89.6% compared to the KNN generate the power itself, where it stores power received from another
and Logistic Regression model for the iterated regularization parameter source for this it is called a storage battery. For example here the 12V
of C=100. battery can supply a current of 10mA for a duration of 12.5 hours,
Several states of the art from the above-proposed prototype and clas- anyhow if consumption of current decreased means duration can be in-
sifier models work on IoT and detect the animals using machine learning creased.
algorithms. But still lack in achieving the expected performance, so this
a R-CNN Model
application is still in infancy for real-time. So here we are proposing a
better model based on IoT and Machine learning algorithms to protect R-CNN approach here used for the object detection uses deep models
the farm field from intruders. as shown in Fig. 2. The R-CNN is composed of four main parts such as se-
lective search, pretrained CNN, category prediction, and bounding box
3. Proposed methodology prediction [14]. For the input images, selective search is applied to select
multiple high-quality proposed regions, which are in multiple scales and
The crop protection from animal intrusion is done using the pro- different sizes and shapes. In between the selective search and output,
posed model using IoT and Artificial Intelligence technologies as shown pretrained CNN is placed. Pretrained CNN works on forward computa-
in Fig. 1. tion to extract the features for output, which gather input as required
The alternating current of 12V is converted into a direct current of by the network through the proposed region. For object classification,
5V through a bridge rectifier circuit and 7805 voltage regulator to op- multiple Support Vector Machines were trained using each proposed re-
erate Raspberry pi. ESP8266 is interfaced with a Raspberry pi board to gion of both features and labeled category. Then for the ground truth
provide firebase cloud connectivity to the system [11]. Raspberry pi4 bounding box prediction linear regression model is trained using each
170
K. Balakrishna, F. Mohammed, C.R. Ullas et al. Global Transitions Proceedings 2 (2021) 169–174
Fig. 1. Block diagram of the proposed model using IoT and artificial intelligence.
Here the SSD adds an extra default box to scale as shown in Eq. (3).
√
𝑠𝑐𝑎𝑙𝑒 = 𝑠𝑐𝑎𝑙𝑒 ∗ 𝑠𝑐𝑎𝑙𝑒𝑎𝑡𝑛𝑒𝑥𝑡𝑙𝑒𝑣𝑒𝑙 (3)
a Tensor Flow
Tensor Flow library is an open source library developed by Google,
which is used to build a numerical computation for the deep machine
learning model [16]. It offers multiple levels of abstraction to build and
train models using API (Application Program Interface). While train-
ing the large machine learning tasks, it used the distributed strategy to
Fig. 2. Object detection using R-CNN detector architecture. train the model on different training on the different hardware config-
urations. This execution allows for immediate iteration and debugging,
which makes it more flexible.
proposed region of both features and labeled bounding box hardware a Twilio
part and it is interfaced with components like a voltage regulator, Pi
Camera, LED lights, WiFi, and Buzzer. The software part for the hard- Twilio is programmable software used for communication functions
ware is done through embedded C and for computer vision prediction using its web API such as make and receive phone calls and text mes-
used machine learning models like R-CNN and SSD for object detection sages globally. Here the Twilio is connected with electronic components,
and predicts the animals. which are called as Cellular IoT connectivity.
Single Shot Multibox Detection (SSD) model consists series of com- The Power supply of 12V is used here, which rectifies it to 5V using
ponents like a base network block and several multiscale feature blocks the bridge rectifier i.e., converting from alternating current into direct
a shown in Fig. 3. In original images, features are extracted using the current using the LM7805IC. Raspberry pi controller works as a heart for
base network block based on the deep Convolutional neural network the proposed hardware part with the interconnected electronic compo-
[15]. Here to detect the small objects in the original images more an- nents powered by a battery of 12V. The controller and other connected
chor boxes are generated using the feature map. Several multiscale fea- components in the hardware part work for a voltage of 5V, so here
ture blocks are used to reduce the size. The multiscale feature blocks regulator is used for stepping down the voltage. To monitor the field
detect the objects of different sizes based on the predicted bounding regularly, the Pi camera module is connected to the controller, which
boxes and anchor boxes. communicates with Pi using the MIPI camera serial interface protocol.
The SSD defines the scale value for each feature map layer manually. The Pi camera is controlled using the Python code and the preview see-
Conv4_3 detects objects starting from the smallest value of 0.2 then in- through by setting an alpha level to 200. Images were captured every 5
creases linearly till it reaches 0.9. Combining the scale value with the seconds, where delay time gives the camera sensor to sense the light lev-
target aspect ratio value to find width and height as shown in Eq. (1) els around the field. The Raspberry Pi communicates with the ESP8266
and (2), here aspect ratio kept as value 1. module through predefined AT commands, UART, and specified Baud
rate. Even the artificial intelligence based machine learning algorithm
√
𝑤 = 𝑠𝑐𝑎𝑙𝑒 ∗ 𝐴𝑠𝑝𝑒𝑐𝑡𝑟𝑎𝑡𝑖𝑜 (1) works on the Raspberry Pi. Images captured are analyzed using deep
machine learning technique as shown in Fig. 4.
The pre-trained Convolutional neural network is taken and retrained
𝑠𝑐𝑎𝑙𝑒 the last layer considering the classification of animals. R-CNN uses selec-
ℎ= √ (2)
𝐴𝑠𝑝𝑒𝑐𝑡𝑟𝑎𝑡𝑖𝑜 tive search to extract the boxes in the image using the basic four regions
171
K. Balakrishna, F. Mohammed, C.R. Ullas et al. Global Transitions Proceedings 2 (2021) 169–174
172
K. Balakrishna, F. Mohammed, C.R. Ullas et al. Global Transitions Proceedings 2 (2021) 169–174
6. Conclusion
Acknowledgement
References
[1] D. Renard, D. Tilman, National food production stabilized by crop diversity, Nature
571 (2019) 257–260, doi:10.1038/s41586-019-1316-y.
[2] M. Abbasi, M.H. Yaghmaee, F. Rahnama, Internet of Things in agriculture: a survey,
in: 2019 3rd International Conference on Internet of Things and Applications (IoT),
2019, pp. 1–12, doi:10.1109/IICITA.2019.8808839.
[3] N.S. Gogul Dev, K.S. Sreenesh, P.K. Binu, IoT Based automated crop protection sys-
tem, in: 2019 2nd International Conference on Intelligent Computing, Instrumen-
tation and Control Technologies (ICICICT), 2019, pp. 1333–1337, doi:10.1109/ICI-
CICT46008.2019.8993406.
[4] S.S. Virnodkar, V.K. Pachghare, V.C. Patil, et al., Remote sensing and machine learn-
ing for crop water stress determination in various crops: a critical review, Precision
Agric 21 (2020) 1121–1155, doi:10.1007/s11119-020-09711-9.
[5] H.S. Kanyal, M. Goel, A.S. Tomar, H.K. Yadav, K. Singh, Object recognition and
security improvement by enhancing the features of CCTV, in: 2020 9th International
Conference System Modeling and Advancement in Research Trends (SMART), 2020,
pp. 245–248, doi:10.1109/SMART50582.2020.9337065.
[6] S. Giordano, I. Seitanidis, M. Ojo, D. Adami, F. Vignoli, IoT solutions for crop pro-
tection against wild animal attacks, in: 2018 IEEE International Conference on En-
vironmental Engineering (EE), 2018, pp. 1–5, doi:10.1109/EE1.2018.8385275.
[7] Smart Crop Protection System from Animals, Int. J. Eng. Adv. Technol. 9 (4) (2020)
2064–2967, doi:10.35940/ijeat.d8732.049420.
Fig. 5. Testing model for the proposed algorithm.
[8] K K Iniyaa, J K Divya, S Devdharshini, R. Sangeethapriya,
Crop protection from animals using deep learning, Int. J. Pro-
gressive Res. Sci. Eng. 2 (3) (2021) 41–44 Retrieved from
Table 2 https://journals.grdpublications.com/index.php/ijprse/article/view/240.
Algorithm performance comparison [9] D. Kalra, P. Kumar, K. Singh, A. Soni, Sensor based crop protection system with IOT
with State-of-the-Art. monitored automatic irrigation, in: 2020 2nd International Conference on Advances
in Computing, Communication Control and Networking (ICACCCN), 2020, pp. 309–
State-of-the-Art mAP∗ (%) 312, doi:10.1109/ICACCCN51052.2020.9362739.
[10] R R., P S., A cohesive farm monitoring and wild animal warning prototype system
Proposed R-CNN 85.22 using iot and machine learning, in: 2020 International Conference on Smart Tech-
Proposed SSD 89.32 nologies in Computing, Electrical and Electronics (ICSTCEE), 2020, pp. 472–476,
S. Giordano et al., [6] 84.51 doi:10.1109/ICSTCEE49637.2020.9277267.
Iniyaa K K et al., [8] 77.45 [11] A. Süzen, B. Duman, B. Şen, Benchmark analysis of Jetson TX2, Jetson nano
and raspberry PI using Deep-CNN, in: 2020 International Congress on Human-
∗
mAP (mean Average Precision). Computer Interaction, Optimization and Robotic Applications (HORA), 2020, pp. 1–
5, doi:10.1109/HORA49412.2020.9152915.
[12] Z. Wan, Y. Song, Z. Cao, Environment dynamic monitoring and remote control of
greenhouse with ESP8266 NodeMCU, in: 2019 IEEE 3rd Information Technology,
Networking, Electronic and Automation Control Conference (ITNEC), 2019, pp. 377–
prediction for the algorithm, SSD outstands in predicting and classifying
382, doi:10.1109/ITNEC.2019.8729519.
the animals compared to R-CNN, and also computation time will be very [13] F. Salih, M.S.A. Omer, Raspberry pi as a video server, in: 2018 International Con-
less. The Twilio API interface decimates the information to the farmers ference on Computer, Control, Electrical, and Electronics Engineering (ICCCEEE),
2018, pp. 1–4, doi:10.1109/ICCCEEE.2018.8515817.
to take necessary action in their farm field [21-25].
[14] R. Gavrilescu, C. Zet, C. Foșalău, M. Skoczylas, D. Cotovanu, Faster R-CNN:an
The R-CNN gives a mean average precision (mAP) of 83.42%, approach to real-time object detection, in: 2018 International Conference and
whereas SSD gives mAP of 89.32% on 300 datasets of animal images Exposition on Electrical And Power Engineering (EPE), 2018, pp. 0165–0168,
for 5 classes as shown in Table 2. However, the architecture of R-CNN doi:10.1109/ICEPE.2018.8559776.
[15] X. Miao, X. Liu, J. Chen, S. Zhuang, J. Fan, H. Jiang, Insulator detection in aerial
is very slow to train and takes approximately 49seconds to generate the images for transmission line inspection using single shot multibox detector, IEEE
test results. Whereas SSD training and testing results will be very faster. Access 7 (2019) 9945–9956, doi:10.1109/ACCESS.2019.2891123.
173
K. Balakrishna, F. Mohammed, C.R. Ullas et al. Global Transitions Proceedings 2 (2021) 169–174
[16] N. Kumar, M. Rathee, N. Chandran, D. Gupta, A. Rastogi, R. Sharma, CrypTFlow: [22] K. Balakrishna, WSN-based information dissemination for optimizing ir-
secure tensorflow inference, in: 2020 IEEE Symposium on Security and Privacy (SP), rigation through prescriptive farming, IJAEIS 11 (4) (2020) 41–54
2020, pp. 336–353, doi:10.1109/SP40000.2020.00092. http://doi.org/10.4018/IJAEIS.2020100103.
[17] N.T. Le, J.W. Wang, C.C. Wang, T.N. Nguyen, Novel framework based on HOSVD [23] M.K. Kumar, B.D. Parameshachari, S. Prabu, S. liberata Ullo, Comparative analysis
for Ski goggles defect detection and classification, Sensors 19 (24) (2019) 5538 p.. to identify efficient technique for interfacing BCI system, IOP Conference Series:
[18] D.L. Vu, T.K. Nguyen, T.V. Nguyen, T.N. Nguyen, F. Massacci, P.H. Phung, A convo- Materials Science and Engineering, 925, IOP Publishing, 2020.
lutional transformation network for malware classification, in: 2019 6th NAFOSTED [24] S. Vadivel, S. Konda, K.R. Balmuri, A. Stateczny, B.D. Parameshachari, Dynamic
Conference on Information and Computer Science (NICS), IEEE, 2019, pp. 234–239. route discovery using modified grasshopper optimization algorithm in wireless
[19] K. Balakrishna, Mahesh Rao, Tomato plant leaves disease classification using KNN Ad-Hoc visible light communication network, Electronics 10 (10) (2021) 1176 p..
and PNN, IJCVIP 9 (1) (2019) 51–63 http://doi.org/10.4018/IJCVIP.2019010104. [25] K. Yu, Z. Guo, Y. Shen, W. Wang, J.C. Lin, T. Sato, Secure artificial intelli-
[20] D.V. Pham, G.L. Nguyen, T.N. Nguyen, C.V. Pham, A.V. Nguyen, Multi-topic misin- gence of things for implicit group recommendations, IEEE Int. Things J. (2021),
formation blocking with budget constraint on online social networks, IEEE Access 8 doi:10.1109/JIOT.2021.3079574.
(2020) 78879–78889.
[21] C. Puttamadappa, B.D. Parameshachari, Demand side management of small scale
loads in a smart grid using glow-worm swarm optimization technique, Microproces-
sors and Microsyst. 71 (2019) 102886 p..
174