Proceedings of

2021 the 13th International Conference on

Computer and Automation Engineering

ICCAE 2021
March 20-22, 2021
Melbourne, Australia

ISBN: 978-0-7381-3219-8
IEEE Catalog Number: CFP2196F-USB
2021 the 13th International Conference on Computer and Automation Engineering (ICCAE 2021)

Copyright ©2021 by the Institute of Electrical and Electronics Engineers, Inc. All rights reserved.

Copyright and Reprint Permission:

Abstracting is permitted with credit to the source. Libraries are permitted to photocopy beyond the
limit of U.S. copyright law for private use of patrons those articles in this volume that carry a code
at the bottom of the first page, provided the per-copy fee indicated in the code is paid through
Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923.

Other Copying, reprint, or reproduction requests should be addressed to IEEE Copyrights

Manager, IEEE Service Center, 445 Hoes Lane, P. O. Box 1331, Piscataway, NJ 08855-1331.

Compliant PDF Files

IEEE Catalog Number: CFP2196F-ART
ISBN: 978-0-7381-3220-4

Conference USB Version

IEEE Catalog Number: CFP2196F-USB
ISBN: 978-0-7381-3219-8

Additional copies of this publication are available from

Curran Associates, Inc.
57 Morehouse Lane
Red Hook, NY 12571 USA
+1 845 758 0400
+1 845 758 2633 (FAX)
email: [email protected]
 2021 the 13th International Conference on
Computer and Automation Engineering
(ICCAE 2021)
Table of Contents
Preface ...................................................................................................................................... v
Conference Committee ............................................................................................................. vi

Image and Signal Processing

Detection and Classification of Whiteflies and Fruit Flies Using YOLO ................................................................... 1
Krystoffer Rowick B. Legaspi, Niño Warren S. Sison, Jocelyn Flores Villaverde

Underwater Image Enhancement for Automotive Wading........................................................................................ 5

Hang Zhu, Rui Song

Local Navigation and Docking of an Autonomous Robot Mower Using Reinforcement Learning and Computer
Vision ...................................................................................................................................................................... 10
Ali Taghibakhshi, Nathan Ogden, Matthew West

Emotion Recognition Using Short-Term Analysis of Heart Rate Variability and ResNet Architecture .................... 15
Lowell Nathaniel B. Singson, Maria Trinidad Ursula R. Sanchez, Jocelyn Flores Villaverde

A Noise Removal Approach from EEG Recordings Based on Variational Autoencoders ...................................... 19
Jamal F. Hwaidi, Thomas M. Chen

A Novel Feature Selection Method on Mutual Information and Improved Gravitational Search Algorithm for High
Dimensional Biomedical Data ................................................................................................................................. 24
Chaokun Yan, Xi Kang, Mengyuan Li, Jianlin Wang

Computer Theory and Technology

Comparative Study of Modeling Road-Vehicle Dynamic Behaviour Using Different Data Based Techniques ...... 31
Youssef M. Fathy, Amr Hassan, Mohammad Tawfik, Hesham H. Ibrahim
User Naming Conventions Mapping Learning for Social Network Alignment ........................................................ 36
Zhao Yuan, Liu Yan, Guo Xiaoyu, Sun Xian, Wang Sen

Evolutionary Games and Distributed Decision-Making in a Multi-agent System for Formation Control ................ 43
Shitong Wang, Chunyan Zhang, Jianlei Zhang
STGRN: A Spatial-Temporal Graph Resonance Network for Social Connection Inference ................................... 48
Shengjie Min, Jing Peng, Guangchun Luo, Zhan Gao, Bo Fang, Dingyuan Rao
Classifying Solving Behavior by Handwriting on Tablets ........................................................................................ 54
Ryosuke Goshima, Fumiko Harada, Hiromitsu Shimakawa

iii
Electronics and Control System
Review on Low Frequency Control of Quasi-Zero Stiffness ................................................................................... 59
Bu-yun Li, Chang-geng Shuai, Wei Xu, Zhao-hao Yang
Non-invasive Smartphone Use Monitoring to Assess Cognitive Impairment ......................................................... 64
Nguyen Dinh Thang, Giles Oatley, Andrew Stranieri, Darren Walker
Influence Analysis of Joint Effectiveness Partial Loss on the Motion Capability in the Space Manipulator ........... 68
Wenqian Xu, Gang Chen, Yingzhuo Fu, Xudong Wang
Reinforcing Intrinsic Walking Power with Nudge Reflecting Day-of-Week Pattern ................................................ 73
Takuya Mori, Hiromitsu Shimakawa, Fumiko Harada
The Effects of Different Artificial LED Light Compensation for Rice Crops ............................................................ 77
Analyn N. Yumang, Elvin F. Cruz, Ralph Joshua N. Riel

VDPred: Predicting Voltage Droop for Power-Effient 3D Multi-core Processor Design ......................................... 83
He Xiao, Monodeep Kar, Saibal Mukhopadhyay, Sudhakar Yalamanchili

Robotic Arm Motion Planning Based on Residual Reinforcement Learning .......................................................... 89

Dongxu Zhou, Ruiqing Jia, Haifeng Yao, Mingzuo Xie

Industrial Control and Manufacturing

Noncyclic Scheduling of Multi-cluster Tools with Multi-type Wafers Based on Pareto Optimization ...................... 95
Yuanyuan Yan, Qinghua Tao, Huangang Wang, Wenhui Fan

The Application of Acceleration-Level Quaternion Interpolation in the Visual-Servoing Process ........................ 100
Xiaoyu Zhao, Yang Liu, Zongwu Xie

Event-Based Data Acquisition for Production Process Analyses: A Systematic Mapping Study ......................... 105
Rocky Telatko, Georg Maurer, Dirk Reichelt

A Modified HLFET Algorithm for Parallelization of Manufacturing Tasks with Causal Dependencies .................. 111
Dirk Frosch-Wilke, Vinitha Thanasekaran

Teleoperation Method of Space Manipulator Considering both Safety and Efficiency ......................................... 116
Xudong Wang, Qingxuan Jia, Gang Chen, Wenqian Xu
Prediction of Tensile Strength and Erosional Effectiveness of Natural Geotextiles Using Artificial Neural Network
.............................................................................................................................................................................. 121
Dante L. Silva, Kevin Lawrence M. de Jesus, Edgar M. Adina, Daelson V. Mangrobang, Mareshah D.
Escalante, Nicole Anne M. Susi

Development of a Low Cost System for Lifting Related Injury Analysis ............................................................... 128
Mahmudul Hasan, Miftahur Rahman

iv
 Preface
Welcome to ICCAE 2021 virtual conference. The conference was initially scheduled to take
place at Melbourne, Australia during March 20-22, 2021. Due to the COVID-19 pandemic and
the travel restrictions in place at the time of the conference, the meeting was switched entirely to
a virtual platform. It’s a very difficult decision, but also an opportunity for us to innovate and try a
new style of conference by a yet safer way in this special period. ICCAE 2021 offered an ideal
platform for presentation, discussion, criticism and exchange of innovative ideas and current
challenges in the field of computer and automation engineering.

On March 21, 2021, Prof. Ljiljana Trajkovic (IEEE Life Fellow), Prof. Rajkumar Buyya (IEEE
Fellow), and Prof. Zabih F. Ghassemlooy gave the live keynote speeches. Four parallel
sessions were arranged in the afternoon. All presentations were granted a virtual oral
presentation. The session chair fully performed their duties and made wonderful comments.
Here we would like to express our gratitude to all those who prepared the work. They not only
made extraordinary contributions in the field of control systems and intelligent robot science and
technology, but also their different expertise and keen insights in the academic and industrial
circles. We look forward to their exciting and academic seminars at ICCAE 2021.

The success of ICCAE 2021 depends on the contributions of many individuals and
organizations. With that in mind, we thank all authors who submitted work to the conference.
The quality of submissions this year remains high and we are satisfied with the quality of the
results procedure. The organizing committee also thanked the members of the technical
planning committee and the chairman of the meeting for their strong support. The organizing
committee is responsible for reviewers, who voluntarily sacrifice valuable time to evaluate the
manuscript and provide authors with useful feedback.

Finally, no conference will succeed without the strong support of its participants. We would like
to thank all the authors and attendees for participating in the conference. We wish you have a
stimulating and fruitful time at the conference.

ICCAE Conference Committee

March 20-22, 2021

v
 Conference Committee
International Advisory Committees
Rajkumar Buyya, The University of Melbourne, Australia (IEEE Fellow)
Ljiljana Trajkovic, Simon Fraser University, Canada (IEEE Fellow)
Chandrajit Bajaj, The University of Texas at Austin, USA (IEEE Fellow)

Honorary Chair
Saman Halgamuge, University of Melbourne, Australia (IEEE Fellow)

Conference Chair
Manolya Kavakli, Macquarie University, Australia (IEEE Member)

Conference Co-chairs
X. F. Tong, Beihang University, China
W. Q. Fu, Beijing Institute of Technology, China

Conference Local Chair

Mohammed Bennamoun, The University of Western Australia, Australia (IEEE Senior
Member)

Conference Program Chairs

Marek Ogiela, AGH University of Science and Technology, Poland (IEEE Senior Member)
Zabih F. Ghassemlooy, Northumbria University, UK (IEEE Senior Member)

Conference Treasure
Michalis Tsiknakis, Institute of Mechanical Engineering, Greece

Publication Chair
Ferdous Sohel, Murdoch University, Australia (IEEE Sinor Member)

Publicity Chair
Prabhat Mahanti, University of New Brunswick, Canada

Conference Technical Committees

A.S.M. Kayes, La Trobe University, Australia (IEEE Member)
Alex Mathews, Bethany College, USA (IEEE Member)
Ankur Singh Bist, KIET, Ghaziabad, India
Bálint Vanek, MTA SZTAKI. Systems and Control Lab., Hungary
B.K. Tripathy, VIT University, India (IEEE Senior Member)
Cheng Siong CHIN, Newcastle University, Singapore (IEEE Senior Member)
Fadi Abu-Amara, Higher Colleges of Technology, UAE

vi
Fairouz Kamareddine, Heriot-Watt University, UK
Fayyaz Ahmad, University of Gujrat (UOG), Pakistan
Garrison Greenwood, Portland State University, USA (IEEE Sinor Member)
G M Rahid uz Zaman Rana, the Oklahoma State University, USA
Hamid Reza Karimi, Politecnico di Milano, Italy (IEEE Senior Member)
Heba Mohammad, Higher College of Technology, Abu Dhabi, UAE
Hidetoshi Ikeda, National College of Technology, Toyama College, Japan
Ibrahim Sultan, Federation University, Australia (IEEE Member)
Jee Hou Ho, The University of Nottingham Malaysia Campus, Malaysia
Jocelyn Flores Villaverde, Mapua University, Philippines
José Manuel Molina López, Universidad Carlos III de Madrid, Spain
Kar Peo Yar, Singapore Institute of Technology, Singapore (IEEE Sinor Member)
Kasturi Kanchymalay, Universiti Teknikal Malaysia Melaka, Malaysia
Ki-Sang Song, Korea National University of Education, South Korea
Kreangkri Ratchagit, Maejo University, Thailand
Lewis Nkenyereye, Sejong University, South Korea
Luisito Lolong Lacatan, Ama University, Philippines
Madya Jamaludin Bin Jalani, University Tun Hussein Onn Malaysia, Malaysia
Maki Habib, American University in Cairo, Egypt
Malik Imran Daud, Foundation University Islamabad, Pakistan
Md. Nazmus Saadat, UNIKL, Malaysia
Miroslav Kulich, Czech Technical University, Czech
Mohammad Salah, Hashemite University, Jordan (IEEE Member)
Mohd Faizal Abdollah, University Teknikal Malaysia Melaka, Malaysia
Mohd Syuhaidi Abu Bakar, Universiti Teknologi MARA, Malaysia
NAMOUNE Abdelhadi, Centre Universitaire Ahmed Zabana, BOURMADIA, Algérie
Nirmalya Thakur, University of Cincinnati, USA (IEEE member)
Noel B. Linsangan, Mapua University, Philippines
Paulo Batista, CIDEHUS.UÉ, University of Évora, Portugal
Pshichopov V. Kh, Southern Federal University, Russia
Rosalina Estacio, Jose Rizal University, Philippines
Ryspek Usubamatov, Kyrgyz State Technical University, Kyrgyzstan
Shamim Ripon, East West University, Bangladesh
Srinivasan Alavandar, Agni College of Technology, India
Stephanie Chua Hui Li, Universiti Malaysia Sarawak, Malaysia
Todsanai Chumwatana, College of Digital Innovation and Information Technology, Thailand
Ts Dr Lam Weng Hoe ALogM, Universiti Tunku Abdul Rahman, Malaysia
Ts Dr Lam Weng Siew ALogM，Universiti Tunku Abdul Rahman, Malaysia
Yogan Jaya Kumar, Universiti Teknikal Malaysia Melaka, Malaysia
Yudong Zhang, University of Leicester, UK (IEEE Senior Member)

vii
 2021 the 13th International Conference on Computer and Automation Engineering

Detection and Classification of Whiteflies and Fruit

Flies Using YOLO
Krystoffer Rowick B. Legaspi Niño Warren S. Sison Jocelyn Flores Villaverde
School of EECE School of EECE School of EECE
Mapua University Mapua University Mapua University
Manila, Philippines Manila, Philippines Manila, Philippines
[email protected] [email protected] [email protected]

Abstract—Pests such as whiteflies and fruit flies are very Recently, CNNs have been effective to perform object
small and are impractical to find and locate using only the naked detection. Some have even been successfully used for pest
eye. Pests Detection and monitoring will greatly help the farmers detection. However, there is still a need for implementations
to detect and locate pests even if they are very small and even if with deeper or more layers and that can classify multiple types
they are too far from their farms. In this study, YOLOV3 was of insects. Another underlying need is an implementation that
used to classify and detect objects, specifically whiteflies and fruit can detect small objects such as pests.
flies. The researchers used a Raspberry Pi camera to gather
images. Moreover, the researcher provides desktop and web The main objective of this paper is to detect and classify
application to display images obtained by the Raspberry Pi whiteflies and fruit flies for monitoring purposes. To achieve
camera. Based on the confusion matrix (Table 3), the model this goal, the following specific goals must be first met: (1) set
obtained an overall accuracy of 83.07% in classifying and up a Raspberry Pi with a camera, monitor, and Linux operating
detecting whiteflies and fruit flies. system to run the programs; (2) gather and label images, of
varying factors such as camera angle, lighting, and noise, of
Keywords—Pests detection, REST API, data augmentation, images containing the two pests to be identified; (3) implement
YOLO, Convolutional Neural Network the YOLO algorithm in a REST API service; (4) create a
I. INTRODUCTION website that can be used to view YOLO’s output remotely; and
(5) use a confusion matrix for the analysis of the system’s
Whiteflies are soft-bodied winged insects around 1/12 inch performance.
in length that sucks plant juices to produce a harmful sticky
substance called honeydew. Fruit flies are tiny, yellowish, red- II. METHODOLOGY
eyed insects that feed on fruits. Early detection and monitoring
of these pests can significantly improve the cultivation process A. Conceptual Framework
and the product quality by preventing these pests from The conceptual framework as shown in Fig. 1 was made
spreading, as well as minimizing the amount of pesticide used assuming that the YOLO network already has trained model
[1]. Since these pests are small, it is slow, impractical, and weights. The sole input is an image. It could just be a plain
inaccurate to locate and find them by manual means [2][3]. For image or frames from a video or live camera feed. These will
a long time, researchers have been struggling to incorporate be pre-processed to fit the required input shape. After pre-
computers to automate the detection and counting process. processing, forward propagation to the YOLO network is done
However, it turns out to be a very difficult task for computers to make predictions of the object classes and locations. Then,
because they don’t see images the way we do. Also, they see bounding boxes with class labels are drawn on the image. The
images as matrices of numbers. In addition, there are several output is the drawn image.
other factors to consider such as the variability in images due
to the presence of noise, different camera angles, occlusions,
and lighting conditions [1].
Fortunately, in the past couple of years, the fields of deep
learning and computer vision have reached an all-time peak.
This is due to the fact the hardware needed, such as GPU
clusters, as well as large-scale training data, such as ImageNet
have become a lot more readily available [4]. In recent years,
detection and monitoring of pests have become important
applications of these fields and a useful tool in precision
agriculture [2]. There have been a lot of methods used for
object detection in the past. Some of which was done by Fig. 1. Conceptual Framework
manually hand-crafted features. These were done using spin
images, SIFT descriptors, HOG, and Haar-like features but B. Hardware Block Diagram
proved to be ineffective because of the high variations of Fig. 2 shows the block diagram of the system. The
appearances, illumination conditions, and backgrounds. Raspberry Pi uses a camera to take images or videos that

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 1

YOLO, ran on the Raspberry Pi, can process. It sends the entropy was used as the loss function and was given by the
image frame to the REST API for processing. Once the equation 3. The loss is directly proportional to the difference
processing is complete, the result is displayed on the monitor. between the actual and predicted value. A high loss means that
This result can also be viewed remotely through a website. The the predictions are inaccurate. The optimizer used is the
power supply used is a power bank that is charged by a solar stochastic gradient descent (SGD). It uses the gradient descent
panel when there is sunlight. equation 3.4, but instead of using the entire dataset for every
iteration, it just uses one of the indicated batch sizes. The
optimizer adjusted Θ to minimize the loss. Afterward, the
network was modified into the proposed one. The loss function
used was a multi-scale sum-squared error function that sums all
the sum-squared error of each element in the 24-dimensional
vector in the N x N output matrix. The optimizer used was still
the same SGD.
𝐽(𝜃) = − ∑𝐶𝑖=1 𝑦𝑖 ln(ℎ𝜃 (𝑥𝑖 )) + (1 − 𝑦𝑖 )ln(1 − ℎ𝜃 (𝑥𝑖 ) (3)
𝜕
𝜃 = 𝜃−𝛼 𝐽(𝜃) (4)
𝜕𝜃

𝐼𝑜𝑈
𝑖𝑛𝑡𝑒𝑟𝑠𝑒𝑐𝑡𝑖𝑜𝑛 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑡ℎ𝑒 𝑐ℎ𝑜𝑠𝑒𝑛 𝑏𝑜𝑥 𝑎𝑛𝑑 𝑐𝑢𝑟𝑟𝑒𝑛𝑡 𝑏𝑜𝑥
=
Fig. 2. Block Digram 𝑢𝑛𝑖𝑜𝑛 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑡ℎ𝑒 𝑐ℎ𝑜𝑠𝑒𝑛 𝑏𝑜𝑥 𝑎𝑛𝑑 𝑐𝑢𝑟𝑟𝑒𝑛𝑡 𝑏𝑜𝑥
(5)
C. Modified Darknet-53 Convolutional Neural Network
D. System Flowchart
Every convolutional layer applies batch normalization and
has a leaky rectified linear unit (leaky ReLU), which is given Fig. 3 presents the main flowchart of the system. It starts
by equation 1, as its activation function. by initializing a Boolean variable called textMessageFlag to
true. It then checks if the video stream from the camera is still
𝑥, 𝑥 > 0 supplying frames to the application. If the end of the video
Φ(𝑥) = { (1)
0.1𝑥, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 stream was reached, then the entire process ends. Otherwise,
It takes a 416 x 416 x 3 image and outputs a matrix of size the next step is to resize, if necessary, the image to 416 x 416,
given by equation 3.2. In this case, N is 52, B is 3, and C is 3, which is the input dimension the proposed network expects.
which resulted in a 52 x 52 x 24 matrix where each cell of the Afterward, it is forward propagated into the network which
dimension is a 24-dimensional vector. Each of these 24- will output a 52 x 52 x 24 matrix. Non-maximum suppression
dimensional vectors is encoded as follows: for each anchor, it is applied to the resulting output matrix from the network to
has C classification scores, the object’s midpoint denoted by (x, get rid of all the unnecessary bounding boxes found.
y), the object’s dimensions denoted by (w, h), and an Afterward, the output module will send the necessary outputs
objectness prediction score, which tells how confident it is that to the devices that need them. Finally, the process loops back
it contains an object. The choice of the number of anchor boxes to the part that checks if the end of the video stream is reached.
B is similar to the choice in the YOLOv3 paper [14], but The non-maximum suppression module eliminates all the
instead of clustering the 3 anchor boxes into 9 by classifying in excess bounding boxes. It starts by creating a list of 24-
3 different scales of N: 13, 26, and 52, only one scale was used. dimensional vectors to store all the valid results. It then adds
The scale used was the largest one is because it specializes in
all bounding boxes with an objectness prediction score greater
small objects, hence the 52 as the value of N. The first half of
than 0.5, which is the chosen threshold. It then proceeds to
the network is the base Darknet-53 network. The other half
simply up samples and concatenates previous intermediate remove overlapping bounding boxes. The bounding box with
feature maps. This allows it to get useful semantic information the highest score is chosen. From there the intersection over
from the clear information from the previous feature maps and union (IoU) of each of the remaining boxes are obtained
up-sampled features [14]. This is done until the required relative to the chosen box. The IoU is obtained from equation
dimension N is obtained, and when it is, the number of 5. Those with an IoU greater than or equal to 0.5 are removed.
channels is reduced to the expected from equation 2. This is repeated until there are no more overlapping boxes.
The output module sends the necessary outputs to the
dimensions of the output matrix = necessary output devices. It first checks if there any pests
N x N x [B ∗ (5 + C)] (2) detected. If there are no pests detected, the system will do
nothing. If there are any pests detected, then, rectangles are
The proposed network was trained to learn the necessary drawn and labeled on the image using the midpoint,
weights to make predictions. This was done by providing a
dimensions, and classifications in the list. Afterwards, the
training set and performing back propagation with it. Initially,
resulting image is displayed on the monitor. Lastly, it sends
the base Darknet-53 CNN was pretrained to classify the objects
of interest: whiteflies and fruit flies. This was done because the output image to the website if it requested them.
pretrained networks improve performance [14]. The cross-

2
 Table II shows the data gathered from the system’s results
on the test set of 140 images. These are broken down into three
classifications, which are whiteflies, fruit flies, and unknown.
The rows correspond to what the system predicted, and the
columns correspond to the actual data. There were 287
predicted whiteflies, 0 fruit fly, and 80 unknowns in the first
column which corresponds to the actual whitefly detected, the
total actual whitefly detected was 367. On the second column,
there are 0 whitefly, 184 fruit flies, and 16 unknowns predicted
with a total of 200 fruit flies detected. On the third column,
there are 0 whitefly, 0 fruit fly, and 0 unknown with a total of 0
actual unknown. Precision of the system in detecting whitefly
and fruit fly is 100%. This means that every positive prediction
done by the system is actually positive. The accuracy of the
system in detecting whitefly, fruit fly, and unknown were
85.89%, 97.17%, and 83.07% respectively. The overall
accuracy is 83.07%. Having a precision that is considerably
better than the accuracy means that the system has more
trouble trying to get all the detections than making correct
detections. This is very apparent when it deals with images that
contain multiple pests specially ones where they are stacked or
very close to one another.

TABLE II. CONFUSION MATRIX

Actual
Fig. 3. Flowchart Predicted Whitefly Fruit Fly Unknown
Whitefly 287 0 0
E. Data Gathering Fruit Fly 0 184 0
Unknown 80 16 0
The proponents gathered images of whiteflies, fruit flies,
and images that have no whiteflies and fruit flies on it. 200
images for fruit flies, 200 images for whiteflies, and 70 images Statistical Treatment - Precision measures how many of
for no whiteflies and fruit flies with a total of 470 original the predicted positives are actually positive and can be
images were obtained. All images were labelled accordingly obtained using equations 7. Accuracy measures how many of
using the Visual Object Tagging Tool (VoTT). the predictions were actually correct and can be obtained
through equation 7. In object detection, both the classification
F. Data Augmentation
score and intersection over union (IoU), equation 5, are used
The data augmentation methods used in this study were to determine if the detection is a TP, FP, or FP. IoU is
flipping and cropping. Flipping simply mirrors the image with calculated with respect to the ground truth bounding box and
respect to the horizontal axis. Cropping takes a part of the the threshold is set to 0.5. TP occur when the Intersection over
original image to the specified resolution. Union (IoU) is above the threshold and the classification are
right. FP occur when the IoU is below threshold. False
III. RESULTS AND DISCUSSION
negative happens when the IoU is above the given threshold
Table I tabulated the distribution of the dataset. There is a but misclassifies the object.
total of 470 images originally gathered. Using transformation
methods for data augmentation, random sets of 470 images 𝑇𝑃+𝑇𝑁
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = (6)
were subjected to two different transformations. This resulted 𝑇𝑃+𝑇𝑁+𝐹𝑃+𝐹𝑁
in 465 new images generated from each transformation which
𝑇𝑃
yielded a total of 930 images from the transformations alone. 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 = (7)
𝑇𝑃+𝐹𝑃
This provided a total of 1400 images in the dataset. The dataset
was then split into a training set and a testing set. Overall, there
were 1260 images used for training and 140 images. Fig. 4 shows the desktop application of the system. For
demo purposes only we added an upload file button for the user
TABLE I. DATASET DISTRIBUTION to upload an image. This image will then go through our model
Number of Images to be able to place a box on the detected object which is either
Original Images 470 labelled as whitefly or fruit fly. If Raspberry Pi camera is
Transformed Horizontal Flip 465 integrated with it, the user can simply click the button to
Images Vertical Flip 465 capture the image obtained by the Raspberry Pi camera, and
1400
the image with boxes is displayed.

3
 [3] W. Ding and G. Taylor, “Automatic moth detection from trap images for
pest management,” Computers and Electronics in Agriculture, vol. 123,
pp. 17–28, 2016.
[4] Z.-Q. Zhao, P. Zheng, S.-tao Xu, and X. Wu, “Object Detection with
Deep Learning: A Review,” IEEE Transactions on Neural Networks and
Learning Systems, Jul. 2018.
[5] K. L. Roldán-Serrato, J. Escalante-Estrada, and M. Rodríguez-González,
“Automatic pest detection on bean and potato crops by applying neural
classifiers,” Engineering in Agriculture, Environment and Food, vol. 11,
no. 4, pp. 245–255, 2018.
[6] N. B. Linsangan and R. S. Pangantihon, “FPGA-Based Plant
Identification Through Leaf Veins,” Proceedings of the 2018 5th
International Conference on Biomedical and Bioinformatics Engineering
- ICBBE 18, 2018.
[7] M. J. Macawile, V. V. Quinones, A. Ballado, J. D. Cruz, and M. V.
Caya, “White blood cell classification and counting using convolutional
neural network,” 2018 3rd International Conference on Control and
Robotics Engineering (ICCRE), 2018.
Fig. 4. Desktop Application [8] J. R. Balbin, M. P. Sejera, C. O. A. Martinez, N. A. M. Cataquis, L. M.
H. Ontog, and J. K. Toribio, “Cloud based color coding scheme violator
plate detection through character recognition using image processing,”
IV. CONCLUSION AND RECOMMENDATIONS 2017 7th IEEE International Conference on Control System, Computing
Pest Detection and monitoring are very important and Engineering (ICCSCE), 2017
especially in farms where most whiteflies and fruit flies go [9] J. Redmon and A. Farhadi, “YOLO9000: Better, Faster, Stronger,” 2017
around. It allows the farmers to take actions to prevent the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
2017.
spread of pests and infections that can cause damages to their
[10] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for
crops. With these tools on hand, it is easy for them to Image Recognition,” 2016 IEEE Conference on Computer Vision and
determine if there are pests present in their crops even when Pattern Recognition (CVPR), 2016.
they are far from their farm because they can simply monitor [11] N. S. Yamanoor and S. Yamanoor, “High quality, low cost education
their farm using a web app which can be run on desktop, laptop, with the Raspberry Pi,” 2017 IEEE Global Humanitarian Technology
and mobile phones. With these, they will still be able to know Conference (GHTC), 2017.
if there are pests present. The model used for this has an [12] A. Fawzi, H. Samulowitz, D. Turaga, and P. Frossard, “Adaptive data
accuracy of 83.07% which is a good value to use to detect augmentation for image classification,” 2016 IEEE International
Conference on Image Processing (ICIP), 2016.
whiteflies and fruit flies properly. With this, the researchers
were able to classify and detect whiteflies and fruit flies [13] J. Shijie, W. Ping, J. Peiyi, and H. Siping, “Research on data
augmentation for image classification based on convolution neural
properly based on the results. networks,” 2017 Chinese Automation Congress (CAC), 2017.
[14] J. Redmon and A. Farhadi, “YOLOv3: An Incremental
REFERENCES Improvement,”arXiv:1804.02767v1 [cs.CV] 8 Apr 2018, Apr. 2018.
[1] M. N. Jige and V. R. Ratnaparkhe, “Population estimation of whitefly [15] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look
for cotton plant using image processing approach,” 2017 2nd IEEE Once: Unified, Real-Time Object Detection,” 2016 IEEE Conference on
International Conference on Recent Trends in Electronics, Information Computer Vision and Pattern Recognition (CVPR), 2016.
& Communication Technology (RTEICT), 2017.
[2] W. Li, P. Chen, B. Wang, and C. Xie, “Automatic Localization and
Count of Agricultural Crop Pests Based on an Improved Deep Learning
Pipeline,” Scientific Reports, vol. 9, no. 1, 2019.

4
 2021 the 13th International Conference on Computer and Automation Engineering

Underwater Image Enhancement for Automotive

Wading
Hang Zhu Rui Song*
School of Computer Science and Engineering College of Artificial Intelligence
University of New South Wales Nankai University
Sydney, Australia Tianjin, China
[email protected] [email protected]

Abstract—To improve the driver safety for automotive commonly used for underwater applications. We will focus on
wading, the underwater image enhancement is designed to help the theory of optical propagation features, implement
drivers to deal with fuzzy images shown in highly scattered traditional imaging enhancement methods and deep learning
underwater environments and detect hidden obstacles. In methods on Convolutional Neural Networks (CNN) to find out
addition to the traditional methods, some neural network the best imaging enhancement method to deal with fuzzy
approaches which may perform better are suggested. The underwater images [3, 4]. The main contribution of the paper is
purpose of this paper is to integrate traditional methods and deep that a new underwater image enhancement fused traditional
learning methods to enhance underwater images. The former methods and deep learning method is modified and proposed,
Deep Underwater Image Enhancement Network (DUIENET)
and the results verify that it is superior to the previous
employs a gated fusion network architecture to learn three
approach.
confidence maps which will be used to combine the three input
images into an enhanced result, the white balance is replaced by II. THEORY AND THE METHODS
Multi-Scale Retinex in the modified approach. Then, some
objective evaluation methods are also adopted, mean square A. Theory of Optical Propagation
error method and peak signal to noise ratio method, to compare
In order to choose an appropriate method to improve the
the performance of the different methods. Finally, we chose some
visual quality of recorded underwater images, optical
test images from test data randomly, the results show that the
developed approach is better than the previous methods, the
propagation features should be known. When the light
clarity of the pictures have been indeed improved. travelling underwater, visibility becomes lower as distance and
turbidity increase. The general laws govern the propagation of
Keywords—underwater image enhancement, Convolutional light in water and the methods for measuring the optical
Neural Network, automotive wading parameters of water. Light power underwater can be described
by Beer-Lambert law [5]
I. INTRODUCTION
𝐼(𝑟) = 𝐼(𝑟0 )𝑒 −𝑐(𝑟−𝑟0) (1)
The driving safety is urgent when the road is covered with
deep water in raining days or a flooded urban environment. In where I is the intensity of light, c = a + b is the light attenuation
order to ensure the stability and functionality of the vehicle, the coefficient (a is absorption, b is scatter), 𝑟0 is the transmitter
instantaneous detection of underwater objects is vital and position, and 𝑟 is the receiver position. The type of particles
should be equipped for the driver. However, few literature has and dissolved in the water substances determine the attenuation
been studied the underwater identification technologies of and absorption.
automotive system. Moreover, the measurements of water clarity, which is
Vehicle underwater imaging is very crucial when the based on the depth that a black and white Secchi disc can be
vehicle is wading in the water. It is hard to detect underwater lowered into a body of water, is also utilized in the research. A
obstacles when a vehicle is passing through deep water, horizontal measurement offers the ability to take a Secchi
especially at night. More research should be devoted to the reading which is better than the depth of the water. The
appropriate underwater imaging technology and the horizontal visibility range can be calculated as:
improvement of the range and accuracy [1]. 𝑍𝑆𝐷= Γ/𝑐 (2)
Modern remote sensing technology can sense the
surrounding environment. For example, radar can be utilized where Γ is the coupling constant. In the measurement, knowing
through fresh water, but cannot operate at high water 𝑍𝑆𝐷 , we can characterize the visibility in terms of the number
conductivity. Sonar detection requires stable water, and sonar of attenuation lengths, which will allow us to compare the
signals can be absorbed by muddy water and reflected and results obtained with the results of others.
reframed by turbulence. Light absorption and scattering in
water are significant for optical wavelengths [2]. Therefore, B. Theories of Image Enhancement Methods
optical imaging systems should be specifically designed to be Some common traditional methods are simply described
used in underwater scenarios. The modern imaging sensors that below. White balance algorithm corrects the color casts of the
can provide dense information at high speed rates are image data. In the algorithm, white balance counts the

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 5

occurrence times of each pixel value on the RGB channels. Applying this technology to image processing is to
Setting the maximum and minimum values of 1% to 255 and 0. calculate the corresponding 𝑅(𝑥, 𝑦) according to the image
The remaining pixels map to the value between 0 and 255, so data 𝐼(𝑥, 𝑦) we have obtained, then 𝑅(𝑥, 𝑦) is considered to be
that the values of each value channel are more evenly
the enhanced image, the key point is to get 𝐼(𝑥, 𝑦). The Retinex
distributed in RGB. The result of color balance is achieved.
theory points out that this 𝐿(𝑥, 𝑦) can be obtained by applying
Histogram equalization is a method to adjust contrast by Gaussian blur to the image data 𝐼(𝑥, 𝑦). Therefore, the detailed
using image histogram in image processing filed [6]. In this path of this algorithm is very simple. The specific steps are as
way, brightness can be better distributed on the histogram. This follows:
can be used to enhance local contrast without affecting the
overall contrast. Histogram equalization achieves this function Input: original image data 𝐼(𝑥, 𝑦) and scale.
by effectively extending the commonly used brightness. In the Processing: After the original image is blurred at the
algorithm, the histogram of the original image changes from a specified scale, Calculate the original image 𝐿(𝑥, 𝑦); Calculate
relatively concentrated range to a uniform distribution in all the value of 𝐿𝑜𝑔(𝑅(𝑥, 𝑦)), according to the calculation method
gray ranges. Histogram equalization is used to stretch the of (6); Quantize 𝐿𝑜𝑔(𝑅(𝑥, 𝑦)) to a pixel value from the range
image non-linearly and redistribute the pixel value of the image, of 0 to 255 as the final output.
so that the number of pixels within a certain gray scale range is
roughly the same. The process realized by the above algorithm is often
referred to as Single Scale Retinex (SSR). To get a better effect,
Gamma correction is used to perform non-linear or inverse people have developed the so-called multi-scale Retinex
operations on the brightness or trichromatic stimulus values of algorithm (MSR). The most classic one is 3-scale, which can
the light in the image. The human eye's perception of not only compress the dynamic range of the image, but also
brightness is not proportional to the physical power, but rather maintain a good consistency of color sense. Compared with
a power function, usually with an exponent of 2.2, known as single scale, the algorithm has different steps when calculating
Gamma. For example, if the power is 50 percent gray, the the value of 𝐿𝑜𝑔(𝑅(𝑥, 𝑦)):
actual perceived brightness of the human eye is:
(1) Gaussian blur should be performed on the original
2.2
√0.5 × 100% = 72.97% (3) image at each scale to obtain the blurred image 𝐿𝑖 (𝑥, 𝑦), where
the small index i represents the number of scales.
While the human eyes feel 50% medium gray, the actual
medium gray is: (2) Each scale is obtained under the accumulative
calculation:
0.52.2 × 100% = 21.76% (4)
𝐿𝑜𝑔(𝑅(𝑥, 𝑦)) = 𝐿𝑜𝑔(𝑅(𝑥, 𝑦)) + 𝑤𝑒𝑖𝑔ℎ𝑡(𝑖) ∗ (𝐿𝑜𝑔(𝐼𝑖 (𝑥, 𝑦)) −
Taking this into account, the small storage range (0~255) 𝐿𝑜𝑔(𝐿𝑖 (𝑥, 𝑦))) (7)
and the balanced proportion of bright and dark parts, the
where weight (𝑖) represents the weight corresponding to each
grayscale value in RGB needs to be Gamma corrected instead
scale. It is required that the sum of weight of each scale must
of directly corresponding to the power value. Therefore, the
be 1. Generally, the weight of each scale is 1/3.
RGB value and the RGB color value cannot be simply added
directly, but it must be converted to the power of physical light In the previous enhancement process, the image noise made
by the power of 2.2 before the next calculation. the color distortion of the local details which cannot show the
real color of the object, the overall visual effect becomes worse
Retinex is a combination of the words "Retina" and
[7]. In view of this deficiency, Multi-Scale Retinex with Color
"Cortex". The theory of Retinex consists of two aspects: an
Restoration (MSRCR) adds color recovery factor C. Based on
object color is determined by its ability to reflect the
MSR, it adjusts the defect of color distortion caused by contrast
wavelengths of light, rather than by the absolute intensity of
enhancement in local areas of the image.
the reflected light. The color of the object is not affected by the
non-uniformity of illumination. According to Retinex theory, 𝑅𝑀𝑆𝑅𝐶𝑅𝑖 (𝑥, 𝑦) = 𝐶𝑖 (𝑥, 𝑦) ∗ 𝑅𝑀𝑆𝑅𝑖 (𝑥, 𝑦) (8)
the brightness perceived by the human eye depends on the 𝐶𝑖 (𝑥, 𝑦) = 𝑓[𝐼𝑖′ (𝑥, 𝑦)]
𝐼𝑖 (𝑥,𝑦)
= 𝑓[∑𝑁 ] (9)
illumination of the environment and the reflection of the 𝑗=1 𝐼𝑗 (𝑥,𝑦)
irradiated light from the object surface. The mathematical 𝑓[𝐼𝑖′ (𝑥, 𝑦)] = 𝛽𝑙𝑜𝑔[𝛼𝐼𝑖′ (𝑥, 𝑦)] (10)
expression is as follows: = 𝛽{𝑙𝑜𝑔[𝛼𝐼𝑖′ (𝑥, 𝑦)] − 𝑙𝑜𝑔[∑𝑁 𝑗=1 𝐼𝑗 (𝑥, 𝑦)]}
𝐼(𝑥, 𝑦) = 𝐿(𝑥, 𝑦) ∗ 𝑅(𝑥, 𝑦) (5) In these equations above, 𝐼𝑖 (𝑥, 𝑦) represents the image of
𝐼(𝑥, 𝑦) represents the image signal that is observed or received the 𝑖 𝑡ℎ channel, 𝐶𝑖 represents the color recovery factor of
by the camera; 𝐿(𝑥, 𝑦) means the exposure component of channel 𝑖 , which is used to adjust the color ratio of three
channels; 𝑓[𝐼𝑖′ (𝑥, 𝑦)] represents the mapping function of the
ambient light; 𝑅(𝑥, 𝑦) represents the reflection component of
color space; 𝛽 is the gain constant; α is the controlled nonlinear
the target object carrying the detail information of image.
strength.
By taking the logarithm of both sides of Equation, the
MSRCR algorithm uses color recovery factor C to adjust
original appearance of the object can be obtained without the
the proportion relationship between three color channels in the
property of incident light, namely, the relation is:
original image, to highlight the relatively dark area information,
𝐿𝑜𝑔(𝑅(𝑥, 𝑦)) = 𝐿𝑜𝑔(𝐼(𝑥, 𝑦)) − 𝐿𝑜𝑔(𝐿(𝑥, 𝑦)) (6) and achieve the elimination of image color distortion defects.
After processing, the local contrast of the image is improved,

6
the brightness is similar to the real scene, and the image is White Balance (WB), Histogram Equalization (HE) and
more realistic under people's visual perception. However, after Gamma Correction (GC) algorithms to an underwater image.
image processing by MSRCR algorithm, pixel value will Specifically, WB algorithm is used to correct the color casts of
generally appear negative value. Therefore, after the an underwater image, while HE and GC algorithms aim to
conversion from logarithmic domain to real domain, the image improve the contrast and light up dark regions, respectively.
needs to be corrected by changing Gain and deviation Offset.
The formula can be expressed as: An end-to-end Deep Underwater Image Enhancement
Network (DUIENET) [9] employs a gated fusion network
𝑅𝑀𝑆𝑅𝐶𝑅𝑖 (𝑥, 𝑦)′ = 𝐺 ∗ 𝑅𝑀𝑆𝑅𝐶𝑅𝑖 (𝑥, 𝑦) + 𝑂 (11) architecture to learn three confidence maps which will be used
In the (10), G represents Gain and 𝑂 represents Offset. to combine the three input images into an enhanced result. The
learned confidence maps determine the most significant
The theory of Multi-Scale Retinex with Color Preservation features of inputs remaining in the last result. Gated fusion
(MSRCP): Because HSI color space is based on human visual network architecture has not been explored in the context of
system as the starting point, using brightness (I), color underwater image enhancement. In addition, the impressive
saturation (S) and hue (H) to describe colors, it is more performance of the fusion-based underwater image
consistent with human visual characteristics than other color enhancement method also encourages us to explore the fusion-
spaces. We first processed the brightness channel I by MSR based networks.
algorithm, and then corrected the processing result
"Gain/Offset" for color restoration. Three derived inputs and original input are fed to the
proposed DUIENET to predict the confidence maps. Before
According to HSI color space, the brightness formula is: performing fusion, we add three Feature Transformation Units
1
𝐼(𝑥, 𝑦) = [𝑅(𝑥, 𝑦) + 𝐺(𝑥, 𝑦) + 𝐵(𝑥, 𝑦)] (12) (FTUs) to refine the three inputs. The purpose of the FTU is to
3
reduce the color casts and artifacts introduced by the WB, HE,
According to the formula of MSR algorithm, the brightness and GC algorithms. At last, the refined three inputs are
channel is processed by MSR algorithm: multiplied by the three learned confidence maps to achieve the
𝐼𝑀𝑆𝑅 (𝑥, 𝑦) = ∑𝐾 final enhanced result.
𝑘=1 𝑞𝑘 {𝑙𝑜𝑔𝐼(𝑥, 𝑦) − 𝑙𝑜𝑔[𝐺𝑘 (𝑥, 𝑦) ∗ 𝐼(𝑥, 𝑦)]}
(13) D. Proposed Approach Based on Methods Above
The results of MSR were modified by Gain/Offset: Based on optical propagation theories and image
′
𝐼𝑀𝑆𝑅 (𝑥, 𝑦) = 𝐾[𝐼𝑀𝑆𝑅 (𝑥, 𝑦) − 𝑏] (14) enhancement methods above, a modified is presented.
Then, the brightness of the corrected image is compared According to the Retinex theory, it can bring color
with that of the original image, and the maximum value of the constancy. While in Fusion-based CNN method, white balance
two is taken, which can effectively avoid the decrease of the is used to correct the color casts of an underwater image.
brightness of the brighter area of the image: Because the Retinex and white balance are related to color
′
𝐼𝑀𝑆𝑅 ′
= 𝑚𝑎𝑥{𝐼𝑀𝑆𝑅 (𝑥, 𝑦), 𝐼(𝑥, 𝑦)} (15) correction, methods related to Retinex including MSRCR
method and MSRCP method are attempted to replace white
Since the above methods only enhance the brightness of the balance in DUIENET.
image. To restore the original color of the image, we should
restore the color of each pixel of the three-color channels R, G The architecture of the modified model is shown in Fig. 1,
and B [8]. The dyeing process which is similar to the MSRCR which is based on the DUIENET model.
algorithm is also adopted here, and the correction result of
"gain/offset" is multiplied by a dyeing factor. After the
"gain/offset" correction, the pixel value has been transferred
from the logarithm field to the display field.
𝑆𝑖 (𝑥,𝑦)
𝐼𝑖" (𝑥, 𝑦) = 𝐼𝑀𝑆𝑅
′
(𝑥, 𝑦) (16)
𝐼(𝑥,𝑦)

𝑆𝑖 (𝑥, 𝑦) is the pixel value of the original image, and 𝐼 stands

for the R, G, and B three-color channels. Finally, the R, G, and
B pixel value of each pixel point is obtained.
C. Referenced Theory and Method
Due to the complicated underwater environment and
lighting conditions, single image processed by underwater
image enhancement method cannot describe image features Fig. 1. Modified fused-network architecture
completely.
III. EXPERIMENTAL RESULTS AND DISCUSSION
In [9], the Fusion-based method achieves decent results,
which benefits from the inputs derived by multiple pre- A. Experimental Setup
processing operations and a fusion strategy [10]. Subsequent experiments are carried out in turbid water tank,
Based on the characteristics of underwater image the experimental setup is shown in Fig. 2. The system consists
degradation, we generate three inputs by respectively applying of a Secchi disc, SJ5000 camera with waterproof (16

7
Megapixel, CMOS sensor) and Underwater LED torch. In the
experiment, Maalox suspense was dissolved in the water to
obtain cloudy water with 83 mg/L total suspended solids.
The active optical imaging is established. The distance
between camera and torch is set as 1m. The captured images (a) (b) (c) (d) (e)
are processed to validate the proposed methods. Fig. 4. Comparisons on Image 1 by different methods

(a) (b) (c) (d) (e)

Fig. 5. Comparisons on Image 2 by different methods

Fig. 2. Experimental setup

B. Method Validation
Before judging the result of the experiment, we use a
dataset called Underwater Image Enhancement Benchmark (a) (b) (c) (d) (e)
Dataset from [9]. Because the paper is limited in pages, we just Fig. 6. Comparisons on Image 3 by different methods
randomly choose 6 images to show the processed images and
results.
In terms of showing results, we use Mean Square Error
(MSE), and Peak Signal to Noise Ratio (PSNR) evaluation
methods to obtain evaluation results. A lower MSE result and a
higher PSNR result can denote the result is closer to the (a) (b) (c) (d) (e)
reference image on image content [11]. Fig. 7. Comparisons on Image 4 by different methods

(a) (b) (c) (d) (e)

Fig. 8. Comparisons on Image 5 by different methods

Fig. 3. Retinex fused with network architecture

To get an appropriate Retinex method, four steps should be

applied in the experiment. Firstly, make sure the DUIENET [9] (a) (b) (c) (d) (e)
model works. Secondly, we modify the method by replacing Fig. 9. Comparisons on Image 5 by different methods
related white balance method with MSRCR method. Thirdly,
TABLE I. THE EVALUATION OF IMAGE 1
we use MSRCP method to replace white balance. Finally, we
get the result that using MSRCP to replace white balance can Evaluation of Different methods
bring clearer images and better result. The better result proves Fig. 4 [9]
MSRCR MSRCP
Only
the MSRCP method performance, to figure out whether only method MSRCP
MSPCP method without HE and CE based on DUIENET [9], MSE 1546.99 1331.33 665.91 6141.68
can make the results better, another fusion-based method is
proposed. This method just uses MSRCP to fuse with CNN PSNR 16.24 16.89 19.90 10.25
model, it is shown in Fig. 3 and named as Only MSRCP fusion
network (Only MSRCP). At last, the result proves the result TABLE II. THE EVALUATION OF IMAGE 2
worse than the method fused with MSPCP, gamma correction,
histogram equalization and CNN. Therefore, the DUIENET Evaluation of Different methods
model modified by MSPCP achieves the best performance. Fig. 5 [9]
MSRCR MSRCP
Only
method MSRCP
From Fig. 4 to Fig. 9, (a) represents the raw image, (b) MSE 1671.98 1183.35 733.27 18225.05
represents [9] method, (c) represents MSRCR modified method,
(d) represents MSRCP modified method, (e) represents only PSNR 15.90 17.40 19.48 5.52
MSRCP fusion network.

8
 TABLE III. THE EVALUATION OF IMAGE 3 clearer visually. In terms of automotive wading, it is easier to
Different methods avoid obstacles when clearer images are obtained.
Evaluation of
[9] Only
Fig. 6 method
MSRCR MSRCP
MSRCP IV. CONCLUSION AND FUTURE WORK
MSE 1361.64 896.17 492.18 2483.10
PSNR 16.79 18.61 21.21 14.18
In the paper, our purpose is to find a method to recognize
obstacles under, so that drivers can steer clear of these
TABLE IV. THE EVALUATION OF IMAGE 4
obstacles while driving. Through the analysis of different
underwater imaging methods, and based on traditional methods
Evaluation of Different methods and deep learning theory, we modify a fusion neural network
Fig. 7
[9]
method
MSRCR MSRCP
Only
MSRCP
architecture. With the new network architecture, we deal with a
MSE 2612.36 1536.37 844.49 12801.54 series of underwater images. The experimental results show the
PSNR 13.96 16.27 18.86 7.06 visibility of the image after modifying is better than it before
modifying.
TABLE V. THE EVALUATION OF IMAGE 5
In the future, we tend to look for more test data to verify
Evaluation of Different methods the correctness of results, and try to investigate and improve
Fig. 8
[9]
MSRCR MSRCP
Only more effective underwater image enhancement models, for
example, we may employ a gated fusion network architecture
method MSRCP
MSE 1300.43 1469.28 948.09 20474.95
PSNR 16.99 16.46 18.36 5.02
to learn four confidence maps which will be used to combine
the four input images into an enhanced result. Finally, after
TABLE VI. THE EVALUATION OF IMAGE 6 underwater image enhancement, we will try to dedicate to
image recognition and install the algorithm to the vehicles.
Evaluation of Different methods
Fig. 9
[9]
MSRCR MSRCP
Only ACKNOWLEDGEMENT
method MSRCP
MSE 1533.32 1323.85 877.45 11986.85 This work was supported by the National Natural Science
PSNR 16.27 16.91 18.70 7.34 Foundation of China (Grant No. 61903202).

From Table I to Table VI, [9] method represents DUIENET REFERENCES

model method; MSRCR represents that based on fusion-CNN, [1] A. Bystrov, E. Hoare, M. Gashinova, M. Cherniakov, and T. Y. Tran.
white balance is replaced by MSRCR; MSRCP represents that “Underwater optical imaging for automotive wading”, Sensors, vol. 18.
2018. pp. 4476.
based on fusion-CNN, white balance is replaced by MSRCP;
[2] C. Ancuti, C. O. Ancuti, T. Haber, and P. Bekaert, “Enhancing
Only MSRCP represents that Only MSRCP fusion network. underwater images and videos by fusion,” in Proc. IEEE CVPR, Jun.
Red font represents the best performance. 2012, pp. 81–88.
C. Method Application [3] C. O. Ancuti and C. Ancuti, “Single image dehazing by multiscale
fusion,” IEEE Trans. Image Process., vol. 22, no. 8, Aug. 2013. pp.
After using the dataset, the results show that the new 3271–3282.
modified method, which is fused with MSRCP, gamma [4] K. Iqbal, M. Odetayo, A. James, R. A. Salam, and A. Z. H. Talib,
correction, histogram equalization and CNN, is better than “Enhancing the low quality images using unsupervised color correction
other methods. Apply the new modified method in the method,” in Proc. IEEE Int. Conf. Syst., Man Cybern. (SMC), Oct, 2010,
pp. 1703–1709.
experiment. The related images are shown below.
[5] Jaffe, J.S. “A historical perspective on underwater optical imaging”, In
Proceedings of the MTS/IEEE OCEANS 2013, Bergen, Norway, 10–14
June 2013, pp. 1–3.
[6] H. Ibrahim, N.S.P. Kong, “Brightness preserving dynamic histogram
equalization for image contrast enhancement”, IEEE Transactions on
Consumer Electronics, vol. 53, No.4, Nov, 2007, pp. 1752-1758.
[7] Z. Shu, W. Ting, D. Junyu, Y. Hui. “Underwater image enhancement via
(a) (b) (c) (d) (e) extened multi-scale Retinex”, Neurocomputing, vol. 245, Jul. 2017. pp.
Fig. 10. The experimental image 1-9
[8] S. Wenbo, L. Lei, J. Jiawei, Y. Yifan. “Low-light-level image
TABLE VII. THE EVALUATION OF EXPERIMENTAL IMAGE enhancemnt based on fusion and Retinex”, Journal of Modern Optics,
Different methods vol. 67, Jul, 2020. pp. 1190-1196
Evaluation of
[9] Only [9] C. Li, C. Cuo, W. Ren, R. Cong, J. Hou, S. Kwong, and D. Tao. “An
experiment MSRCR MSRCP
method MSRCP underwater image enhancement benchmark dataset and beyond”. IEEE
MSE 3489.43 5628.38 777.91 1522.20 Trans. Image Process, 2019, pp. 4376-4389.
PSNR 12.70 10.63 19.22 16.30 [10] C. Ancuti, C. O. Ancuti, and P. Bekaert, “Enhancing underwater images
and videos by fusion,” in Proc. of IEEE Int. Conf. Comput. Vis. Pattern
Rec. (CVPR), 2012, pp. 81-88. 2, 4, 5, 6, 7, 8, 10, 11
The result shows that in the experiment, the MSE score is
[11] Q. Hyunh-Thu, M. Ghanbari, “Scope of validity of PSNR in
the minimum and the PSNR is the maximum, which means image/video quality assessment”, Electronics letters, vol. 44, 2008,
that the result is closer to the reference image in terms of image pp.800-801
content. At the same time, the image with MSRCP method is

9
 2021 the 13th International Conference on Computer and Automation Engineering

Local Navigation and Docking of an Autonomous

Robot Mower Using Reinforcement Learning and
Computer Vision
Ali Taghibakhshi Nathan Ogden Matthew West
Department of Mechanical Science John Deere Department of Mechanical Science
and Engineering Technology Innovation Center and Engineering
The University of Illinois Champaign, Illinois, USA The University of Illinois
at Urbana-Champaign [email protected] at Urbana-Champaign
Urbana, Illinois, USA Urbana, Illinois, USA
[email protected] [email protected]

Abstract—We demonstrate a successful navigation and docking and may induce further complications in the design of an
control system for the John Deere Tango autonomous mower, autonomous vehicle. It is thus desirable to produce systems
using only a single camera as the input. This vision-only system which use only vision as the control input.
is of interest because it is inexpensive, simple for production,
and requires no external sensing. This is in contrast to existing Over the past decade, reinforcement learning techniques
systems that rely on integrated position sensors and global and algorithms have been able to solve complicated decision
positioning system (GPS) technologies. To produce our system making problems, both in single-agent [4], [5] and multi-
we combined a state-of-the-art object detection architecture, You
Look Only Once (YOLO), with a reinforcement learning (RL) agent frameworks [6]–[8]. With the rise of deep reinforcement
architecture, Double Deep Q-Networks (Double DQN). The object learning, there have been multiple studies on implementing
detection network identifies features on the mower and passes its value-based learning methods, mostly deep Q-learning [5],
output to the RL network, providing it with a low-dimensional in the field of autonomous driving. Using a visual encoding,
representation that enables rapid and robust training. Finally, authors in [9] utilized a low dimensional representation of the
the RL network learns how to navigate the machine to the
desired spot in a custom simulation environment. When tested driving environment as the observation input for model-free
on mower hardware, the system is able to dock with centimeter- RL agents, whose aim is to achieve urban autonomous driving.
level accuracy from arbitrary initial locations and orientations. [10] has proposed a deep RL framework for autonomous
driving, where the RL agents observe raw sensor outputs and
Index Terms—Reinforcement Learning, Deep Q-Learning, Ob- take driving actions.
ject Detection, YOLO, Mower
This paper focus on the docking procedure for the John
I. I NTRODUCTION Deere Tango robot mower, which is not reliable in it’s current
The ever-growing field of autonomous vehicles is a research form. This mower was designed to dock using a guide loop
hotbed for applying Artificial Intelligence (AI) and Machine wire buried underground, which leads the mower toward the
Learning (ML). In the past few years, there have been a charging station by inducing an electric current in the wire,
wide variety of improvement in autonomous vehicles. These which is then sensed by a detection system in the mower.
improvements include, but are not limited to, single agent scale Docking failures can occur when the wire is misplaced under
tasks such as path planning, lane changing, and self driving, the ground or there is a bump in the mower’s path. This
and multi-agent scale tasks such as collision avoidance and motivated us to investigate a system that it is more robust
lane management. For instance, in [1], authors have investi- to variable initial position and environmental conditions.
gated an optimal lane change decision model for autonomous In this paper, we have used a combination of a supervised
vehicles in high-capacity urban roads. In recent studies [2], and reinforcement learning algorithms to locally navigate the
[3], researchers have introduced a novel lane management in robot mower and orient it toward the docking station. Firstly,
heterogeneous framework with autonomous vehicles. using transfer learning, we train a YOLO network to detect
Recently, many companies have invested efforts in develop- two pre-existing markers on the robot mower to provide
ing ML-aided driving for increased comfort and safety of their positioning and orientation information. Secondly, using the
vehicles. However, many autonomous driving and navigation output of the object detection network, we train a DQN agent
systems still rely on sensing modalities such as Laser Range to learn how to move the robot mower to the desired position
Finder (LRF), Light Detection and Ranging (LIDAR), and employing a curriculum training technique. The real-world
GPS, to name but a handful. These systems can be expensive scenario is shown in Fig. 1.

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 10

 P∞
t=0 γ t rt , where 0 < γ ≤ 1 is the discount factor. The state-
action value function is denoted by Q(s, a), and maps a pair
Marker 2 Marker 1 of a state and an action to a real number; Q : S×A → R. The
Q-learning update [16] is
Q(st , at ) ← Q(st , at )
 
Camera + α rt − Q(st , at ) + γ max
0
Q(st+1 , a0 )
a

where α is the learning rate. Mnih et al. [5] used the Q-

learning algorithm in conjunction with deep neural networks
and a replay buffer to produce the DQN (Deep Q-Network)
algorithm. This uses two networks, the primary and target.
Charging Station
The primary network is the one that is being updated using
Mower Stochastic Gradient Descent (SGD) at every iteration. The
target network is the latest copy of the primary network, and
Fig. 1: Real-world docking scenario. The mower is shown it is used for evaluating the action values. The target network
approaching the charging station under the direction of the is updated once in every N ∈ N iterations to evaluate the
RL network, which receives inputs from the YOLO network action values using a recent version of the primary network.
which localizes the markers in the camera feed. Marker 1 is However, the max operator in DQN algorithm is prone to
the stop button and marker 2 is the John Deere logo. overestimating the state-action value function since it selects
the maximum value of the same Q network. To mitigate this
function approximation overestimation, one needs to decouple
II. BACKGROUND the selection and evaluation tasks. This leads to the Double
DQN algorithm [17], which is used in this research:
A. YOLO
Real time object detection has been a significant achieve- Algorithm 1 Double DQN
ment of deep convolutions neural networks (CNN) in the field initialize the primary network Qθ , the target network Qθ0 , the
of computer vision. Using residual connections, deep CNNs replay buffer D, and τ  1
can extract complex features from the observed image and are for each iteration do
highly accurate in localizing different objects [11]. Generally, for each environment step do
these networks are trained on multiple object training sets. observe state st and select at ∼ π(st , at )
Given an image, they draw a bounding box, labeled with the execute at and observe the next state st+1
object’s tag, around it. The R-CNN algorithm [12] is arguably and reward rt = R(st , at )
the pioneer object localization algorithm and, ever since its store (st , at , rt , st+1 ) in the replay buffer D
introduction, many other algorithms have been proposed for end for
the purpose of object detection, including modified versions of for each update step do
R-CNN such as fast R-CNN and faster R-CNN [13], [14]. One sample et = (st , at , rt , st+1 ) ∼ D
of the fastest and most accurate object detection algorithms compute target Q value:
is You Only Look Once (YOLO), which achieves object Q∗ (st , at ) ≈ rt + γQθ0 (st+1 , argmax Qθ (st+1 , a0 ))
a0
detection using a fixed-grid regression [15]. perform gradient descent step on
(Q∗ (st , at ) − Qθ (st , at ))2
B. Deep Q-Learning
update the target network parameters: θ0 ← τ θ+(1−τ )θ0
Value learning is a way of approaching reinforcement learn- end for
ing problems. Deep Q-Learning (DQN) is a value-learning end for
algorithm that has demonstrated success on a wide range
of tasks, including achieving human-level control in Atari
III. S IMULATION AND T RAINING
games [5]. The algorithm combines the traditional Q-learning
update with neural network function approximation. Similarly A. Motivation for Object Detection with RL
to many RL algorithms, the problem is modeled as a discrete- The charging station of the mower is viewed by a Logitech
time Markov Decision Process (MDP). At any time, the agent c270 webcam, and the aim is to navigate the mower toward
is in a certain state of the environment’s state space, S = the docking station and either stop it at a desired position or
{s1 , s1 , ..., sn } and has some corresponding actions available to help it dock, using only the vision input. Therefore, the
from environment’s action space, A = {a1 , a1 , ..., am }, which environment of the problem we are trying to solve, the video
influence the transition to the next state. Transitioning from feed of the camera, is not only very high dimensional, but it
one state to another provides the agent with a reward r, and the is also dependent on where the setup is located, and it varies
goal of agent is to maximize the sum of its discounted rewards, from yard to yard. Hence, we lower the dimensionality of the

11
environment and extract key features of the video feed to both phases are distinguished by different reward functions rti
improve system robustness and accelerate RL training. defined by
We use the YOLO algorithm to locate bounding boxes 
around two markers on the mower, one in the front and the 

 Ri if Y ≥ 1 m and |u1 |, |u2 | < ci1 ,
0 if Y ≥ 1 m and |u1 |, |u2 | < 2ci1 ,

other at the back of the top surface of the mower. The output 

i
of the YOLO network is then passed to the RL network as rt = −(10|v 1 (t) − v01 |+ otherwise,
input. Accordingly, the RL agent’s observation space of the 


 10|v 2 (t) − v02 |+
world is low dimensional and it is easier to train in this space. 
ci |u1 | + ci |u2 |)

2 2
The fact that the two markers are located at different ends
of the mower allows the RL agent to sense the angle that the where (ci1 , ci2 ) are constants with values (0.05, 2), (0.05, 5),
mower makes with the straight line toward the docking station. (0.02, 5), (0.02, 10) for i = {1, 2, 3, 4}, respectively; Ri = 0
This is due to the fact that both bounding boxes will be in the for i = 1, and Ri = 150 for i = {2, 3, 4}; u1 , v 1 , u2 , v 2 ∈
center of the image if the the mower is exactly oriented toward (−1, 1) are the normalized coordinates of the center of the two
the docking station. Therefore, in the setup designed in this markers; v01 , v02 are the y components of the markers when the
paper, the centers of the bounding boxes in the picture are the mower is at Y = 1 m, X = 0 m, θ = 0◦ . The training data
information that is passed to the RL network, which outputs is shown in Fig. 4.
linear velocity and steering rate as actions.
C. Object Detection Network
B. RL Simulation Environment
There are two YOLO object detection networks used in this
We have designed a simple simulation environment for study, one for detecting the markers in the simulation and
training the RL agent. The kinematics of the agent in the another for detecting the real-world markers from the actual
simulation environment are ODEs driven by the linear ve- camera images. For each of the networks, a pre-trained YOLO
locity and angular steering rate. The simulation environment network was further trained to detect the markers. The training
simulates the motion and computes the view of the markers data for the networks is shown in Fig. 5. Each of the networks
from the point of view of the camera. The markers are drawn was trained on about 3000 labelled images of the markers.
as quadrilaterals in the simulation, one in red and the other The pre-trained feature extraction used ResNet50 [11], and
in black, and the rest of the simulated image is just a white the architecture is re-trained after the 20th ReLU function.
background, representing the remainder of the environment.
The mower initially starts at a uniformly distributed random IV. H ARDWARE E XPERIMENTS
position, (X, Y ) ∼ U (−0.2, 0.2) × U (−0.2, 0.2) in meters,
A. Experiment Setup
and orientation, θ ∼ U (−30, 30) in degrees. Throughout the
paper, X and Y coordinates correspond to the position of the A Mosquitto MQTT broker was used to send the agent’s
rear axle of the mower. The environment is shown in Fig. 2. actions to the mower via an Ethernet cable, and the connection
We stop training the agent when the average reward reaches between the computer and the mower was controlled by a
a certain amount, which was set experimentally. Moreover, in Raspberry Pi. As in the simulation, the mower started from
training, we manually reset the mower when its Y component an initial position (X, Y ) and orientation θ. The actions were
exceeds 1 m. The mower stops when the Y component of its
position exceeds 1 m and the goal is to minimize the X and
θ offsets from zero when it stops. This target goal was chosen Above View of the
because it enables the two metallic rods on the front of the Y Simulation Environment
physical mower to connect to the metal pads in the charging
station. Camera Camera View of the
The DQN agent takes the center positions of the bounding 𝜃 𝜈 Two Markers
boxes at the last three time-steps as its input. The reason for Front Wheel Marker 1
using the data at the last three time-steps is to provide the agent
Marker 2
with some information about the past to allow the estimation
Desired Final Marker 2
of velocity and acceleration. The agent outputs the desired
Mower Position
linear velocity and angular rate based on its observation. The 𝑢
Marker 1 Real Wheel
DQN network is shown in Fig. 3, where the state component
has two fully connected layers with 16 and 32 neurons and
the action component has a single layer with 32 neurons. X
For training the agent, we used curriculum training with
four phases, numbered i = 1, . . . , 4. In the first phase the
agent is rewarded to go forward, with a small reward for Fig. 2: Simulation environment. Left: the view from above
arriving near the target docking location. The subsequent of the mower approaching the charging station. Right: the
phases increasingly reward more accurate docking. The four simulated camera view of the two markers.

12
 Tanh
ReLU
Markers
Position Real-World Environment
YOLO ReLU
12 nodes 𝑢
Markers
16 nodes Y OLO Marker 1 Positions 𝜈
32 nodes 𝑢
Marker 2
ReLU
+ 𝜈
1 node

(a)
A ctions; Pairs of 32 nodes
Linear and Angular
Velocities Simulation Environment
15 nodes Y
𝑢
Markers
32 nodes 𝜈
Marker 1 Positions 𝜈
Y OLO
Fig. 3: The RL network architecture. The observation head 𝑢
𝑢
Marker 2
receives labeled markers in the last three time-steps of the X
𝜈

environment and feeds it through the network. The action head

also passes each action through a single layer. The last layers
of each head are added together and passed through the final (b)
layer to output the Q values. Fig. 5: The two YOLO networks. a) YOLO network trained on
real-wold mower data. b) YOLO network trained on simulation
images. In both cases the image is sent to the corresponding
YOLO network and the outputs are the bounding boxes.
Finally, the normalized positions of the center of the bounding
boxes are extracted as a 4 dimensional vector.

TABLE I: Experiment results.

Error Measure X Offset (cm) Y Offset (cm) θ Offset (deg)

Max Abs. Error 3.800 3.642 6.200
Mean Abs. Error 0.822 0.934 1.533
RMSE 0.896 1.182 1.661
(a) (b)

based the X, Y , and θ offsets of the mower when it stopped

(when its observed Y component has exceeded 1 m). The
experiment data is shown in Fig. 6 and summary results are
given in Tab. I, giving maximum absolute error, mean absolute
error (MAE), and root mean squared error (RMSE). The
mower had a maximum final position error of less than 4 cm
in both X and Y directions and a maximum final orientation
(c) (d) error of less than 7◦ , representing successful positioning of the
mower in all cases. The mean absolute error was less than 1 cm
Fig. 4: Training graphs of the agent during the four curriculum
in both X and Y directions and less than 2◦ in orientation.
phases, with panels a–d corresponding to reward functions r1
through r4 .
V. C ONCLUSION
We demonstrated a cheap and effective control system for
sent to the mower at a frequency of 5 Hz. The only input to autonomous docking of a robotic lawn mower (the John Deere
the controlling RL agent was from the camera. Tango mower), using only vision from a single camera as the
sensor. This system was shown to be robust in hardware tests,
B. Experiment Results achieving centimeter-level docking precision.
A total of 90 tests were performed with the initial position The controller was a neural network trained using rein-
and orientation of the mower (X, Y, θ) taken as all combi- forcement learning (Double DQN) in a simple simulated
nations of X = −0.2, 0, 0.2 m, Y = −0.2, 0, 0.2 m, and environment. To avoid the need to simulate realistic vision
θ = −30, −15, 0, 15, 30◦ , with each initial condition being inputs, we trained an object detection network (YOLO) to
tested twice. The performance of the agent was measured isolate two markers on the mower. The location of these

13
 Y (m) [4] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van
1
.7
Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam,
1
.6 M. Lanctot et al., “”mastering the game of go with deep neural networks
Desired Final Location and tree search”,” nature, vol. 529, no. 7587, pp. 484–489, 2016.
1
.5
of the Mower [5] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G.
1 Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski
.4
et al., “”human-level control through deep reinforcement learning”,”
1
.3 nature, vol. 518, no. 7540, pp. 529–533, 2015.
1
[6] O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik,
.2
J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev et al., “”grand-
1
.1
master level in starcraft ii using multi-agent reinforcement learning”,”
Final Position of the Rear Axel Nature, vol. 575, no. 7782, pp. 350–354, 2019.
1
.0 [7] A. Shojaeighadikolaei, A. Ghasemi, K. R. Jones, A. G. Bardas,
M. Hashemi, and R. Ahmadi, “”demand responsive dynamic pricing
0
.9
Y framework for prosumer dominated microgrids using multiagent rein-
-0 00 0
.2 . .2 forcement learning”,” arXiv preprint arXiv:2009.10890, 2020.

𝜃 (deg); Final Orientation of the

[8] A. Ghasemi, A. Shojaeighadikolaei, K. Jones, M. Hashemi, A. G.
Desired Final
Position of the
Bardas, and R. Ahmadi, “”a multi-agent deep reinforcement learning
Rear Axel approach for a distributed energy marketplace in smart grids”,” pp. 1–6,
X
2020.
[9] J. Chen, B. Yuan, and M. Tomizuka, “”model-free deep reinforcement
learning for urban autonomous driving”,” in 2019 IEEE Intelligent
0°
Transportation Systems Conference (ITSC). IEEE, 2019, pp. 2765–
30° −30°
2771.
0.2 [10] A. E. Sallab, M. Abdou, E. Perot, and S. Yogamani, “”deep reinforce-
ment learning framework for autonomous driving”,” Electronic Imaging,
vol. 2017, no. 19, pp. 70–76, 2017.
Initial Positions [11] K. He, X. Zhang, S. Ren, and J. Sun, “”deep residual learning for image
-0.2 0.0 0.2 X (m) and Orientations
of the Mower
recognition”,” in Proceedings of the IEEE conference on computer vision
and pattern recognition, 2016, pp. 770–778.
-0.2 [12] R. Girshick, J. Donahue, T. Darrell, and J. Malik, “”rich feature
hierarchies for accurate object detection and semantic segmentation”,”
Fig. 6: Experiment configurations. The mower started from in Proceedings of the IEEE conference on computer vision and pattern
one of the positions and orientations shown in green, and recognition, 2014, pp. 580–587.
from each initial configuration two tests were performed. The [13] R. Girshick, “”fast r-cnn”,” in Proceedings of the IEEE international
conference on computer vision, 2015, pp. 1440–1448.
desired final position is marked by a red cross. The final [14] S. Ren, K. He, R. Girshick, and J. Sun, “”faster r-cnn: Towards real-time
positions of the mower are magnified and the final X, Y , object detection with region proposal networks”,” in Advances in neural
and θ values are shown. information processing systems, 2015, pp. 91–99.
[15] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “”you only look
once: Unified, real-time object detection”,” in Proceedings of the IEEE
conference on computer vision and pattern recognition, 2016, pp. 779–
markers was the only input to the controller agent, making 788.
[16] R. S. Sutton and A. G. Barto, ”Reinforcement learning: An introduc-
it easy to produce similar inputs in simulation. tion”. MIT press, 2018.
In addition to the ease of training, the use of an initial object [17] H. Van Hasselt, A. Guez, and D. Silver, “”deep reinforcement learning
detection network made the final system robust to different with double q-learning”,” arXiv preprint arXiv:1509.06461, 2015.
backgrounds and other environment variations. Our choice of
markers was opportunistic (we used existing features on the
mower) and it is likely that custom markers may be even better.
We believe that this paradigm of an object detection network
coupled with an RL agent could be an effective strategy for
other robot motion control tasks.
ACKNOWLEDGMENT
This research was supported by the John Deere Technology
Innovation Center.
R EFERENCES
[1] P. Cao, Y. Hu, T. Miwa, Y. Wakita, T. Morikawa, and X. Liu, “”an opti-
mal mandatory lane change decision model for autonomous vehicles in
urban arterials”,” Journal of Intelligent Transportation Systems, vol. 21,
no. 4, pp. 271–284, 2017.
[2] P. Karimi Shahri, S. Chintamani Shindgikar, B. HomChaudhuri, and
A. H. Ghasemi, “”optimal lane management in heterogeneous traffic
network”,” in Dynamic Systems and Control Conference, vol. 59162.
American Society of Mechanical Engineers, 2019, p. V003T18A003.
[3] P. K. Shahri, A. H. Ghasemi, and V. Izadi, “”optimal lane management
in heterogeneous traffic network using extremum seeking approach”,”
SAE Technical Paper, Tech. Rep., 2020.

14
 2021 the 13th International Conference on Computer and Automation Engineering

Emotion Recognition Using Short-Term Analysis of

Heart Rate Variability and ResNet Architecture
Lowell Nathaniel B. Singson Maria Trinidad Ursula R. Sanchez Jocelyn Flores Villaverde
School of EECE School of EECE School of EECE
Mapua University Mapua University Mapua University
Manila, Philippines Manila, Philippines Manila, Philippines
[email protected] [email protected] [email protected]

Abstract—Human emotions serve to have an important role in sadness, peace, fear, and angry, through the use of ECG heart
human behaviors. This study will identify and recognize rate variability and convolutional neural network.
emotions with the use of facial expressions for the convolutional
neural network (CNN) using ResNet architecture and with the II. METHODOLOGY
use of physiological signals, particularly, heart rate variability
and electrocardiogram (ECG). The testing subjects' images were A. Conceptual Framework
taken, serving as an input to the CNN while the ECG device was The conceptual framework shown in Fig. 1, shows how the
attached to them. The emotions of the testing subjects would be process works to recognize human emotions. The inputs are the
the output on the resulting ECG signals and face emotion face recognition where face images will serve as the input for
recognition. These emotions could be happy, sad, neutral, fear, the convolutional neural network and the ECG signals which
and anger. The data gathered from the testing subjects with the serve as the output gathered from the ECG device based on the
use of the researcher’s device worked as intended. The heart rate variability of the test subject [9]. The Convolutional
accuracy's obtained from the ECG based on the results was Neural Network (CNN) will utilize the edge detection
68.42%. algorithm into the image, to recognize any pattern that was
either present or absent [10]. To obtain its maximum accuracy,
Keywords—Facial Emotion, Electrocardiography,
Convolutional Neural Network, ResNet Architecture, Statistical
various parameters, both real-valued and binary are extracted
Analysis and analyzed for the image [11]. CVRR and Frequency domain
analysis is used to determine the test subject’s emotion via
I. INTRODUCTION heart rate variability (HRV) using a wearable ECG device [12].
Short-term analysis of HRV is used to detect an emotion at a
Emotions affect how a person functions daily as it serves to
lesser time but not less than 10 seconds [13]. This process will
have an important role in understanding human behaviors [1].
output the emotions recognized from the test subjects.
With emotions, humans tend to express it differently and/or
simultaneously through hand and body gestures, speeches, and
facial expressions [2]. Facial expressions are often the basis of
what emotions do someone expresses based on what they show
with their faces. These facial expressions could be a smiling
face, a frowning face, and any other facial emotions. Besides,
facial expressions are a non-verbal way of interacting and
communicating with people [3]. Through the advancement of
modern-day technologies such as in machine learning, it is
possible to detect the facial emotions of a person from the
taken images [4]. Emotions can also be recognized based on
physiological signals [5]. Particularly based on
electrocardiogram (ECG) and heart rate. With that, it can be Fig. 1. Conceptual Framework
used to predict human emotions [6].
The main objective of this paper was to developed a device B. HRV Statistical Analyses
that recognized an emotion through the use of an With certain statistical analyzes skewness and kurtosis are
electrocardiogram and neural network. Specifically, it monitors used in order to better define the position and uncertainty of the
variability in heart rate to predict emotion. and implement data collection. These two are used to evaluate the distribution
ResNet architecture in a neural network, using images to properties of the heart rate variability (HRV) and are used for a
determine facial emotions. The proposed system test the calculation to test the normality of the given data set.
accuracy of the convolutional neural network using ResNet Skewness is the degree of distortion from the normal
architecture [7]. The study’s testing subjects are young adults, distribution [14]. Regardless of this the skewness distribution
ranging from ages 12 to 30 [8]. Furthermore, this paper only of normality is zero. The asymmetry of the skewness
focused on detecting the five human emotions – happiness, distribution of probability in the data set is calculated using:

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 15

 ∑𝑇 Table I-1 shows the percentages of the CRVV, classifying
𝑖=1(𝑥𝑖 −𝑥)
𝑆𝑘 = (1) it with the five emotions used in this study for emotion
𝜎3 recognition. On the other hand, Tables I-2 to I-3 shows the
From equation (1) shown, 𝑥𝑖 represents the HRV data set, 𝑥 values of LHratio emotion recognition as to which emotions
is the mean value, and 𝜎 represents the standard deviation. are classified.
There are two types of skewness: positive and negative
skewness. For the positive skewness, the data is mostly located D. Data Architecture
on the left side of the mean where the average is greater than The ResNet-18 architecture shown in Fig. 2, contains 18
the average. For the negative skew the data is mostly on the layers. From the first convolution layer, it is with a filter size of
right side of the mean where the mean is lower than the 7x7 and stride 2 after which is where the skip connection
average [15]. begins. A skip connection is an identity map wherein the
previous layer’s input will be added to the output of the next
Kurtosis, on the other hand, is all about the tails of the layer. With this, the input from the first convolutional layer
distribution. It measures the heavy tails or outliers of the will be added to the output of the second convolutional layer
distribution. The equation of kurtosis is: which is the 3x3 max pool layer, stride 2, and a two
𝑁 convolution layers with a filter size of 3x3 and 64 number of
1 𝑥𝑖 − 𝜇 4
𝐾 = 𝑁∑ ( ) (2) filters which makes it the first residual block – the input of
𝑖=1 𝜎 layer added to the output from the other layer [16]. After that,
where 𝑥𝑖 represents the HRV data set, 𝜇 represents the mean the output of the first residual block will be added to the output
value, and 𝜎 represents the standard deviation. Positive of the two convolution layers with a filter size of 3x3 and 128
kurtosis in the set of data indicates a heavy-tailed or outlier number of filters which makes it the second residual block.
distribution while negative kurtosis in the set of data indicates a Then, the output of the second residual block through skip
light-tailed or lack of outer distribution. There are three forms connection will be added to the output of the two convolution
of kurtosis occur: mesokurtic, leptokurtic, and platykurtic. The layers with a filter size of 3x3 and 256 number of filters which
mesokurtic has a kurtosis closely like the normal distribution in makes it the third residual block. And for the fourth residual
which the kurtosis of a normal distribution is three. Then its block, the output of the third residual block via skip connection
distribution for the leptokurtic is of heavy-tailed and its is added to the output of the two convolution layers with a 3x3
kurtosis is greater than three. And for the platykurtic its filter size and 512 filter number. Finally, the fourth residual
distribution is of light-tailed and less than three of its kurtosis. block’s output will go through the average pooling then it will
receive a feature map that will be given to the layers of the
C. Data Gathering Procedure fully connected by the softmax function for the final output to
The images will be taken with the use of a smartphone be received.
camera which will be used in the system as an input for the
convolutional neural network emotion recognition while the
ECG device is attached to the test subject. With that, the ECG
device will measure the test subject's heart rate throughout a
short duration of time, not less than 10 seconds. After its
CRVV and LHratio are processed, the emotion recognized by
the system will be transmitted to the device.

TABLE I-1. CRVV AND LHRATIO EMOTION RECOGNITION

Emotion Anger/Fear Peace Happy/Sad
Recognition
Analysis
CRVV Greater Than Between Less Than 5.5%
6.5% 6.5%-5.5%

TABLE I-2. CRVV AND LHRATIO EMOTION RECOGNITION

Emotion Recognition Anger Fear Peace
Analysis

LH ratio Greater Than Between 1-0.65 Less than

1 0.65 Fig. 2. ResNet-18 Architecture

E. Residual Network Convolutional Neural Network (ResNet

TABLE I-3. CRVV AND LHRATIO EMOTION RECOGNITION CNN)
Emotion Recognition Happy Sad Peace The data is gathered from the images taken classifying the
Analysis five human emotions used in this paper – happiness, sadness,
LH ratio Greater Than Between 1-0.65 Less than fear, anger, and neutral which will be the input for ResNet
1 0.65

16
CNN. These data will be split into training and testing datasets. Fig. 4. The program uses the heartpy module for the signal’s
With the use of a training dataset, the ResNet CNN shall be analysis and pandas, numpy, scipy, for data processing. The
trained until it reached its acceptable accuracy. Then, through upper figure shows the entire length of the filtered ECG signal
the test dataset, it will validate the ResNet CNN and after while the lower figure shows the zoomed-in signals with the R-
which the overall accuracy of the ResNet CNN is determined. peaks labeled by the red dot. The parameters of the processed
signals are showed on the right control panel, stated with their
F. Hardware Development respective values.
The materials used for the hardware development of the
system are as follow:
a. Arduino Uno: A programmable microcontroller board that
has a microchip which is ATmega328p. The board is
equipped with sets of digital and analog input/output pins
that may be interfaced with various expansion boards and
other circuits.
b. ECG Shield and Cable: its stackable headers up to 6
channels may be stacked and wired to A0-A6 analog
inputs. Calibration signal generation by D4/D9 digital Fig. 4. ECG Signal Analysis
output. Precise the Trimmer potentiometer for calibration.
TABLE II. DATA CLASSIFICATION
Input connector for normal or active electrodes. Works
with both 3.3V and 5V for boards. Trial CRVV LF HF LH ratio AE MPE
1 8.205 0.43 4.6 0.89 Neutral Neutral
c. ESP8266 (Wi-Fi Module): It's easy to use and popular
2 9.505 0.57 13.21 0.17 Happy Neutral
module for developing projects and applications related to 3 6.21 0.43 4.6 0.89 Neutral Neutral
the Internet of Things-IoT and WiFi. A low cost module 4 9.78 0.41 3.93 0.85 Neutral Neutral
significantly simplifies the design of IoT devices. There 5 14.27 0.4 3.93 0.85 Neutral Unknown
6 3.04 0.9 8.54 2.37 Neutral Sad
are many variants to the ESP8266 module. The most 7 6.21 0.41 4.6 0.85 Happy Neutral
popular and easiest version of this ESP-01 variant for the 8 2.9 0.91 5.34 1.52 Sad Sad
interface module. 9 8.09 0.32 8.7 2.43 Happy Neutral
10 3.4 1.13 4.46 0.87 Happy Happy
G. Schematic Diagram 11 2.96 0.94 8.93 2.47 Neutral Sad
12 36.97 0.22 8.19 1.4 Unknown Unknown
The schematic diagram of the system is shown in Fig. 3.
13 11.47 0.25 7 2.31 Unknown Unknown
Arduino Uno serves as the main processing unit of the system. 14 2.9 0.91 5.34 1.52 Sad Sad
The ad8232 heart rate sensor is the module used to gather ECG 15 3.5264 2.783 13.03 0.359 Unknown Neutral
signals through a 3-pin sensor cable. On the other hand, a 7805 16 11.47 0.72 0.92 0.253 Unknown Unknown
voltage regulator is used as a power source of the module. The 17 32.20 2.5 8.34 0.55 Unknown Unknown
output pin is connected to the Arduino’s Analog 0 pin while 18 32.20 2.5 -1.08 1.166 Unknown Unknown
L0- and L0+ is at D11 and D10, respectively. NodeMCU is 19 2.86 2.474 8.928 0.942 Happy Happy
used as a serial communication system on Arduino, where a *Legends: LF – Low Frequency, HF – High Frequency,
smartphone is connected to transfer images into it. AE – Actual Emotion, MPE – Machine Predicted Emotion

Confusion Matrix – The confusion matrix for the ECG is

shown in Table II. The accuracy of the data seen on the table is
calculated from the sum of the main diagonal cells (top left cell
to bottom right cell) over the sum of all cells. The main
diagonal cells have an overall sum of 13. While the overall
sum of all the cells is 19. Resulting in an accuracy of 68.42%.

TABLE III. PREDICTED EMOTIONS COMPARED TO ACTUAL EMOTIONS

Predicted Emotion
H S N U
H 2 0 2 0
S 0 2 0 0
Fig. 3. Schematic Diagram Actual N 0 2 4 1
Emotions U 0 0 1 5
III. RESULTS AND DISCUSSION *Legends: H – Happy, S – Sad, N – Neutral, U – Unknown
Data Classification – The output of the python program
that analyses the ECG signals by its RR-Interval and R-peaks Sum of Main Diagonal
Accuracy = ∗ 100
to find the CVRR, Lhratio, Kurtosis, and Skewness is shown in Sum of all Cells

17
 IV. CONCLUSION AND RECOMMENDATION [7] Pawel Tarnowski, Marcin Kolodziej, Andrej Majkowski, Remigiuskz J.
Rak (2017) Emotion recognition using facial expressions, Volume 108,
This paper classified the five chosen human emotions that 2017, Pages 1175-1184, https://doi.org/10.1016/j.procs.2017.05.025.
are happy, sad, and neutral through the use of heart rate [8] S. V. Martínez-Ortega and M. Sarria-Paja, "Alternative Measurement
variability and convolutional neural network using ResNet Instrument to Evaluate Executive Functions in Children of All Scholar
architecture. Images were taken from the testing subjects which Range," 2019 XXII Symposium on Image, Signal Processing and
Artificial Vision (STSIVA), Bucaramanga, Colombia, 2019, pp. 1-3, doi:
serves as the input of the CNN while the ECG device was 10.1109/STSIVA.2019.8730230.
attached to them. The results from Table II show that when the
[9] Nayak G.S., Puttamadappa C., Kamatha A.S. (2008) Statistical Analysis
LHratio greatly increased then the emotion was either happy or of ECG signals for Arrhythmia Detection. In: Abu Osman N.A., Ibrahim
angry while the value of CVRR increased slightly during states F., Wan Abas W.A.B., Abdul Rahman H.S., Ting HN. (eds) 4th Kuala
of rage and fear, and decreased in happy and sad states, making Lumpur International Conference on Biomedical Engineering 2008.
Table III true. Acquiring an accuracy of the ECG to 68.42%. IFMBE
Proceedings,vol21.Springer,Berlin,Heidelberg,https://doi.org/10.1007/9
The researchers recommend the creation of the system to 78-3-540-69139-6_65
add assessments to be used by psychologists to understand [10] A. M. Cadayona, N. M. S. Cerilla, D. M. M. Jurilla, A. K. D. Balan and
daily human behaviors through human emotions. As well as J. C. d. Goma, "Emotional State Classification: An Additional Step in
Emotion Classification through Face Detection," 2019 IEEE 6th
adding additional emotions, like disgust, anticipation, joy, etc. International Conference on Industrial Engineering and Applications
Besides the ResNet architecture used, the researchers (ICIEA), Tokyo, Japan, 2019, pp. 667-671, doi:
recommend using other convolutional neural network 10.1109/IEA.2019.8715171
architectures to see which outputs the highest accuracy of the [11] D. Hendrawan, C. Carolina, F. Fauzani, H. N. Fatimah, F. Kurniawati,
system. The researchers also suggest implementing Artificial and M. A. M. Ariefa, "The construction of android computer-based
Intelligence (AI) in ECG classification. application on neurocognitive executive function for early age children
inhibitory control measurement," 2016 7th IEEE International
REFERENCES Conference on Cognitive Infocommunications (CogInfoCom), Wroclaw,
2016, pp. 000409-000414, doi: 10.1109/CogInfoCom.2016.7804584.
[1] Sanchez, Maria Trinidad Ursula R. and Singson, Lowell Nathaniel B. [12] Wang, Chendi & Wang, Feng. (2012). An emotional analysis method
“Student’s Executive Function Assessment Tool using Convolutional based on heart rate variability. 10.1109/BHI.2012.6211518
Neural Network”(2020)
[13] Y. Zhou, F. Ren, S. Nishide and X. Kang, "Facial Sentiment
[2] A. A. Varghese, J. P. Cherian, and J. J. Kizhakkethottam, "Overview on Classification Based on Resnet-18 Model," 2019 International
emotion recognition system," 2015 International Conference on Soft- Conference on Electronic Engineering and Informatics (EEI), Nanjing,
Computing and Networks Security (ICSNS), Coimbatore, 2015, pp. 1-5. China, 2019, pp. 463-466, doi: 10.1109/EEI48997.2019.00106.
[3] Abisado, Mideth & Linsangan, Noel. (2016). Tutoring System [14] Han Wen Guo, Koichi Haraikawa, Jiann Shing Shein (2016) Heart Rate
Implementing Facial Expression Recognition Through Learner's Affect Variability Signal Features for Emotion Recognition by Using Principal
Classification. CAITE 2016 4th National Conference, At Philippines, Component Analysis and Support Vectors Machine, 2016 IEEE 16th
Volume: 4. International Conference on Bioinformatics and Bioengineering (BIBE),
[4] Mehendale, N. Facial emotion recognition using convolutional neural Taichung, 2016, pp. 274-277, doi: 10.1109/BIBE.2016.40.
networks (FERC). SN Appl. Sci. 2, 446 (2020). [15] Han Wen Guo, Koichi Haraikawa, Jiann Shing Shein (2015) Short-term
https://doi.org/10.1007/s42452-020-2234-1 Analysis of Heart Rate Variability for Emotion Recognition via a
[5] J. Cai, G. Liu, and M. Hao, "The Research on Emotion Recognition Wearable ECG Device, International Conference on Intelligent
from ECG Signal," 2009 International Conference on Information Informatics and Biomedical Sciences (ICIIBMS), Okinawa, 2015, pp.
Technology and Computer Science, Kiev, 2009, pp. 497-500. 262-265, doi: 10.1109/ICIIBMS.2015.7439542.
[6] J. R. Balbin et al., "Development of a scientific system for assessment of [16] A. Zhou, J. Chen, J. Ding, and Z. Pan, "Face Recognition Based on
post-traumatic stress disorder patients using physiological sensors and Two-stage CNN Combined with Transfer Learning," 2019 IEEE 4th
feature extraction for emotional state analysis," 2017IEEE 9th International Conference on Image, Vision, and Computing (ICIVC),
International Conference on Humanoid, Nanotechnology, Information Xiamen, China, 2019, pp. 401-406, doi: 10.1109/ICIVC47709.2019.89.
Technology, Communication and Control, Environment and
Management (HNICEM), Manila, 2017, pp. 1-6, doi:
10.1109/HNICEM.2017.8269424.

18
 2021 the 13th International Conference on Computer and Automation Engineering

A Noise Removal Approach from EEG Recordings

Based on Variational Autoencoders
Jamal F. Hwaidi Thomas M. Chen
Department of Electrical and Electronic Engineering Department of Electrical and Electronic Engineering
City, University of London City, University of London
London, United Kingdom London, United Kingdom
[email protected] [email protected]

Abstract—This paper presents a novel approach for reducing In this paper, a novel unsupervised learning approach in
noise in electroencephalography (EEG) signals using the Vari- noise removal by using Variational AutoEncoders (VAE),
ational AutoEncoders (VAE) algorithm. VAE’s are attractive as compared with the FastICA algorithm which is an efficient
they are designed on top of standard function approximators
neural networks and can be trained with stochastic gradient and popularly used algorithm for noise removal, and a mis-
descent to produce the desired output. Moreover, VAE has been classification as opposed to the classification accuracy were
compared with standard fast fixed-point algorithm for indepen- presented.
dent component analysis (FastICA) algorithm to measure the The remainder of the paper is organised as follows: Section
performance quantitatively by using machine learning algorithms II provides a comprehensive review of the main approaches
like Support Vector Machines, Naive Bayes, and Decision Tree.
The catch of this algorithm is utilised using the concept of developed for noise removal. Section lll describes the proposed
misclassification as opposed to the classification accuracy of the VAE and FastICA algorithms. Section IV and V provide the
above mentioned algorithms. experimental results and discussion of the methods and finally,
Index Terms—EEG, Noise removal, misclassification, FastICA, conclusions are drawn in section VI.
VAE
II. L ITERATURE R EVIEW
I. I NTRODUCTION In this on-going research some effective noise reduction
techniques are discussed. Nikam et al. [7] used FastICA
The human brain includes nerve cells, consisting of axons, algorithm with kurtosis and Negentropy as a measure of
dendrites, and cell bodies. Stimulated by various types of non-Gaussianity for extracting clean fetal electrocardiograms
stimuli (for example, the chemical in synaptic activity) the ions (FECG) signals which is essential to monitor fetal health. This
in the cell bodies are exchanged in the direction determined approach showed that Negentropy is a better measure of non-
by the membrane potential across the neuron membrane [1]. Gaussianity than kurtosis for the FastICA algorithm. Further-
The electrical impulses are distributed along the axon and the more, the algorithm is proposed based on a maternal R-peak
signals are transmitted by dendrites to other cells. Measuring suppression method based on FastICA that provides 90.35%
the electrical field over a human subject’s scalp gives a reading extraction of fetal R-peaks when validated by direct scalp
of the signals or Electroencephalography (EEG) signals. FECG. Mishra et al. [8] combined Stationary Wavelet Trans-
Electroencephalogram (EEG) records the electrical activity form (SWT) and Independent Component Analysis (ICA) as
of the brain by measuring voltage fluctuations on the surface a unique approach to mitigate noise from EEG signals. More
of the scalp by placing electrodes on the skin [2]. The brain precisely, FastICA with kurtosis technique was implemented
signals are brain activities that are determined by the unique to achieve maximum non-Gaussian and artefacts modelled on
pattern of neural pathways of the person and therefore cannot varying SNR values as Gaussian noises were applied to the
be imitated [3]. Such signals can be affected by the individual’s original EEG signal. The SWT–ICA method has been eval-
mood and mental state [4], making it very hard to obtain them uated by using the SNR parameter which indicates adequate
under force and threat. Moreover, the brain signals are linked filtering of Additive White Gaussian Noise (AWGN) at varying
to the genetic information of the subject, making it unique to intensity levels. The acquired technique eliminates the higher-
everyone and remaining stable over time [5]. rate noise at lower SNR values and simplifies the use of single-
The EEG signal is very useful when it comes to study brain channel EEG for several applications. Moreover, Yuan et al.
functions, but the recording of it contains various sorts of [9] proposed an improved FastICA method which is based on
noises since the signal records have small amplitudes, they can the overrelaxation factor to extract fetal electrocardiograms
be electrical or can be produced by our bodies [6] which can (FECG). The proposed algorithm relaxes the requirement of
affect the analysis of the brain signals. In addition, in EEG initial weight vector which leads to increasing the efficiency
recordings, various objects such as eye blinking or muscle of convergence while preserving the convergence rate and has
movements create noise which is a complex task to detect. better performance in signal-to-noise ratio (SNR).

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 19

 In another study Gajbhiye et al. [10] introduced a unique where DKL (k) symbolises the non-negative kullback-Leibler
technique for elimination of noise from electroencephalogram divergence between the true and the estimated posterior. Max-
signals by applying multi-tier solution total variation (MTV) imising L (θ, φ, y, z) is therefore equivalent to maximising the
and multi-tier solution weighted total variation (MWTV) noise lower bound to the true objective in (1)
filtering schemes which segregate the EEG signal into vari-
ous sub-band signals by applying discrete wavelet transform log pθ (y | z) ≥ L (θ, φ, y, z) (3)
(DWT). A multi-tier solution is imposed on the sub bands
to compute the difference between approximated value and equals to
the interference signal. The accuracy of the signals generated max Eqφ (z | y) [log pθ (y | z)] − DKL (qφ (z | y) k p (z))
φ,θ
through multi-tier solution wavelet-based reconstruction are (4)
measured using signal to noise ratio and the percentage of To achieve the optimisation of the objective in (3) and (4), ob-
reduction in correction coefficients. servations are typically made that are tractable. As Gaussians
Hadiyoso et al. [11] used Empirical Mode Decomposition
with a diagonal covariance matrix, the prior p(z) and posterior
(EMD) which produced two crucial parts of the signal, the
pθ (y | z) allocations are parametrised; the prior is set to the
intrinsic mode function (IMF) and residue to remove the
isotropic unit Gaussian N (0,1). In this way, parametrising the
artefacts in EEG signals. The root mean square error (RMSE)
distributions allows the ”reparametrization trick” to be used to
and signal power were calculated to measure the performance.
estimate lower bound gradients with respect to the parameters
Results show an average value of RMSE 0.0295 which indi-
φ, where each random variable zi ∼ qφ (zi | y) = N (µi , σi ) is
cates a very small error, and the signal power is 0.004 dB at
parametrised as a differentiable noise variable transformation
frequencies below 1 Hz which is proof that EMD can minimize
N (0,1).
low-frequency power.
Alharbi [12] developed a new approach to analyse EEG
zi = µi + σi  (5)
signals. The work concentrates on eigenvalues distribution
of a scaled Hankel matrix. This process helps to identify B. FastICA
the count of eigenvalues that can be used for removal of The fast fixed-point algorithm for independent component
noise and extraction of signals for singular spectrum analysis. analysis (FastICA) is an Iterative Fixed-Point Minimisation
Moreover, it has versatile works such as differentiation of of shared information between approximate components [16]
normal EEG signals and epileptic seizure and attractive pattern [17].
extraction. The outcome shows the ability to discriminate and The observed signal x was centralised and whitened before
eliminate noise from both EEG signals. Moreover, Sun et using the FastICA algorithm. To remove the mean vector
al. [13] proposed a one-dimensional residual Convolutional of the signal from the observed signals, the mean removal
Neural Networks (1D-ResCNN) model for removing noise process was performed such that the observed signal becomes
from raw waveform-based EEG. An improved EEG unknown zero mean. hence, simplifying the FastICA algorithm. Using
noise detection CNN has a better outcome. Deep conventional the principal component analysis (PCA) whitening algorithm,
one-dimensional Neural network is used to create a regression the observation signal was whitened so that the components
model to depict a complex non-linear relation between noisy were uncorrelated after the whitening. The aim of the FastICA
and noiseless EEG signals and improved noiseless signal algorithm based on the iterative structure of the fixed point was
generation is emphasised by experimental values of reduced to make z = y T x the maximum non-Gaussianity, where y was
RMSE and Signal to noise ratio in number of cases. Further- a row of the separation matrix Y . In accordance with Kuhn-
more, an enhanced performance under unknown waveforms is Tucker condition, the optimisation of E

G y T
x


could be
also a key achievement for this approach. 

achieved under the constraint of E y T x =k y k2 = 1:
III. M ETHODS
E xg y T x − βy = 0


A. Variational Autoencoder (VAE) (6)
Variational Autoencoder (VAE) has become one of the most Where β is a constant and g(.) is a nonlinear function, which is
common approaches to unsupervised learning of complicated the derivative of G(.). We can obtain the fixed point iteration:
distributions [14] [15]. The purpose of the variational au-
y ← E xg y T x


toencoder (VAE) is to learn in such a generative method the (7)
marginal probability of the data:
After each iteration, Y must be normalised, but the con-
max Eqφ (z | y) [log pθ (y | z)] (1) vergence property of this formula is not apparent, as non-
φ,θ polynomial moments do not have the same strong algebraic
where the distributions of the VAE encoder and the decoder output as actual cumulants. The iteration of the fixed point is
φ, θ are respectively parameterised as the below equation thus modified as follows:

y = E xg y T x ⇔ (1 + α) y = E xg y T x + αy (8)



log pθ (y | z) = DKL (q (z | y) k p (z)) + L (θ, φ, y, z) (2)

20
The next step is the normalisation of y as a unit standard. The
Newton method is used to determine the α coefficients in the
formula. The newton iterative method was employed to solve
(6) as:

w ← E xg y T x − E g 0 y T x y



(9)
IV. E XPERIMENTAL R ESULTS
A. Dataset
The GigaDB dataset [18] used in this paper was provided by
Fig. 2. VAE functioning with the dataset.
Handong Global University and it consists of EEG recordings
of 52 healthy subjects with left and right-hand motor imagery
tasks, of which 19 were female subjects (mean age: 24.8 ± 3.86 for the signal exists, such that the ”source signals” or elements
years). The EEG data were obtained using 64 active Ag/AgCl of s have a positive kurtotic density, thus, in which case the
electrodes on the basis of the international 10–10 system. In ICA transform provides non-Gaussian components.
this experiment, 10 subjects were used for the implementation
After performing FastICA, we find the threshold value for
of the noise removal algorithm and there were 68 trials which
unwanted signals. After that based on that threshold value, we
represent the left side movement for each subject. Therefore,
perform Stationary Wavelet Transforms (SWT) and Inverse
we have a dataset for (10 subjects X 64 channels X 68 trials).
Stationary Wavelet Transforms (ISWT) which minimise the
B. Signal Reconstruction noise level. (SWT) removes the signal based on that threshold
The EEG signals were analysed based on the framework value and it gives an outcome based on wavelet form. So, we
illustrated in Fig 1. First, we add a Gaussian noise and mix it reconstruct the signal by (ISWT).
with the signals to create a noised signal for further processing. For VAE, as shown in Fig 2 the dataset is split into an
The noisy image model is described as: original signal set and a mixed noisy signal set. In VAE model
the mixed signal is set to target the original signal and the VAE
x=y+n (10) algorithm in the iterative process tries to minimise the loss
Where non-correlated noise is n, with elements in the window during the training process by tuning the hyper-parameters like
indexed in the same manner as y and x is a calculated learning rate, filters, kernel, strides, and number of iterations.
window that is distorted by noise. Moreover, n is considered Thus, in this process a denoised signal is obtained as the output
as Gaussian and y is considered as non-Gaussian. with a certain percentage of accuracy.
In brief, if we model the density of z by (ICA) and presume C. Classification
n Gaussian, the Maximum Likelihood (ML) solution for z can
be determined in the signal model if the measurement y. It is For comparison results four classification algorithms were
possible to evaluate the (ML) solution easily, albeit roughly, employed: 1) a parametric approach – Naive Bayes [19]. 2,
by using a decomposition that is a variant of the orthogonality 3, 4) nonparametric approaches – Support Vector Machine
of (ICA). The conversion is given by (SVM) [20], SVM (sigmoid) [21], and Decision Tree [22].
1) SVM: The SVM model is a representation of several
xy = xz + xn (11) points in space, mapped such that a simple distance that
is as large as possible separates the examples of the
Where xy is an orthogonal matrix which is the best orthog- various categories.
onal approximation of the mixing matrix inverse of the ICA. 2) SVM (sigmoid): is the non-linear implementation of
xn is still a Gaussian and white noise term. Nevertheless, with SVM.
a correctly selected orthogonal transform X, the density of 3) Decision Tree: is a flowchart-like diagram in which each
xz = s becomes extremely non-Gaussian, e.g., with a strong internal node represents a label, each branch represents
positive kurtosis super-Gaussian. This, of course, depends on the outcome of the label, and each leaf node represents
the original x signals, since we presume that a model x = wT s a class label (decision arrived after computing the total
attributes).
4) Naive Bayes: Naive Bayes classifiers are a basic proba-
bilistic classifier group based on applying the theorem of
Bayes with strong assumptions of (naive) independence
between the features. This algorithm is highly scalable
and has very high efficiency due to linear time rather
than iterative approximation.
The logic here is to use the amount of misclassification
Fig. 1. Block diagram for the noise removal framework in this study. percentage provided by each classification algorithms as a

21
 signal. The performance is 54% recovery from the noised
signal.
The performance may be checked by using other forms of
VAE like native Bayes VAE , Vector Quantized Variational
AutoEncoder (VQ-VAE), etc. From the architecture point of
view the improvement can be done by adding more layers or
by introducing the layers of any standard algorithm like VGG
Neural Networks, Residual Networks (ResNet), etc.
The main advantage of this algorithm is
1) The basis of it is unsupervised by nature.
2) VAE has more parameters to tune that gives significant
control over how we want to model our latent distribu-
tion.
3) The decoder is a generative model projecting from latent
random variables to the input space. The encoder learns
what kind of noise has to be inserted in the code layer,
Fig. 3. Output of original signals, mixed signals, FastICA and PCA. in order to make the decoder a good generative model.
The reparametrisation trick makes it possible to use
back-propagation for training the whole model. Both,
performance measure. The higher the misclassification is the the encoder and decoder, typically consist of 2-hidden
better is the denoised signal. The misclassification is carried layers each, which enables the Universal approximation
between the original signal to the denoised signal by using the theorem for both of them individually: approximate any
above supervised algorithms. function (within a broad class) to an arbitrary degree
of accuracy; given at least 2 hidden layers and enough
V. D ISCUSSION neurons.
Fig 3 illustrates qualitatively how the mixed waveform is Table I represents the misclassification percentage between
reconstructed to the original waveform. Apparently the true the original signal matrix to denoised signal matrix for VAE
signals shows that the signals are on the baseline and saturated and FastICA.
through the amplitude increasing while in the mixed signals The overall picture suggests that VAE produces better signal
it’s becoming distorted as we add noise. Moreover, the signal reconstruction than the FastICA. This may be attribute to the
is about to diverge at some points as opposed to FastICA better nonlinear mapping of the signals as compared to the
and PCA signals as seen from the figure. Hence, PCA based FastICA. Normally on average the VAE accuracy was 54%
whitening is used to get better convergence but how much which very well matches with SVM classifier model. As a
quantitatively can be seen through misclassification output. result, we can conclude that accuracy is linearly separated
The following performance graph in Fig 4 depict the use across the original value to the reconstructed value. Hence,
of simple Gaussian approach towards variational autoencoder This data should be validated by using the above results in ap-
(VAE) in reconstructing the original signal from the noised plication where denoised signals are used. Furthermore, these
applications will show how efficient the extraction percentage
of noise removal means.
VI. C ONCLUSION AND FUTURE WORK
In this paper, a new misclassification technique was used
for the decomposition of EEG signals, which is based on the
Variational Autoencoders (VAE).
This efficiency metrics can also be used for other algorithms
by using Maximum Likelihood (ML) as the tool. Results fur-
ther show that better non-linear function can more efficiently
capture and remove the noisy envelope over the original data.
To distinguish the noise, other variants of Variational Au-
toencoder can be implemented and Generative Adversarial
Network (GAN) can also be used to distinguish the noise.
Furthermore, it will be interesting to observe the dependency
of various EEG signals on various networks as described
above. It will also be very important to study the architecture
of the VAE to implement various changes to improve upon
Fig. 4. VAE efficiency performance graph. the efficiency of the noise removal.

22
 TABLE I.
N OISE F ILTRATION P ERFORMANCE .

FastICA VAE
Subjects
SVM SVM (Sigmoid) Naive Bayes Decision Tree SVM SVM (Sigmoid) Naive Bayes Decision Tree

S1 0.5662 0.4485 0.0515 0.0956 0.5862 0.5485 0.1516 0.1957

S2 0.5221 0.5147 0.0588 0.0735 0.5221 0.5147 0.2588 0.2737
S3 0.5074 0.0735 0.0221 0.0368 0.5174 0.5735 0.1921 0.1328
S4 0.5882 0.5956 0.0735 0.0882 0.5982 0.6956 0.3735 0.3882
S5 0.4662 0.4678 0.0815 0.0956 0.6120 0.5485 0.4515 0.4956
S6 0.5521 0.5647 0.0788 0.0735 0.5621 0.6148 0.2588 0.2735
S7 0.5074 0.0735 0.0782 0.0368 0.5874 0.4735 0.4221 0.3368
S8 0.5769 0.5877 0.0789 0.0882 0.5982 0.6356 0.3735 0.3882
S9 0.5621 0.5547 0.0728 0.0715 0.5821 0.6048 0.3588 0.2935
S10 0.5073 0.0742 0.0221 0.0365 0.5823 0.6134 0.2888 0.2335

R EFERENCES [16] A. Hyvarinen, “Fast and robust fixed-point algorithms for independent
component analysis,” IEEE transactions on Neural Networks, vol. 10,
[1] B. Obermaier, C. Neuper, C. Guger, and G. Pfurtscheller, “Information no. 3, pp. 626–634, 1999.
transfer rate in a five-classes brain-computer interface,” IEEE Transac- [17] E. Oja and Z. Yuan, “The fastica algorithm revisited: Convergence
tions on neural systems and rehabilitation engineering, vol. 9, no. 3, pp. analysis,” IEEE transactions on Neural Networks, vol. 17, no. 6, pp.
283–288, 2001. 1370–1381, 2006.
[2] E. Maiorana, G. E. Hine, D. La Rocca, and P. Campisi, “On the [18] H. Cho, M. Ahn, S. Ahn, M. Kwon, and S. C. Jun, “Eeg datasets for
vulnerability of an eeg-based biometric system to hill-climbing attacks motor imagery brain–computer interface,” GigaScience, vol. 6, no. 7, p.
algorithms’ comparison and possible countermeasures,” in 2013 IEEE gix034, 2017.
Sixth International Conference on Biometrics: Theory, Applications and [19] P. A. Flach and N. Lachiche, “Naive bayesian classification of structured
Systems (BTAS). IEEE, 2013, pp. 1–6. data,” Machine Learning, vol. 57, no. 3, pp. 233–269, 2004.
[3] R. Palaniappan and D. P. Mandic, “Biometrics from brain electrical [20] C.-C. Chang and C.-J. Lin, “Libsvm: a library for support vector
activity: A machine learning approach,” IEEE transactions on pattern machines,” ACM transactions on intelligent systems and technology
analysis and machine intelligence, vol. 29, no. 4, pp. 738–742, 2007. (TIST), vol. 2, no. 3, pp. 1–27, 2011.
[4] S. Marcel and J. d. R. Millán, “Person authentication using brainwaves [21] K. S. Durgesh and B. Lekha, “Data classification using support vector
(eeg) and maximum a posteriori model adaptation,” IEEE transactions machine,” Journal of theoretical and applied information technology,
on pattern analysis and machine intelligence, vol. 29, no. 4, pp. 743– vol. 12, no. 1, pp. 1–7, 2010.
752, 2007. [22] L. Rokach and O. Maimon, “Top-down induction of decision trees
[5] M. Näpflin, M. Wildi, and J. Sarnthein, “Test–retest reliability of classifiers-a survey,” IEEE Transactions on Systems, Man, and Cyber-
resting eeg spectra validates a statistical signature of persons,” Clinical netics, Part C (Applications and Reviews), vol. 35, no. 4, pp. 476–487,
Neurophysiology, vol. 118, no. 11, pp. 2519–2524, 2007. 2005.
[6] P. Khatwani and A. Tiwari, “A survey on different noise removal
techniques of eeg signals,” International Journal of Advanced Research
in Computer and Communication Engineering, vol. 2, no. 2, pp. 1091–
1095, 2013.
[7] S. Nikam and S. Deosarkar, “Fast ica based technique for non-invasive
fetal ecg extraction,” in 2016 Conference on Advances in Signal Pro-
cessing (CASP). IEEE, 2016, pp. 60–65.
[8] A. Mishra, V. Bhateja, A. Gupta, and A. Mishra, “Noise removal in
eeg signals using swt–ica combinational approach,” in Smart Intelligent
Computing and Applications. Springer, 2019, pp. 217–224.
[9] L. Yuan, Z. Zhou, Y. Yuan, and S. Wu, “An improved fastica method
for fetal ecg extraction,” Computational and mathematical methods in
medicine, vol. 2018, 2018.
[10] P. Gajbhiye, R. K. Tripathy, A. Bhattacharyya, and R. B. Pachori, “Novel
approaches for the removal of motion artifact from eeg recordings,”
IEEE Sensors Journal, vol. 19, no. 22, pp. 10 600–10 608, 2019.
[11] S. Hadiyoso and I. Wijayanto, “Noise removal in mild cognitive impair-
ment eeg recording using empirical mode decomposition,” in 2019 IEEE
International Conference on Signals and Systems (ICSigSys). IEEE,
2019, pp. 74–78.
[12] N. Alharbi, “A novel approach for noise removal and distinction of eeg
recordings,” Biomedical signal processing and control, vol. 39, pp. 23–
33, 2018.
[13] W. Sun, Y. Su, X. Wu, and X. Wu, “A novel end-to-end 1d-rescnn
model to remove artifact from eeg signals,” Neurocomputing, vol. 404,
pp. 108–121, 2020.
[14] C. Doersch, “Tutorial on variational autoencoders,” arXiv preprint
arXiv:1606.05908, 2016.
[15] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv
preprint arXiv:1312.6114, 2013.

23
 2021 the 13th International Conference on Computer and Automation Engineering

A Novel Feature Selection Method on Mutual

Information and Improved Gravitational Search
Algorithm for High Dimensional Biomedical Data
Chaokun Yan, Xi Kang, Mengyuan Li, Jianlin Wang*
School of Computer and Information Engineering
Henan University
Kaifeng, China
[email protected], [email protected], [email protected], [email protected]

Abstract—In the past few decades, the field of bioinformatics Feature selection is a common method to analyze and process
has accumulated a large amount of gene expression data which these high-dimensional data [2]. The essence of feature
provided important support for the diagnosis of disease. selection is to select the smallest feature subset from all the
However, high dimensionality, small sample sizes, and redundant features, which can effectively remove the redundant and
features often adversely affect the accuracy and the speed of irrelevant features in the data. As an effective dimension
prediction. Existing feature selection models cannot obtain the
reduction method, feature selection has been widely concerned
information of these datasets accurately. Filter and wrapper are
two commonly used feature selection methods. Combining the in the biomedical field. In general, feature selection methods
advantages of the fast calculation speed of the filter and the high are categorized into filter, wrapper and embedded [3]. The
accuracy of the wrapper, a new hybrid algorithm called MI- filter method is based on the evaluation features of
IBGSA, is proposed, which hybridizes mutual information and mathematical functions, which can reduce the calculation time
improved Gravitational Search Algorithm (GSA). First, mutual and is conducive to the processing of large data sets.
information is used to rank and select important features, these Information Gain [4], ReliefF [5], MRMR [6], and Mutual
features are further chosen into the population of the wrapper Information [7] are typical filter methods. However, many
method. Then, due to the effectiveness of the GSA algorithm, filter methods tend to ignore the dependency relationship
GSA is adopted to further seek an optimal feature subset. between features, which is not conducive to obtaining feature
However, GSA also has the disadvantages of slow search speed subset with high classification accuracy. The wrapper method
and premature convergence, which limit its optimization ability.
uses the classifier feedback to select the best feature subset in
In our work, a scale function is added to the speed update to
enhance its searchability, and an adaptive 𝒌𝒃𝒆𝒔𝒕 particle update
the feature space, which can obtain better classification
formula is proposed to improve its convergence accuracy and accuracy but has more calculation cost [8]. The feature
propose a fitness sharing strategy to enhance the randomness of selection of the embedded method can be regarded as a special
particle populations and searchability through the niche type of wrapper method. The selection process is carried out in
algorithm of fitness sharing. We used 10- fold-CV method with the training process, generally for a specific learning machine,
the KNN classifier to evaluate the classification accuracy. and the learning process can not be separated from feature
Experimental results on five publicly available high-dimensional selection [9].
biomedical data sets show that the proposed MI-IBGSA has
superior performance than other algorithms. Different methods have their advantages and disadvantages.
Therefore, the hybrid method combining multiple methods is
Keywords—Feature selection, Gravitational Search Algorithm, an excellent way that can achieve high accuracy and low time
Mutual Information, scale function, adaptive 𝒌𝒃𝒆𝒔𝒕 particle update, consumption. In this study, we proposed a hybrid feature
fitness sharing strategy, Niche algorithm selection method MI-IBGSA based on Gravitational Search
Algorithm (GSA) and Mutual Information (MI) that improve
I. INTRODUCTION the accuracy of feature selection on the high-dimensional
In the past many years, a large number of biomedical data biomedical data. In our proposed method, the scaling function
have been generated and accumulated at an incredible speed is used to change the update formula of velocity to improve the
[1]. Most biomedical data sets describe patient records through searchability of the GSA algorithm, the adaptive 𝑘𝑏𝑒𝑠𝑡 particle
various features. Some of these features may be redundant and formula is used to improve the convergence accuracy, and the
useless. It is hard to make accurate judgment or treatment for niche algorithm with fitness sharing is used to change the
patients, and it takes up a lot of memory space to store these solution of inertia mass, and enhance the particle randomness
data, and ordinary compute methods also need to use large and search performance of the algorithm. MI-IBGSA uses MI
computing resources. Some advances in data mining and to filter candidate feature subset, which can accelerate the
machine learning provide an alternative method. However, the convergence of the algorithm.
performance of machine learning tasks is degraded because of
the irrelevant or noisy features of these biomedical data.

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 24

 II. RELATED WORK A. The Gravitational Search Algorithm (GSA)
In recent years, the filter-based method can save time and The gravitational search algorithm proposed by Rashedi et
computational cost, but it may not be able to obtain a feature al. [20] is a relatively new optimization technique, which used
subset with high classification accuracy. The wrapper-based the interaction between particles to complete information
method can obtain a feature subset with higher classification sharing. The binary form of GSA, namely BGSA [21], is
accuracy, but it also needs higher computational cost. superior to other intelligent algorithms in terms of convergence
speed and solution accuracy and is used by many scholars in
In this section, we will introduce several new methods image processing problems [22], function optimization
based on wrapper and hybrid filter wrapper as benchmark problems [23], production scheduling problems [24] and other
methods. Based on wrapper methods such as Artificial Bee fields. According to Newton’s law of universal gravitation.
Colony (ABC) [10], Binary Salp Swarm Algorithm (BSSA) Consider an isolated particle (masses) system, each particle in
[11], are exhibited detail. In addition, two hybrid methods such GSA can be regarded as a moving object with mass in space.
as Information Gain Genetic Algorithm (IG-GA) [12], The particle with a large mass has a better position,
Information Gain Particle Swarm Optimization (IG-PSO) [13] corresponding to a better fitness value.
and Correlation Feature Selection based improved-Binary
Suppose there are 𝑁 particles in d-dimensional space, and
Particle Optimization (CFS-BPSO) [14], which exhibited the
the position of particle 𝑖 is expressed as:
performance of combined filter and wrapper method.
For Cleveland heart disease dataset, Artificial Bee Colony 𝑃𝑖 = (𝑃𝑖1 , 𝑃𝑖2 , . . . . . . , 𝑃𝑖𝑑 ) (1)
(ABC) [10] only uses seven features to obtain the best here, 𝑃𝑖𝑑 is the position of particle 𝑖 in d dimension, and the
classification accuracy, and its performance is better than other position of each particle represents a solution. The force from
feature selection methods with reverse ranking. ABC algorithm particle 𝑗 on particle 𝑖 is:
is used to select the features of the disease, and KNN classifier
is used to verify the performance, which shows excellent 𝑀𝑖 (𝑡) × 𝑀𝑗 (𝑡)
𝐹𝑖𝑗𝑑 = 𝐺(𝑡) × × (𝑃𝑗𝑑 (𝑡)
performance on Cleveland heart disease dataset [15]. 𝑅𝑖𝑗 + 𝑐 (2)
The BSSA [11], as a wrapper method, introduced the − 𝑃𝑖𝑑 (𝑡))
transfer function to convert the continuous version of SSA into 𝑀𝑖 and 𝑀𝑗 are the inertial mass of particles 𝑖 and 𝑗, 𝐺(𝑡) is the
binary, and the crossover operator is used to replace the gravitational constant, C is a small constant to prevent the
average operator to enhance the exploration ability of the divide by zero errors, 𝑅𝑖𝑗 is the Euclidean distance between
algorithm. Compared with other algorithms, BSSA achieves particle 𝑖 and particle 𝑗:
superior classification accuracy on ColonTumor and Leukemia.
𝑡 𝛽
Sarafrazi et al. proposed a GSA-SVM method to select 𝐺(𝑡) = 𝐺(0) × ( ) (3)
𝑡𝑚𝑎𝑥
features [16]. J. Xiang combined K-Nearest Neighbor
technique with BGSA to obtain the optimal feature subset [17]. 𝐺(0) is the initial value of gravity, 𝑡𝑚𝑎𝑥 is the maximum
Nagpal used GSA and the K-Nearest Neighbor for feature number of iterations, 𝛽 < 1 . The resultant force and
selection of medical data sets. The average number of features acceleration of particle 𝑖 in the d dimension are defined as:
is reduced by 66%, and the accuracy of prediction is greatly 𝑁
improved [18].
𝐹𝑖𝑑 (𝑡) = ∑ 𝑟𝑗 × 𝐹𝑖𝑗𝑑 (𝑡) (4)
Even though these wrapper methods have been 𝑗=1,𝑗≠𝑖
successfully applied to biomedical research, which still
requires high computational costs when dealing with large 𝑟𝑗 is the random number between [0,1], 𝐹𝑖𝑗𝑑 (𝑡) represents the
amounts of characteristic data. The filter method can quickly gravity value of particle j to particle i:
select important information. Combined the advantages of 𝐹𝑖𝑑 (𝑡)
filter and wrapper, which is a promising hybrid method, Salem 𝑎𝑖𝑑 (𝑡) = (5)
H et al. [12] provided a hybrid method combined information 𝑀𝑖 (𝑡)
gain and genetic algorithm (IG-GA). The genetic programming The updating formula of velocity and position of particle i
(GP) was judged the quality of cancer classification and in d dimension is as follows:
achieved excellent performance on some biomedical datasets.
𝑉𝑖𝑑 (𝑡 + 1) = 𝑟𝑗 × 𝑉𝑖𝑑 (𝑡) + 𝑎𝑖𝑑 (𝑡) (6)
In addition to IG and some filter methods has mentioned, as
an effective method to access the importance of features, the 𝑃𝑖𝑑 (𝑡 + 1) = 𝑃𝑖𝑑 (𝑡) + 𝑉𝑖𝑑 (𝑡 + 1) (7)
mutual information method is used to select important features
𝑉𝑖𝑑 (𝑡) and 𝑃𝑖𝑑 (𝑡) are the velocity and position of particle i. 𝑟𝑗
for many classification tasks [19].
can increase the randomness. The inertial mass of particles is
III. THEORETICAL BACKGROUND determined by the fitness value. The larger the inertia mass, the
closer to the optimal solution
In this part, we will briefly introduce the Gravitational
Search Algorithm and the MI (mutual information) algorithm. 𝑓𝑖𝑡𝑛𝑒𝑠𝑠𝑖 (𝑡) − 𝑓𝑖𝑡𝑤𝑜𝑟𝑠𝑡(𝑡)
𝑚𝑖 (𝑡) = (8)
𝑓𝑖𝑡𝑏𝑒𝑠𝑡(𝑡) − 𝑓𝑖𝑡𝑤𝑜𝑟𝑠𝑡(𝑡)

25
 𝑚𝑖 (𝑡) improved binary GSA(BGSA) algorithm with superior
𝑀𝑖 (𝑡) = 𝑁 (9) searching will obtain the optimal feature subset with high
∑𝑗=1 𝑚𝑗 (𝑡)
accuracy and a small number of features. Fig. 1 will
𝑓𝑖𝑡𝑛𝑒𝑠𝑠𝑖 (𝑡) is the fitness value of the t-th times iteration, and demonstrate the flow of the MI-IBGSA algorithm.
𝑓𝑖𝑡𝑏𝑒𝑠𝑡(𝑡) and 𝑓𝑖𝑡𝑤𝑜𝑟𝑠𝑡(𝑡) are the optimal and the worst
values of the fitness.
For our purpose, we have employed the binary version of
the gravity search algorithm (BGSA) [21]. In the discrete space,
we have initialized the position of the population randomly
with the values 0 and 1. And the function 𝑇(𝑉𝑖𝑑 (𝑡)) transfers
𝑉𝑖𝑑 into a probability function.
𝑇(𝑉𝑖𝑑 (𝑡)) = |𝑡𝑎𝑛ℎ( 𝑉𝑖𝑑 (𝑡))| (10)
Then the particle will then move as follows if the random
number rand is less than 𝑇(𝑉𝑖𝑑 (𝑡)),
𝑃𝑖𝑑 (𝑡 + 1) = 1 − 𝑃𝑖𝑑 (𝑡) (11)
Otherwise the particle position is as follows:
𝑃𝑖𝑑 (𝑡 + 1) = 𝑃𝑖𝑑 (𝑡) (12)
Fig. 1. The flow of the MI-IBGSA algorithm
B. MI Algorithm
The MI (Mutual Information) method is a basic concept of A. Initialization Based on Mutual Information
information theory [25], and its definition is closely related to The initializing population, vectors of BGSA, plays an
information entropy. Mutual information can measure the important role in algorithm iteration. We change the original
relationship between features and categories [26]. Mutual initialization process of BGSA and adopt the mutual
information is used to evaluate the amount of information held information method for initialization. By calculating and
by random variables, which can measure the correlation sorting the information between features and categories, and
between two random variables, so it can be applied to feature improve the efficiency of the algorithm. The initialization
selection naturally. Entropy can measure the uncertainty of the process is described as follow:
value of the random variable, X is the value of random variable
𝑥𝑖 (𝑖 = 1,2. . . ) , corresponding to different probabilities Each particle is regarded as a binary one-dimensional
vector, also known as the solution vector with M features. For
𝑃(𝑥𝑖 )(𝑖 = 1,2. . . ) , the entropy of classification problem is
defined as: each binary variable 𝑋𝑖𝑗 , 𝑖 is the index of the solution and 𝑗 is
the dimension of the solution., and 𝑋𝑖𝑗 ∈ {0,1}. If the feature 𝑓𝑗
𝑙
is selected then 𝑋𝑖𝑗 = 1, else 𝑋𝑖𝑗 = 0.
𝐻(𝑋) = − ∑ 𝑃(𝑐𝑖 ) 𝑙𝑜𝑔2 ( 𝑃(𝑐𝑖 )) (13)
𝑖=1
The population is initialized by Mutual Information method,
calculating the 𝐼(𝑡, 𝑐) by Eq. (17), 𝐼(𝑡, 𝑐) indicates that the
here, 𝐶(𝑐1 , 𝑐2 . . . ) is a variable and l is the number of categories. feature contains more informational gene for classification.
In the preliminary screening, mutual information is usually
used to calculate the relationship between features. B. Performance Metrics
Considering the distribution of feature t and category c, the In this paper, we used six evaluation indexes, all of which
mutual information between feature t and category c is defined were calculated by four values, TP(True Positive), TN(True
as: Negative), FP(False Positive), FN(False Negative).
𝑃(𝑡, 𝑐) Accuracy is the most basic evaluation index, which
𝐼(𝑡, 𝑐) = 𝑙𝑜𝑔2 (14)
𝑃(𝑡) × 𝑃(𝑐) represents the percentage of correct samples in the total
samples.
If 𝐼(𝑡, 𝑐) = 0 , it can be concluded that feature t is
independent of category c. Using mutual information for 𝑇𝑃 + 𝑇𝑁
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = (15)
feature selection can provide as much category information as 𝑇𝑃 + 𝐹𝑃 + 𝐹𝑁 + 𝑇𝑁
possible for classification. The greater the amount of
information between the two variables, the higher the degree of Precision is the proportion of correct classification in
correlation. Conversely, the degree of correlation is lower. predicted individuals.
𝑇𝑃
IV. THE PROPOSED FEATURE SELECTION METHOD: MI-IBGSA 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 = (16)
𝑇𝑃 + 𝐹𝑃
In this section, a novel feature selection method MI-IBGSA
is presented. First, the feature subset containing important Recall is the probability that the actual positive sample is
features is filtered out by using MI method. Subsequently, the predicted to be a positive sample.

26
 𝑇𝑃 In the last iteration, 𝑓𝑖𝑛𝑎𝑙_𝑝𝑒𝑟 represents the percentage of
𝑅𝑒𝑐𝑎𝑙𝑙 = (17)
𝑇𝑃 + 𝐹𝑁 𝑘𝑏𝑒𝑠𝑡 particles in the population. t is the current number of
Sensitivity is the proportion of correct classification in iterations, 𝑚𝑎𝑥𝑡 is the maximum number of iterations.
positive samples, and specificity is the proportion of correct Then, the population is randomly divided into k groups
classification in negative samples. corresponding to k niches, and the shared fitness is obtained by
𝑇𝑃 dividing the original individual fitness through the shared value.
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 = (18)
𝑇𝑃 + 𝐹𝑁 𝑑𝑖𝑐 𝛼
𝑠𝑖 = 𝑛𝑐 − 𝑛𝑐( ) (23)
𝑇𝑁 2𝑑𝑚𝑎𝑥
𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 = (19)
𝑇𝑁 + 𝐹𝑃 here, 𝑑𝑖𝑐 represents the center distance from individual 𝑖 to
C. Objective Function group 𝑐, 𝑛𝑐 represents the number of particles in group 𝑐, and
𝛼 is 1. 𝑑𝑚𝑎𝑥 is the maximum distance between two cluster
The fitness value of the individual is calculated by the centers, 𝑑𝑚𝑖𝑛 represents the minimum distance between cluster
objective function to evaluate the quality of the solution. In this centers.
paper, the fitness function is adopted as used in the literature
[18]: Then, take the first k particles as the center of the niche and
check whether the niche distance is greater than 𝑑𝑚𝑖𝑛 . If the
𝑓𝑖𝑡𝑛𝑒𝑠𝑠 = 𝛼 × 𝛾 + 𝛽 × (𝑇 − 𝑆)/𝑆 (20)
center distance is less than 𝑑𝑚𝑖𝑛 , the niches were merged, and a
𝛾 is the accuracy of selected features, 𝑇 is the total number of new niche individual center was selected. The distance
features, and 𝑆 is the number of features selected from the between the unselected particles and the existing niche center
feature subset. 𝛼, 𝛽 is a constant that controls the accuracy of will be calculated respectively, if the distance greater than
the classification and the length of feature subsets. 𝛼 ∈ [0,1]. 𝑑𝑚𝑎𝑥 , the particle is regarded as the center of the newborn
niche. Otherwise, it is placed in the nearest niche. The distance
D. Implementation Steps of MI-IBGSA between the niche center and 𝑑𝑚𝑖𝑛 , if the distance was less
In order to overcome the shortcomings of the GSA than 𝑑𝑚𝑖𝑛 , the corresponding niches were merged. After the
algorithm and improve its explore capacity, the fitness sharing niche center is placed, the current center is fixed, and then all
of the Niche algorithm [27] is combined with the algorithm to particles are arranged in the nearest niche. The shared values of
form a quality sharing mechanism to enhance the search individuals are calculated according to formula (24). Calculate
performance of the algorithm. In addition, an adaptive 𝑘𝑏𝑒𝑠𝑡 the fitness value after particle sharing. In the gravitational
update formula is added to improve the convergence accuracy. search algorithm, the mass is obtained by fitness. We transform
The specific operation of the MI-IBGSA is demonstrated as fitness value sharing into mass sharing. The particle mass is
follows. 𝑀[𝑖].
Step 1: Population initialization. In order to improve search 𝑀[𝑖] × 𝑑𝑚𝑎𝑥
performance and convergence speed, MI is used to initialize 𝑀[𝑖] = (24)
the population. 2𝑛𝑐(1 − 𝑑𝑖𝑐 )
Step 2: Evaluation of the fitness of each particle. Update Due to the mechanism of the algorithm, the change of mass
𝑓𝑖𝑡𝑏𝑒𝑠𝑡(𝑡), 𝑓𝑖𝑡𝑤𝑜𝑟𝑠𝑡(𝑡),𝐺(𝑡) and 𝑀[𝑖] for 𝑖 = 1,2 … 𝑁. will affect the update of acceleration, velocity, and position.
Furthermore, the local search performance of the algorithm is
At the beginning of the iteration, the particles move improved.
towards the larger mass particles. The scale function and
particle position are added into the particle velocity updating Step 3: Computation of the total value of force acting in
formula, the velocity can be dynamically adjusted so that the various directions.
algorithm can obtain the balanced searchability. The updated
velocity formula is. Step 4: Computation of the values of velocity (𝑉𝑖𝑑 (𝑡)) and
acceleration (𝑎𝑖𝑑 (𝑡)).
𝑉𝑖𝑑 (𝑡 + 1) = 𝑟𝑗 × 𝑉𝑖𝑑 (𝑡) + 𝑐1 × 𝑎𝑖𝑑 (𝑡) + 𝑐2 (21)
× (𝑥𝑏𝑒𝑠𝑡 − 𝑥𝑖 ) Step 5: Update of the position (𝑃𝑖𝑑 (𝑡)) of particles.
Among them, 𝑐1 = 1 − 𝑡/𝑚𝑎𝑥𝑡 , 𝑐2 = 𝑐 × 𝑒𝑥𝑝( − Step 6: Repeating steps 2 to 5 till the stopping condition has
𝑡/ 𝑚𝑎𝑥𝑡 ), 𝑥𝑏𝑒𝑠𝑡 is the best position of the particle, t is the been attained.
current number of iterations, 𝑚𝑎𝑥𝑡 is the maximum number of
iteration. V. EXPERIMENT AND DISCUSSION
Use the evaluation function Eq. (23) to calculate the value This section describes the experimental results on some
for each individual and set 𝑘𝑏𝑒𝑠𝑡 particles. 𝑘𝑏𝑒𝑠𝑡 particles public datasets and the comparison methods: ABC [10], BSSA
ordered to maintain the diversity of the population, we added [11], IG-GA [12], IG-PSO [13]. BSSA is a novel wrapper
an adaptive formula to make it shrink quickly in the iteration method had proposed in recent years. IG-GA and IG-PSO are
process to improve the convergence accuracy. representative hybrid methods.

𝑘𝑏𝑒𝑠𝑡 = 𝑓𝑖𝑛𝑎𝑙_𝑝𝑒𝑟 + (1 − 𝑡/ 𝑚𝑎𝑥𝑡 ) × (100 In order to make our results have statistical significance, we
(22) used the KNN classifier based on Euclidean distance to
− 𝑓𝑖𝑛𝑎𝑙_𝑝𝑒𝑟)
evaluate the classification accuracy. In our experiment, each

27
biomedical dataset is divided into 80% training sets and 20% B. Parameter Setting
test sets. The training set is used to train the classifier to In this section, the parameters set for the proposed method
recognize features, while the test set is used to evaluate the and comparison methods are shown in Table II.
trained classifier by using the test set. All algorithms are
executed on the same partition and evaluated on the same test TABLE II. PARAMETER SETTING
data. The average value of five evaluation indexes in five tests
Algorithm Parameter Value
is used as the basis for comparing the performance of the
method. MI-IBGSA N=20; 𝐺(𝑡) = 1; TopN=50; 𝑘𝑏𝑒𝑠𝑡 = 20; 𝛼 = 0.8;
𝛽 = 0.2;Number of iteration=200
A. Datasets
ABC Population =20; Number of employed bees and
In our experiment, five public biomedical datasets are used onlooker bees =12; Maximum number of iterations
to evaluate the performance of proposed algorithm MI-IBGSA =100; Limit=10
and other advanced algorithms. Three datasets ALL-AML,
Colon Tumor and Ovarian Cancer can be download form the BSSA Population =10; Maximum number of iterations =100;
website: http://csse.szu.edu.cn/staff/zhuzx/Datasets.html. α=0.99, β=0.01
Otherwise, GSE10245 and GSE4115 can be downloaded from IG-GA Population =100; Crossover Probability = 0.8;
GEO website: http://www/ncbi.nlm.nih.gov/geo. The Mutation Probability=0.1
information of dataset shown in Table I.
IG-PSO Population =20; value of rand between [0,1];
TABLE I. THE BENCHMARK OF DATASETS Acceleration factors=2; Inertia weight w=1; mutation
rate=0.1; Crossover rate=1; Iteration =100
Dataset Instances Attributes Classes
ALL-AML 70 7129 AML:25
ALL:47 C. Results and Ayalysis
Colon Tumor 62 2000 Tumor:40 We verify the performance of the proposed MI-IBGSA
Normal:22 through the massive number of experiments. MI-IBGSA is
Ovarian 235 15154 Cancer:162 compared with other methods such as BSSA, ABC, IG-GA,
Cancer Normal:91
GSE4115 192 22216 Cancer:97
and IG-PSO. Then, the most selected genes of MI-IBGSA is
Normal:95 analyzed by 10-fold cross validation.
GSE10245 58 54676 AC:40
SCC:18

TABLE III. THE PROPOSED APPROACH COMPARED WITH THE NOVEL APPROACHES IN TERMS OF SIX PERFORMANCE METRICS

Accuracy AvgN Sensitivity Specificity Precision Recall

MI-IBGSA 99.71 7.8 98.90 97.0 99 100
ABC 83.63 168.8 92.75 67.43 84.99 92.75
ALL-AML BSSA 83.23 2552.5 98.19 56.35 82.39 98.18
IG-GA 95.91 25.7 98.11 92.68 96.85 98.10
IG-PSO 98.95 24.0 99.10 93.48 98.14 99.28
MI-IBGSA 90.90 11.3 91.35 85.66 91.46 89.45
ABC 69.26 145.4 85.80 39.67 72.32 85.79
Colon Tumor BSSA 75.92 683.8 94.54 38.30 75.15 94.64
IG-GA 82.43 33.85 88.39 68.67 86.58 88.39
IG-PSO 90.63 27.5 92.36 70.19 88.45 89.14
MI-IBGSA 99.64 15.3 100 99.69 99.57 99.65
ABC 98.05 199.8 97.56 98.79 99.38 97.48
Ovarian Cancer BSSA 91.66 5964.8 98.17 80.22 90.06 98.26
IG-GA 98.46 14.03 99.38 96.67 98.28 99.38
IG-PSO 98.11 15.6 99.65 95.67 97.68 98.44
MI-IBGSA 72.92 20 84.56 69.77 72.99 84.31
ABC 58.88 267.8 58.89 59.74 57.88 59.82
GSE4115 BSSA 66.17 8812.2 70.12 62.16 64.48 71.01
IG-GA 65.16 38.6 59.98 70.34 66.91 59.95
IG-PSO 60.65 31.1 55.42 68.61 61.23 58.31
MI-IBGSA 98.33 5 90.0 100 99.95 91.0
ABC 93.06 414.2 86.67 97.15 96.67 86.67
GSE10245 BSSA 88.04 18146.6 78.02 95.57 86.67 78.0
IG-GA 93.23 15.4 86.67 95.76 83.33 86.67
IG-PSO 95.45 13.2 85.69 95.81 90.13 88.28

All experimental results with each dataset are shown in performance on unbalanced data. In terms of Sensitivity, MI-
Table III. It can be seen from Table III, the proposed MI- IBGSA performs optimally on Ovarian Cancer, GSE4115,
IBGSA performs best in precision, AvgN (average number of GSE10245, the proposed methods MI-IBGSA performs best in
features), precision, and recall, which can better reflect the Specificity on ALL-AML, Colon tumor, Ovarian Cancer,

28
GSE10245. For each dataset, in terms of accuracy, MI-IBGSA 210757_x_at, 216858_x_at. Their official gene symbols
is not prominent than other hybrid method IG-GA and IG-PSO correspond to Themis and Kctd11 from the NCBI
respectively, but its advantage are more obvious than wrapper website(https://www.ncbi.nlm.nih.gov/). For Themis, this gene
method ABC and BSSA. For example, in terms of accuracy, encodes a protein that plays a regulatory role in both positive
the proposed MI-IBGSA does not have great advantages, it is and negative T-cell selection during late thymocyte
only 0.76% higher than IG-PSO in ALL-AML and 0.27% development. The protein functions through T-cell antigen
higher than IG-PSO in ColonTumor, but it chooses the least receptor signaling and is necessary for proper lineage
features. The proposed MI-IBGSA is only disadvantage in commitment and maturation of T-cells. Alternative splicing
terms of Sensitivity in ALL-AML and ColonTumor. In terms results in multiple transcript variants. There are 35 references
of Recall in ColonTumor BSSA is higher 5.09% than MI- to the gene on the website, we can see from the literature that
INGSA. The overall performance of the proposed algorithm is THEMIS is required for the pathogenesis of cerebral malaria
better than other algorithms. and protection against pulmonary tuberculosis [30]. For
Kctd11, a novel, developmentally regulated gene that promotes
D. Feature Analysis neural cell differentiation, we can see from the literature that it
In this section, we will further analyze the features selected is related to tumor growth [31][32][33].
by the proposed method. Table IV lists some features identified
by MI-IBGSA from five datasets. After 10-fold cross- Meanwhile, we also proceed with the Go enrichment
validation, these features are selected more than five times by analysis from selected features of ALL-AML, and observed the
the proposed algorithm. These features correspond to the corresponding ID, PValue and gene ontology are shown in
optimal solution with the highest fitness value and the Table V. GO analysis returns a PValue for each GO of a
minimum number of feature subsets. Among them, some genes different gene, if PValue is small, that demonstrates the
contained in these characteristic subgroups represent important enrichment degree of different genes in GO. Fig. 2 represents
potential genes that may be associated with corresponding different modules. For each module, if there are more than 5
diseases. Here, take GSE4115 for example, through our terms, the top 5 with the highest enrich ratio will be displayed.
proposed MI-IBGSA method, we can obtain the gene ID of
TABLE IV. THE BEST SOLUTIONS OBTAINED BY THE MI-IBGSA
Dataset Accuracy Gene ID
ALL-AML 99.71 D88270_at, HG1612-HT1612_at, M11722_at, M22960_at, U05259_rna1_at, M12959_s_at,
M31211_s_at, X97267_rna1_s_at
Colon Tumor 90.90 T61609, T52642, T60155, T52343, H77348, R87126, D14812, T51849, M76378.2, H24030, T86473,
U09587, T68098, H77536, U26312, X62048, T57468
Ovarian Cancer 99.64 MZ2.8234234, MZ417.73207, MZ435.07512
GSE4115 72.92 200986_at, 201384_s_at, 202413_s_at, 203818_s_at, 210411_s_at, 210757_x_at, 212939_at, 213125_at,
213361_at, 213890_x_at, 214130_s_at, 216858_x_at, 216996_s_at, 217405_x_at, 220558_x_at,
222339_x_at
GSE10245 98.33 1555962_at, 216693_x_at, 219507_at, 229105_at
TABLE V. RESULTS OF ALL-AML GO ENRICHMENT ANALYSIS
ID Term PValue Genes
GO:0050853 B cell receptor signaling pathway 0.000438108822969 IGL, CD79A
GO:0005576 extracellular region 0.000816298253755 VPREB1, CSTA, PLTP, IGL
GO:0050900 leukocyte migration 0.00106532469848 VPREB1, IGL

VI. CONCLUSION
MI-IBGSA provides a new method for feature selection of
high-dimensional data sets, which calculates the amount of
information contained in features by filter method mutual
information. The initial strategy not only selects some highly
relevant gene features, but also improves the convergence
ability of the algorithm. The dynamic velocity update formula
improves the local search ability of the algorithm, the adaptive
𝑘𝑏𝑒𝑠𝑡 particle update formula improves the convergence
accuracy, and the fitness sharing improves the global search
ability of the population. The experimental results show that
MI-IBGSA can select a small amount of genetic information
and obtain high classification accuracy. It can better mine the
function of biological data sets in the field of disease diagnosis
Fig. 2. GO enrichment ratio and ameliorate the efficiency of disease diagnosis.

29
 In the future work, MI-IBGSA will be applied to more [15] Subanya B, Rajalaxmi R. A novel feature selection algorithm for heart
biological data sets, and will also be used with more classifiers disease classification[J]. International Journal of Computational
Intelligence and Informatic, 2014, 4(2).
such as Ann, SVM and RF to verify this method. Our research
[16] Sarafrazi S, Nezamabadi-pour H. Facing the classification of binary
provides an idea for hybrid feature selection, we will continue problems with a GSA-SVM hybrid system[J]. Mathematical and
to improve the search performance of MI - IBGSA, for Computer Modelling, 2013, 57(1-2): 270-278.
example, we can attempt in combination with other more [17] Xiang J, Han X H, Duan F, et al. A novel hybrid system for feature
superior mentioned filter methods to improve the efficiency of selection based on an improved gravitational search algorithm and k-NN
the population initialization, screen better classification method[J]. Applied Soft Computing, 2015, 31: 293-307.
performance features, also can be combined with more random [18] Nagpal S, Arora S, Dey S. Feature selection using gravitational search
strategy to improve the performance of BGSA algorithm. algorithm for biomedical data[J]. Procedia Computer Science, 2017, 115:
258-265.
ACKNOWLEDGMENT [19] Hoque N, Singh M, Bhattacharyya D K. EFS-MI: an ensemble feature
selection method for classification[J]. Complex & Intelligent Systems,
This work was supported by the National Natural Science 2018, 4(2): 105-118.
Foundation of China (No. 61802114, 61802113, 61602156, [20] E. Rashedi, et al. Gsa: A gravitational search algorithm. Information
61972134). Science and Technology Development Plan Project Sciences 2009; 179 (13): 2232–2248.
of Henan Province (No 202102210173, 212102210091). China [21] E. Rashedi, et al. BGSA: binary gravitational search algorithm, Natural
Postdoctoral Science Foundation (No. 2020M672212). Henan Computation 2009; 9(3): 1-19.
Province Postdoctoral Research Project Founding. [22] Zhao W. Adaptive image enhancement based on gravitational search
algorithm[J]. Procedia Engineering, 2011, 15: 3288-3292.
REFERENCES [23] Wang J S, Song J D. A Hybrid Algorithm Based on Gravitational Search
[1] Lee K, Man Z, Wang D, et al. Classification of microarray datasets and Particle Swarm Optimization Algorithm to Solve Function
using finite impulse response extreme learning machine for cancer Optimization Problems[J]. Engineering Letters, 2017, 25(1).
diagnosis[J]. Neural Computing & Applications, 2013, 22(3-4):457-468. [24] Güvenç U, Sönmez Y, Duman S, et al. Combined economic and
[2] Hancer E, Xue B, Zhang M. Differential evolution for filter feature emission dispatch olution using gravitational search
selection based on information theory and feature ranking[J]. algorithm[J].Scientia Iranica, 2012, 19(6): 1754-1762.
Knowledge-Based Systems, 2018, 140: 103-119. [25] Cover T M, Thomas J A. Elements of information theory[M]. John
[3] Guyon, A. Elisseeff. An introduction to variable and feature selection. J. Wiley & Sons, 2012.
Mach. Learn. Res 2003; 3: 1157–1182 [26] Lancaster H O, Seneta E. Chi‐square distribution[J]. Encyclopedia of
[4] Martín-Valdivia M T, Díaz-Galiano M C, Montejo-Raez A,et al. Using biostatistics, 2005, 2.
information gain to improve multi-modal information retrieval [27] Li Z H, Zhang T S. Research of fitness sharing niche genetic algorithms
systems[J]. Information Processing & Management, 2008, 44(3):1146- based on extension clustering[J]. Journal of Harbin Institute of
1158. Technology, 2016, 48(5): 178-180.
[5] Kononenko I. Estimating attributes: analysis and extensions of [28] Shukran M A M, Chung Y Y, Yeh W C, et al. Artificial bee colony
RELIEF[C]//European conference on machine learning. Springer, Berlin, based data mining algorithms for classification tasks[J]. Modern Applied
Heidelberg, 1994: 171-+182. Science, 2011, 5(4): 217.
[6] Peng H, Long F, Ding C. Feature selection based on mutual information [29] Ni B, Liu J. A hybrid filter/wrapper gene selection method for
criteria of max-dependency, maxrelevance, and min-redundancy[J]. microarray classification[C]//Proceedings of 2004 International
IEEE Transactions on pattern analysis and machine intelligence, 2005, Conference on Machine Learning and Cybernetics (IEEE Cat. No.
27(8): 1226-1238. 04EX826). IEEE, 2004, 4: 2537-2542.
[7] Jinjie Huang,Yunze Cai,Xiaoming Xu. A hybrid genetic algorithm for [30] Torre S, Faucher SP, Fodil N, Bongfen SE, Berghout J,
feature selection wrapper based on mutual information[J]. Pattern Schwartzentruber JA, Majewski J, Lathrop M, Cooper AM, Vidal SM,
Recognition Letters,2007,28(13). Gros P. THEMIS is required for pathogenesis of cerebral malaria and
[8] Hu Z, Bao Y, Xiong T, et al. Hybrid filter–wrapper feature selection for protection against pulmonary tuberculosis. Infect Immun. 2015
short-term load forecasting[J]. Engineering Applications of Artificial Feb;83(2):759-68.
Intelligence, 2015, 40: 17-27. [31] Argenti B, Gallo R, Di Marcotullio L, Ferretti E, Napolitano M,
[9] T.N Lal, et al. Embedded methods. Stud. Fuzziness Soft Comput. 2006; Canterini S, De Smaele E, Greco A, Fiorenza MT, Maroder M,
207: 137-165. Screpanti I, Alesse E, Gulino A. Hedgehog antagonist REN(KCTD11)
regulates proliferation and apoptosis of developing granule cell
[10] Subanya B, Rajalaxmi R R. Feature selection using Artificial Bee progenitors. J Neurosci. 2005 Sep 7;25(36):8338-46.
Colony for cardiovascular disease classification[C]//2014 International
Conference on Electronics and Communication Systems (ICECS). IEEE, [32] Argenti B, Gallo R, Di Marcotullio L, Ferretti E, Napolitano M,
2014: 1-6. Canterini S, De Smaele E, Greco A, Fiorenza MT, Maroder M,
Screpanti I, Alesse E, Gulino A. Hedgehog antagonist REN(KCTD11)
[11] Faris H, Mafarja M M, Heidari A A, et al. An efficient binary salp regulates proliferation and apoptosis of developing granule cell
swarm algorithm with crossover scheme for feature selection progenitors. J Neurosci. 2005 Sep 7;25(36):8338-46.
problems[J]. Knowledge-Based Systems, 2018, 154: 43-67.
[33] Canettieri G, Di Marcotullio L, Greco A, Coni S, Antonucci L, Infante P,
[12] Salem H, Attiya G, El-Fishawy N. Classification of human cancer Pietrosanti L, De Smaele E, Ferretti E, Miele E, Pelloni M, De Simone
diseases by gene expression profiles[J]. Applied Soft Computing, 2017, G, Pedone EM, Gallinari P, Giorgi A, Steinkühler C, Vitagliano L,
50: 124-134. Pedone C, Schinin ME, Screpanti I, Gulino A. Histone deacetylase and
[13] Sahu B. A combo feature selection method (filter+ wrapper) for Cullin3-REN(KCTD11) ubiquitin ligase interplay regulates Hedgehog
microarray gene classification[J]. International Journal of Pure and signalling through Gli acetylation. Nat Cell Biol. 2010 Feb;12(2):132-42.
Applied Mathematics, 2018, 118(16): 389-401.
[14] Jain I, Jain V K, Jain R. Correlation feature selection based improved-
binary particle swarm optimization for gene selection and cancer
classification[J]. Applied Soft Computing, 2018, 62: 203-215.

30
 2021 the 13th International Conference on Computer and Automation Engineering

Comparative Study of Modeling Road-Vehicle

Dynamic Behaviour Using Different Data
Based Techniques
Youssef M. Fathy Amr Hassan Mohammad Tawfik
Mechatronics department Mechatronics department Academy of Knowledge
German university in cairo German university in cairo Cairo, Egypt
Cairo, Egypt Cairo, Egypt [email protected]
[email protected] [email protected]

Hesham H. Ibrahim
Mechatronics department
German university in cairo
Cairo, Egypt
[email protected]

Abstract—The current work provides data based modeling of by using different types of road profiles as excitation to the
passive automotive suspension dynamics of a half-car model, system. Soliman et al. [4] presented a theoretical evaluation of
by implementing different machine learning techniques, which the active suspension performance, in terms of ride comfort,
model the passive dynamics. The passive vertical and pitch
suspension dynamics, using vector auto-regressive with exogenous of a seven-degrees-of-freedom full car model using the fuzzy-
input (VARX), recurrent neural network (RNN), and artificial skyhook controller. Their simulation results were introduced
neural network (ANN), and their accuracy will be compared. in time and frequency domains and it was shown that the
The training data is obtained from a half car model simulated ride comfort could be significantly enhanced by the proposed
by Matlab/Simulink which intended to mimic suspension sensors controller. Konoiko et al. [5] developed a feed-forward neural
readings. It is found that the VARX and ANN models showed very
high identification accuracy, added to the superior computational network and automated search technique in order to control
efficiency realized by the VARX technique. an active suspension system. The training data were obtained
Keywords—Half car model , Vector Auto-Regressive, Deep using an optimal PID controller, and the network was trained
learning using a back propagation algorithm. The trained network was
experimentally implemented in a test rig for a quarter-car
I. I NTRODUCTION model, and it was found that the developed deep network
Suspension systems are intended to isolate the vehicle’s outperforms the optimal PID controller. Hanafi and Rahmat
occupants from the strong vibrations induced by road surface [6] presented an identification technique for the suspension
irregularities, alongside with maintaining the contact between dynamics, using a Hammerstein passive quarter car model,
the wheels and the road, which provide the vehicle’s comfort by implementing a multilayer perception neural networks.
and stability. Accurate identification of suspension dynamic Training data was acquired by driving a car on a special
properties is crucial in order to avoid the sup-optimal control road event, and the network learning algorithm was based
of such systems. Machine learning techniques proved to be a on Fisher’s scoring method. Hanafi et al. [7] identified a
very useful modeling tool in the automotive industry. In partic- quarter car model based on real test data using autoregression
ular, it could be a very useful tool in the correct identification exogenous model (ARX). Real car data set were used in
of suspension dynamics, which leads to optimum suspension the model training process. Linear least-square estimation
controllers design as well as suspension health monitoring. The algorithms were used to identify ARX coefficients. The sprung
paper organization is divided into state of art , methodology mass vertical acceleration was the output variable and tire
and results. Many machine learning applications, in automo- vertical acceleration was the input variable. The results showed
tive modeling, could be found in the literature such as vector that the identified model is acceptable and the ARX can be
autoregression with exogenous input, artificial neural network used for analyzing and designing the suspension controller.
(ANN), recurrent neural network, deep learning algorithms, Finally, the input order of the ARX model was two, the time
etc.Darus and Sam [1] investigated the active performance of delay was one and The output order of the model was four.
a suspension system using linear, seven degrees of freedom, Pourqorban et al. [8] Proposed nonlinear optimal control of
full car model, and Linear Quadratic Regulator (LQR) control. an active suspension system. The model was an active half-
All simulations are performed using Matlab [2]/Simulink [3] car model. Taylor series expansion was used for predicting the

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 31

states. Minimization of local difference between the desired will be represented as follow
states and the predicted states was used for introducing the  
a1 a2
a3 a4 a5 a6 a7 a8
control law. The results showed that the state tracking error 0 10 0 0 0 0 0
was minimized using the control law. Gogaa etal siven focused 
 b3

b4
b1 b2 b5 b6 b7 b8 
on the optimization of the suspension system parameters 
0

01 0 0 0 0 0
using genetic algorithms. Half a car mathematical model was A=  c3
 (2)
c4
c5 c6 c1 c2 c7 c8 
used and the optimization performed on Matlab/Simulink. The 
0

00 0 1 0 0 0
simulations showed that the optimized parameters have faster 
d3

d4
d5 d6 d7 d8 d1 d2 
stabilization which it compared with the model that has the
0 00 0 0 0 1 0
original parameters. In this paper a four degrees of freedom  
passive suspension system will be identified using recurrent 0 0 b9 0 0 0 0 0
transpose(B) = (3)
neural network (RNN) , artificial neural network (ANN) and 0 0 0 0 c9 0 0 0
vector autoregressive with exogenous input (VARX) based on So by the time series analysis the suspension system model
it’s data. This data will be obtained from a Simulink model is represented . The final step is to capture the coefficients
of a passive half car which mimic the sensors reading. And using the machine learning technique which is regression.
the techniques performance will be compared in terms of So illustrating the technique for the sprung mass equation
computational time and accuracy. only and the other equations will be captured using the same
technique. Firstly starting preparing the training A matrix as
follow.  
II. M ETHODOLOGY 1 Xs1 Xs0 Xu11 Xu10 Xu21
1 Xs2 Xs1 Xu12 Xu11 Xu22 
A(1) =  .
 
. . . . ..
 .. .. .. .. .. 
. 
In this section the methodology of identifying the sus-
1 Xsn−1 Xsn−2 Xu1n−1 Xu1n−2 Xu2n−1
pension system based on suspension system data will be
(4)
introduced. The techniques that will be used in this paper are 
Xu20 Θ1 Θ0

VARX,RNN and ANN. Firstly a SIMULINK model of half car  Xu21 Θ2 Θ1 
is made in order to mimic the sensors data that’s needed for the A(2) = 

.. .. .. 

(5)
identification process. The model is build based on the passive  . . . 
half car equations of motion and the output data that will be Xu2n−2 Θn−1 Θn−2
used from this model in the machine learning techniques are A matrix is the concatenation between A(1) and A(2) which is
the sprung mass displacement (Xs) , the front unsprung mass the matrix used for training the coefficients. Then preparing the
displacement (Xu1) , the rear unsprung mass displacement matrix for training the output data by the following equation
(Xu2) and finally the pitch angle (Θ). The inputs data are the  
front road disturbance and the rear road disturbance. Our firstly Xs2
technique is an integration between the time series analysis  Xs3 
Xst+δ =  .  (6)
 
and the regression technique which is vector auto-regressive  .. 
with exogenous input (VARX). Each state in the suspension Xsn
system can be represented by an equation of it’s history and the
history of all other states. The number of the time steps of the So now using regression the coefficients that will map the input
history is a tuning parameter. So after many experiments and values and output values will be captured by the following
comparison of the results using RMSE. The model that fit the equation
half car model is represented using the following equations. 
a = a0 a1 a2 a3 a4 a5 a6 a7 a8 a9

(7)
   T   −1   t  
    a = ( A ∗ A ) ∗ A ∗ Xst+δ (8)
Xs(t) Xs(t − 1)
 Xs(t − 1)   Xs(t − 2)  By the same way the b,c,d vectors coefficients can be captured.
   
 Xu1(t)  Xu1(t − 1) An artificial neural network is considered as black-box used to
   
Xu1(t − 1) model dynamical systems, Artificial neural network is a group
 = A∗Xu1(t − 2)+B∗ Xr1t−1
   

 Xu2(t)  Xu2(t − 1) Xr2t−2 of neurons that is able to learn complex non-linear function,
   
Xu2(t − 1) Xu2(t − 2) This neuron is combined together to make different layers like
   
 Θ(t)   Θ(t − 1)  input, output, and hidden layers. A neural network is usually
Θ(t − 1) Θ(t − 2) used to identify a static system that does not depend on the
(1) history of the system. A modified version of the neural network
Where Xr1 and Xr2 are the front road disturbance and the that will be used to identify the suspension system will use
rear road disturbance respectively. A and B matrices are the previous inputs and outputs which defines the dynamics of
unknown matrices that will be captured by the regression and the suspension system to predict the future values of the

32
outputs, Actually, a feed-forward neural network with three unsprung mass displacement and pitch angle (Θ). The time
layers and non-linear activation function in the hidden layer steps that fit the passive half-car suspension system are two
with the suitable number of neurons can be able to identify any for the states and one for the front road excitation and the rear
input-output relation [9]. There are many tuning parameters road excitation. For the ANN in system identification,The
for Artificial neural networks which are: number of hidden
layers, number of neurons in each hidden layers, Activation
function in hidden and output layers, Training algorithm that
used to modify weights during the training phase, Number
of delayed inputs, and outputs, And the number of epochs
used in training which represents the number of passes that
the machine learning algorithm has completed for the whole
data set used in training. All the above-mentioned parameters
are stunned using many experiments and the results of the
experiments are compared using RMSE. MATLAB neural
networks toolbox will be used for simulations and training
of the Artificial neural network. Long short term memory
(LSTM) is a development of the well known deep learning
technique which is a recurrent neural network (RNN) which
helps in solving some of RNN limitations as the vanishing
gradient problem with changing the structure of the neuron. Fig. 1. Sprung mass displacement using VARX
The LSTM is depending on time series analysis which helps
in dynamic capturing. There are many hyper-parameters for
LSTM which are identified based on many experiments and
the results of the experiments is compared together using
RMSE until the optimal values are reached. The training input
consists of sprung mass displacement, front unsprung mass
displacement, rear unsprung mass displacement, front road
input, rear road input, and pitch angle. The training outputs
consists of sprung mass displacement, front and rear unsprung
mass displacements, and pitch angle. ADAM [10] optimizer
was the optimizer that used to train the network weights that
fit the model. The same data-set that was used in ANN and
VARX after being normalized into a range between zero and
one was used for training and validating the LSTM.
III. R ESULTS AND D ISCUSSION
In this section, a comparative study between different ma-
Fig. 2. Front unsprung mass displacement using VARX
chine learning techniques in system identification of passive
half-car model with parameters shown in table I will be done.
The models will be compared using root mean square error
between SIMULINK values and the predicted values from
each technique to measure the accuracy of each model in
identifying the suspension system.

TABLE I.
H ALF CAR PARAMETERS

No. Parameter Value Unit

1 Ms 580 kg
2 mu1 , mu2 40 kg
4 kt1 , kt2 190000 N/m
6 ks1 , ks2 235 N/m
7 cs1 , cs2 70,80 N s/m
8 Is 1100 kgm2
Fig. 3. Rear unsprung mass displacement using VARX
Figures 1,2,3,4 show a comparison between the predicted
(VARX) and the actual value (SIMULINK) of the sprung best architecture that fitted the model for passive half car
mass displacement, front unsprung mass displacement , rear identification is chosen to have 1 hidden layer with twenty-five

33
 Fig. 7. Rear Wheel Displacement using ANN

Fig. 4. Pitch angle (Θ) using VARX

hidden neurons with sigmoid activation function in the hidden

layer and linear activation function in the output layer. The
maximum epochs for training are 1000. Two previous steps of
inputs and outputs were chosen to predict the next outputs. The
model is trained using levenberg-marquardt training algorithm
on MATLAB. Figures 5,6,7 and 8 show a comparison between
the predicted (ANN) and the actual value (SIMULINK) of the
sprung mass displacement, front unsprung mass displacement ,
rear unsprung mass displacement and pitch angle (Θ).

Fig. 8. Pitch Angle (Θ) using ANN

LSTM was used for identifying the passive half car suspen-
sion system by using 1 hidden layer with 200 neuron and 20
% drop-out regularization which prevented over-fitting,linear
activation function for the output layer , two time steps ,32
batches and 100 number of epochs. Figures 9,10,11 and 12
are showing a comparison between the predicted (RNN) and
the actual value (SIMULINK) of the sprung mass displace-
ment, front unsprung mass displacement , rear unsprung mass
displacement and pitch angle (Θ).

Fig. 5. Sprung Mass Displacement using ANN

Fig. 6. Front Wheel Displacement using ANN Fig. 9. Sprung mass displacement identification using RNN

34
 is showed the best accuracy. RNN showed the highest root
mean square error in all the states which means the worst
accuracy. Thus VARX and ANN are better than RNN in
system identification of the suspension system.

TABLE II.
ROOT MEAN SQUARE ERROR

State VARX ANN RNN

Xs 0.0017 0.0039 0.007
Xu1 0.0014 0.0015 0.005
Xu2 9.4188e-04 9.9367e-04 0.025
Θ 8.48e-04 0.0028 0.029

IV. C ONCLUSION
Fig. 10. Front wheel identification RNN The work presented data based modeling of passive au-
tomotive suspension dynamics of a half car, by implement-
ing different machine learning techniques, which model the
passive dynamics . The passive vertical and pitch suspension
dynamics, using (VARX), recurrent neural network (RNN),
and artificial neural network (ANN), and their accuracy will
be compared. The training data is obtained from a half car
model simulated by Matlab/Simulink which intended to mimic
suspension sensors readings. VARX showed the best accuracy
and lowest computational time, as a system identification tool
of passive half-car models, compared with ANN and RNN.
RNN showed the worst accuracy and highest computational
time in training.
R EFERENCES
[1] R. Darus and Y. M. Sam, “Modeling and control active suspension
system for a full car model,” in 2009 5th International Colloquium on
Fig. 11. Rear wheel identification using RNN
Signal Processing & Its Applications. IEEE, 2009, pp. 13–18.
[2] MATLAB, R2019b. Natick, Massachusetts: The MathWorks Inc., 2019.
[3] S. Documentation, “Simulation and model-based design,” 2019. [On-
line]. Available: https://www.mathworks.com/products/simulink.html
[4] M. Soliman, M. M. Kaldas, D. C. Barton, and P. C. Brooks, “Fuzzy-
skyhook control for active suspension systems applied to a full vehicle
model,” International Journal of Engineering and Technology Innova-
tion, vol. 2, no. 2, p. 85, 2012.
[5] A. Konoiko, A. Kadhem, I. Saiful, N. Ghorbanian, Y. Zweiri, and
M. N. Sahinkaya, “Deep learning framework for controlling an active
suspension system,” Journal of Vibration and Control, vol. 25, no. 17,
pp. 2316–2329, 2019.
[6] D. Hanafi and M. F. Rahmat, “System identification of hammerstein
model a quarter car passive suspension systems using multilayer per-
ceptron neural networks (mpnn),” Jurnal Teknologi, vol. 43, no. 1, pp.
95–109, 2005.
[7] D. Hanafi, M. S. Huq, M. S. Suid, and M. F. Rahmat, “A quarter
car arx model identification based on real car test data,” Journal
of Telecommunication, Electronic and Computer Engineering (JTEC),
vol. 9, no. 2-5, pp. 135–138, 2017.
[8] A. Kashtiban, N. Pourqorban, G. Alizadeh, and I. Hasanzadeh, “Non-
linear optimal control of a half car active suspension,” in 2009 Second
Fig. 12. Pitch angle identification using RNN International Conference on Computer and Electrical Engineering,
vol. 2. IEEE, 2009, pp. 460–464.
[9] C. Wong, J. Shippen, and B. Jones, “Neural network
Regarding the results, The VARX showed the lowest com- control strategies for low specification servo actuators,”
putational time. The recurrent showed the highest. Table II International Journal of Machine Tools and Manufacture,
show a root mean square error comparison between the three vol. 38, no. 9, pp. 1109 – 1124, 1998. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S0890695597000679
machine learning technique. The VARX showed the lowest [10] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
root mean square error in all the states which means that arXiv preprint arXiv:1412.6980, 2014.

35
 2021 the 13th International Conference on Computer and Automation Engineering

User Naming Conventions Mapping Learning for

Social Network Alignment
Zhao Yuan Liu Yan Guo Xiaoyu
Zhongyuan Network Security State Key Laboratory of State Key Laboratory of
Research Institute Mathematical Engineering and Mathematical Engineering and
Zhengzhou University Advanced Computing Advanced Computing
Zhengzhou, 450001, China PLA Strategic Support Force PLA Strategic Support Force
[email protected] Information Engineering Information Engineering
University University
Zhengzhou, 450001, China Zhengzhou, 450001, China
[email protected] [email protected]

Sun Xian Wang Sen

State Key Laboratory of Teaching Archive Office of
Mathematical Engineering and Teaching and Research
Advanced Computing Guarantee Center
PLA Strategic Support Force PLA Strategic Support Force
Information Engineering Information Engineering
University University
Zhengzhou, 450001, China Zhengzhou, 450001, China
[email protected] [email protected]

Abstract—The existing research on social network alignment in multiple social networks is the main purpose of cross-social
using usernames is mainly based on the similarity between network user social network alignment, which is also called
usernames calculated by different classifiers. However, if the social network node alignment. Social network node alignment
number of available annotations and training time are limited refers to finding corresponding users among multiple input
and feature extraction is incomplete, the accuracy of social social networks, and these corresponding users in different
network alignment would have been be reduced. Based on the social networks correspond to the same natural person in the
above, this paper proposes a BP neural network mapping for real world.
social network alignment (BSNA). The BP neural network is
used to realize the mapping between two social network user Social network alignment is the foundation of many
name vectors, and the classification problem is transformed into cross-social network applications and the premise of deep data
a mapping problem between vectors. The experimental results on mining across different platforms. People record different
several social network data sets show that compared with the information in different social networks, including personal
benchmark method, the social network alignment precision of information (self-description, location, hobbies, etc.), close
the proposed model is improved by 4%, and the experiments relationships, shared links, check-in places, latest status and so
with smaller training set ratio and less training time have higher on. Together, this information constitutes a complete image of
precision and faster convergence than the benchmark method. users on the Internet. By aligning multiple accounts of the
same user in different networks, data fusion can be recognized,
Keywords—Social network alignment, BP neural network
mapping, user naming conventions
and personal information of users can be collected, integrated
and improved to the maximum extent, so that massive social
I. INTRODUCTION metadata of users can be fully mined, which has very important
practical significance in practical applications such as
Nowadays, different social networks provide various information retrieval, cross-platform recommendation system
services such as communication, entertainment, study, and network security.
shopping and work. People can join different social networks
according to the needs of work and life, with an average of 8.5 The existing algorithm solves the problem of user social
social media accounts [1] each person. However, because the network alignment to a certain extent by using user's public
single sign-on technology is not widely used at present, attribute information [2-7] or network structure information
multiple accounts registered by the same user on different [8-11]. Attribute-based methods are simple and have good
social networks are often isolated, and the information of users results in certain scenes. However, most of the user attribute
in different networks is rarely communicated with each other, information relied on in these methods is user self-reported
and there is also a lack of a user identity that can be associated attribute. For privacy protection, this part of information is
with each social account. With the absence of a unified user usually unavailable, noisy and not particularly reliable. In
identity, associating accounts belonging to the same real user addition, when there are great differences in attributes and few

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 36

common attributes in social networks, the applicable scope of considers neighborhood structure and community structure at
such methods is limited. However, the method based on the same time in the process of learning user node vector
network structure doesn't put the regularity of network representation, and preserves the structural information of the
structure in to its calculation, which leads to the problems such original network as much as possible; Then, BP neural network
as being extremely sensitive to the noise and changes of is used to learn the nonlinear mapping function between the
network structure, having non-enough robustness, and leading source network and the target network, so as to realize the user
to inaccurate modeling of the relationship between users across social network alignment.
networks. The method of using username can solve these
problems to a certain extent. Inspired by the above work, in order to solve the
dependence of training set ratio and training times in general
Zafarani et al. [12] found that 59% of users use the same classification problems, this paper attempts to use usernames
username on multiple social networks, and the username is and introduce a BP neural network mapping (BSNA) to realize
usually available and highly accessible. From a computational social network alignment.
point of view, it is more convenient to compare the username
with rich online profile attributes or activities of two accounts, The main contributions of this paper are summarized as
so it provides a new idea for social network alignment, which follows:
has been verified by existing methods. Li et al. [13] first  Using BP neural network mapping learning model,
analyzed the username naming patterns of users in different this paper studies the new problem of social network
social networks, then measured and analyzed redundant alignment based on supervised neural network from
information in three ways: length similarity, character the perspective of username.
similarity and letter distribution similarity, and compared with
the display names of different individuals, and finally analyzed  An objective function is designed to utilize available
the evolution of redundant information with time. Then, a anchor user annotations, which map the points of the
cross-social network user identification model based on source network identifiers to the positions close to
username and display name (UISN-UD) [14] is proposed, their corresponding target identifiers, thus minimizing
which contains abundant information redundancy. Liu et al. the distance between two network username feature
[15] used username similarity to identify some matching users vectors.
as training data, then used the training data to train Support  Compared with the benchmark method, the accuracy
Vector Machine (SVM) [16], and used the trained SVM to get of the model is improved by 4%. Experimental results
the matching probability of account pairs, and finally weighted on synthetic data sets also show the superior
the matching probability and username similarity to get the performance of the proposed model.
account pair similarity. Zafarani et al. [12] and Liu et al. [17]
only use usernames to match user accounts in social networks, The structure of this paper is as follows: In the second
and deeply analyze the limitations of environment, personality section, the main ideas of comparison method are introduced.
and even human beings. Zafarani et al. [12] proposed the In the third section, the concept is expounded, and the social
MOBIU model (Modeling Behavior for Identifying Users network alignment problem based on username BP neural
Across Sites). Extract 414 features based on usernames, network mapping learning is defined. The fourth section
including length possibilities, typing patterns, and string introduces the method and model of this paper. The fifth
similarity. Then L1 regularized Logistic regression model is section is the experimental analysis. Finally, this paper is
used to train the classifier. This method can achieve good summarized in the sixth section.
results when selecting candidate users. The most obvious
advantage of this method is that it not only protects personal II. RELATED WORK
privacy, but also is easy to access. Nowadays, most methods Many methods of using usernames have been verified, and
based on usernames need a large number of manually marked the following mainly introduces one of the benchmark methods
samples to train classifiers, so as to separate matching identity in this paper. Li et al. [14] proposed a cross-social network
pairs from unmatched identity pairs, which increases the user identification model UISN-UD to solve the problem that
difficulty of obtaining data. Therefore, finding new ideas in the username is a digital string automatically assigned by
classification is an urgent problem to be solved. websites in some social networking sites, such as QQ and
Man et al. [18] put forward PALE model, which captures Foursquare. The specific implementation framework of this
the potential features of known links through network paper is divided into the following four steps: feature
embedding technology, and obtains a mapping function of extraction, basic classifier construction, fusion classifier
predicted links across social networks by training neural construction and one-to-one constraint implementation.
networks. Zhou et al. [19] proposed a user identification Firstly, using redundant information in usernames and
algorithm DeepLink based on deep neural network, which uses similar names, the username is considered as a group of letters,
network representation learning to map the network structure characters or numbers. The display name is a set of words. 70
to low-dimensional vector space, and trains deep neural features are extracted from each username and display name.
network through dual learning to realize identification. Guo et After feature extraction, a piece of training data
al. [20] put forward a user identification algorithm based on  X mk
1 ,X2 ,y
mk mk  is constructed, to train supervised classifiers.
social structure representation learning, which links Secondly, n supervised classifiers are selected and based on the
community structure with user identification problem, labeled data  X mk
1 ,X2 ,y
mk mk  . And train them, N trained basic

37
 User Set VS

Username

Feature Vectors
Feature Vectors
NS

... ...
Anchor User X

... ...
US UT
Username

Social Network
Alignment
BP Neural Network
NT

User Set VT

Fig. 1. BSNA model framework

classifiers are obtained. Based on these basic classifiers, for a users. N represents a set of usernames, and each user vi by 𝑑
pair of user accounts (vm , vk ) , n classification results can be dimension feature vector 𝑢𝑖 representation, which forms a
obtained. Then, based on the results of the basic classifier, a feature matrix U  R d n .
new training data set is obtained and the fusion score is trained.
Finally, if the user account with the highest probability is The problem studied in this paper is defined as follows:
matched on the result of the fusion classifier, some 1-n or n-n Given two partially aligned social network account sets
relations will appear in the matching result. In order to avoid S  {VS , NS ,U S } , T  {VT , NT ,UT } , and a subset of known
this situation, a one-to-one constraint is added in user
identification. anchor user pairs X  {(vsx , vtx ) | vsx VS , vtx VT } , there is a
mapping relationship of social network accounts. The purpose
The main innovations of this paper are as follows: (1) of this paper is to find all other identity pairs with mapping
Provide cross-social network user identification solutions relationship Y  {(vs , vt ) | vs VS , vt VT ,(vs , vt )  X } , in
according to usernames and display names. (2) A two-stage
supervised learning classifier is proposed to solve the problem which vs and vt belong to the same natural person.
of user identification. (3) Apply one-to-one constraint to the In order to solve the above research problems, this paper
recognition result to optimize the final matching result. uses neural network to transform the classification problem in
However, this method still has the following limitations: (1) existing research into vector mapping, which can be used to
Both the basic classifier and the fusion classifier have great learn the mapping function of social network alignment. Given
dependence on the number of labeled data, the ratio of positive the source network username feature vector us , the target
and negative examples and the training times. (2) In the aspect network feature vector is ut . And anchor user pairs
of feature extraction, features related to common substring,
common subsequence and editing distance are extracted from X  {(vsx , vtx ) | vsx VS , vtx VT } , the mapping point is defined
each username/display name, but the feature extraction is not as: f (u ) . The problem studied can be understood as a
comprehensive enough to fully characterize the user's naming function f . The goal of social network alignment is to learn a
habits. Therefore, the text only uses the username, extracts
mapping function that meets the following two conditions f :
more comprehensive features, and achieves social network
alignment through the method of neural network mapping. (1) f should be possible to minimize the source network
III. PROBLEM DEFINITION mapping node vector f (us ) and the target network node
Definition 1: User and account number. Users are users of vector 𝑢𝑏 the distance between the two vectors can be as
social network applications and individual entities living in the close as possible; (2) For the matching identity pairs (vsx , vtx )
real world. Account number is the carrier used by users to in X , f should be make the mapped source network
express their identities when using social networking
applications. username feature vector f (usx ) and target network username
feature vector usx . The distance between them is the smallest.
Definition 2: Anchor user. Users who participate in two
social networks at the same time are defined as anchor users After the training process, a given source network user vs ,
(or anchor nodes), while other users are non-anchor users according to the target network to match users vt ground
(non-anchor nodes). distance of d ( f (us ), ut ) select its candidate. The smaller
Given triplets O  {V , N ,U } to represent a username in a ground distance means that two identities have a greater
social network, where V  {v , v , , vn } is a user set with n chance of becoming the same natural person.

38
 2 6Top-k
x x
us 1
ut 1
x x
us 1 3 ut
2

...
Loss 2

... ... ...

x US f U S  x
us i
ut i

usm 5 4 utm
... ...
usn utn
Reward

Fig. 2. BP Neural Network Mapping Model

IV. MODEL AND FRAMEWORK the distance to see how many times it takes to change
The existing research on social network alignment of social one string into another; Jaccard distance, the
network usernames is mainly based on the similarity between similarity between two samples is measured by the
usernames calculated by different classifiers. However, when proportion of different elements in two sets.
the number of available annotations and training times are Based on the above thirteen types of features, each type of
limited, this may lead to poor performance, and incomplete feature is represented by several-dimensional vectors, and
feature extraction will also reduce the alignment accuracy. finally the social network username will be transformed into a
Therefore, according to the naming habits of users, the BP 358-dimensional model input vector.
neural network mapping learning model is introduced to
achieve social network alignment. B. BP Neural Network Mapping Model
The basic idea of BP algorithm is that the learning process
The main method is shown in Fig. 1. The first step is to
consists of two processes: forward propagation of signals and
obtain data from heterogeneous social networks and extract backward propagation of errors. In forward propagation, the
usernames; The second step is data representation, which input samples are introduced from the input layer, processed
represents the social network username as a feature vector; The by each hidden layer, and then transmitted to the output layer.
third step is to use BP neural network to learn vector mapping; If the actual output of the output layer does not meet the
Finally, the alignment between two network nodes is realized requirements, the error is transferred to the backward
and applied. propagation stage. Error back propagation is to propagate the
A. Characterization of Social Network Usernames output error layer by layer from hidden layer to input layer in a
In this paper, the user naming patterns analyzed by certain form, and distribute the error to all units in each layer,
Zafarani et al. [12] are adopted, and 358-dimensional vectors so as to obtain the error signal of each layer unit, which serves
are extracted from three major aspects: human limitations, as the basis for correcting each unit. The weight adjustment
exogenous factors and endogenous factors to characterize these process of each layer of signal forward propagation and error
patterns, and the social network usernames are vectorized. backward propagation is carried out repeatedly. The process of
Specific features are as follows: constant adjustment of weights is the learning and training
process of network. This process is carried out until the error of
 Human limitations: The same username; The length network output is reduced to an acceptable level or until the
of username; The possibility of creating a unique preset learning times.
username.
Among them, error back propagation [21] is the most
 Exogenous factors: The proportion of typed letters successful neural network learning algorithm so far. When
and previous letters typed by the same hand; The neural networks are used in practical tasks, BP algorithm is
proportion of typed letters and previous letters typed mostly used for training [22]. In order to solve the problem of
with the same finger; The percentage of keys entered social network alignment, the weights can be adjusted
by hand with each finger; Percentage of keys pressed repeatedly, and the mapping function needed in this paper can
on rows. be obtained by learning. After obtaining the node vector
representation space of each social network, BSNA uses BP
 Endogenous factor: Letter distribution; Calculating algorithm to train the multi-layer perceptron in a supervised
the information entropy of username letter distribution way, aiming at obtaining the nonlinear mapping function
as a measure of their randomness; The longest between the two vector spaces f.
common substring; Normalized longest common
subsequence length for detecting abbreviations; Edit

39
 Given any pair of anchor nodes (vsx , vtx ) and their vector is crawled to obtain the associated accounts (or other valid
information) of all other social networking sites. The social
representations (usx , utx ) , by the first node vector usx will
network accounts we obtained are shown in Table I.
mapped to another vector space through f (usx ) , the purpose
of this paper is to minimize two vectors f (usx ) and utx the TABLE I. SOCIAL NETWORK DATA SET

distance between them. In order to calculate the distance Dataset Source Network Target Network X
between vectors, there are many ways, such as Euclidean
distance, Chebyshev distance and cosine distance. In this paper, Twitter-Instagram Twitter (1856) Instagram (1856) 1856
cosine distance is used to formally express the loss function as Facebook-Instagram Facebook (1857) Instagram (1857) 1857
follows:
Twitter-Facebook Twitter (4014) Facebook (4014) 4014
u , u   1  cos  f u  , u 
x
s
x
t
x
s
x
t (1) Twitter-Google Twitter (10574) Google (10574) 10574

where cos() represents the cosine similarity between two B. Evaluating Indicator
vectors with a range of [-1, +1]. The larger the value, the Many scholars have proposed many excellent social
smaller the difference between the two vectors. network algorithms for different application scenarios in order
The set of known anchor users is X , the sub-vector space to solve the problem of social network alignment across social
networks. In order to measure the performance of different
composed of anchor nodes is used respectively U sx  R d  X algorithms, it is necessary to use a unified evaluation index. In
and Utx  R d  X represents, where X is the number of this paper, the social network alignment problem is
transformed into a similarity recommendation problem, which
anchor node pairs, the objective function of the mapping
learning process can be formally expressed as: is selected Precision @ k ( P @ k ) [24] as an evaluation index.
Precision @ k ( P @ k ) index is a commonly used evaluation
U x
s   
,U tx   arg min 1  cos f U sx  ,U tx ; W,b
W ,b
 (2) index in information retrieval field, recommendation system
and ranking problem. In this paper, it is defined as follows:
where 𝐖, 𝐛 are weight parameters and bias parameters n
obtained after training neural network by back propagation Precision @ k   li success @ k / n (3)
algorithm. i

Model training. The known anchor node pairs are taken as where n indicates the number of anchor nodes in the test set,
supervision information, and a random gradient descent li success @ k is used to judge whether the node
algorithm is adopted [23] Minimize the loss function and learn
the mapping function f , as step 1, 2, 3, 4 shown in Fig. 2. corresponding to node vi in the source network in the target
network appears in top  k , if it appears in top  k , it is 1,
Constructing a non-anchor node top  k . Aiming at the otherwise it is 0.
non-anchor nodes in the source network vs . First, express it as
C. Benchmark Method and Parameter Setting
a vector f (us ) input into the trained BP neural network
Compare the model with the following classical methods
model to get the mapping vector f (us ) ; Then find the k
 MOBIUS [12], a method of analyzing the naming
most similar nodes to f (us ) in the target network, that is the
habits of social network users and modeling based on
top  k of node vs , as step 5, 6 shown in Fig. 2. the behavior patterns displayed by users on social
media and rooted in behavioral theories of sociology
V. EXPERIMENT and psychology.
A. Data Set  UISN-UD [14], a social network user identification
Users can list their other identities (user accounts) on scheme based on username and display name, trains a
different websites. This can be regarded as the marking data of two-stage classification framework, and solves the
model learning task, which provides mapping between user identification problem according to the extracted
identities. For example, use Instagram to visit URL, features.
http://Instagram.com/user/ID to obtain its public personal data. The parameters of the benchmark method are set as in the
Instagram provides users with cross-site linking function. The original text. For the algorithm BSNA proposed in this paper, a
user can connect his Instagram account to Facebook and four-layer neural network (two hidden layers) is adopted, and a
Twitter [4]. This makes it possible to access the user's back propagation algorithm is applied to train the nonlinear
username on Instagram/Twitter/Facebook. Firstly, select a user mapping function between the source network and the target
account, and obtain the Followers account and Following network. The dimensions of the two hidden layers are set to
account of the account; Then do the same operation on the 500 and 800, respectively. The dimensions of the input layer
obtained account to obtain as much user account information and the output layer are consistent with the dimensions
as possible; Finally, the homepage information of each account

40
represented by the node vector, with a learning rate of 0.001
and a batch size of 16. 1
0.9
D. Experimental Results and Analysis
0.8
(1) Algorithm performance analysis 0.7

precision@1
In this paper, the importance of different features is 0.6
analyzed. As shown in Figure 3, individual intrinsic factors can 0.5
better extract key information from usernames. In experiments 0.4
on the Twitter-Facebook dataset based only on these features, 0.3
MOBIUS
Precision @ still has 95.9%. 0.2
UISN-UD
0.1 BSNA
In order to get the best experimental results, firstly, we 0
evaluate the influence of several basic parameters on the 1000 10000 100000 1000000 10000000
experimental results, such as the ratio of anchor nodes used for iteration i
training, and the number of training iterations. First, select (b) Performance to iteration i
Twitter-Facebook data set, change one parameter at a time, and Fig. 4. Algorithm performance on Twitter-Facebook dataset
keep the other parameter unchanged. Figs. 4 (a) and 4 (b) show
the variation of experimental results with experimental For social network alignment algorithm, the ratio r of
parameters. anchor nodes used for training is an important parameter. As
shown in Fig. 4(a), with the increase of r from 5% to 90%,
the performance of BSNA algorithm is obviously superior to
other benchmark methods. Even when the proportion of anchor
nodes used for training is only 0.1 or 0.2, BSNA algorithm
performs very well. It can be seen from Fig. 4(b) that the
performance of each algorithm varies with the number of
iterations. Compared with the other two benchmark methods,
BSNA algorithm is not only more accurate, but also faster in
convergence.
(2) Experimental analysis of real data sets
Based on the performance analysis, the experimental
parameters are set as follows: the ratio of anchor nodes used
for training r is 0.8, and the number of training iterations i is
100,000, respectively Precision @ to compare the
performance of the algorithm, as shown in Table II.
Through the experimental results, it can be clearly seen that
the performance of this method is better than that of the
benchmark method on several data sets, and the best results in
Fig. 3. Feature Importance Analysis comparison with the benchmark method are all about four
percentage points higher. But in the Twitter-Google dataset,
Precision @ . Compared with other datasets, the value of is
1 slightly reduced, which may be mainly due to the existence of
0.9 languages other than English in Twitter-Google dataset, which
0.8 has some influence on the experimental results.
0.7
In order to analyze the reasons for the different accuracy of
precision@1

0.6
the above data sets, according to the different components of
0.5
social network usernames, 1000 pairs of users whose
0.4
usernames contain languages and numbers other than English
0.3 are extracted to form two data sets: language and number, and
0.2 MOBIUS
UISN-UD
experiments are carried out on the two data sets. As shown in
0.1
BSNA
Fig. 5, compared with the general data sets, languages other
0 than English and numbers have a certain influence on the
5 10 20 30 40 50 60 70 80 90 accuracy of the experiment, among which languages other than
train ratio r(%) English have the greatest influence. However, when compared
(a) Performance to rain ratio with the two benchmark methods, the method in this paper still
has good results on the two data sets.

41
 TABLE II. STATISTICS OF EXPERIMENTAL RESULTS OF BSNA AND [3] Ye N, Zhao Y, Dong L, et al. User identification based on multiple
BENCHMARK METHODS attribute decision making in social networks [J]. China Communications,
2013, 10(12):37-49.
Twitter- Facebook- Twitter-Facebook Twitter-Google
Instagram Instagram [4] O. Goga, D. Perito, H. Lei, et al. Large-scale correlation of accounts
MOBIUS 0.8856 0.8506 0.7885 0.7743 across social networks, TR-13-002[R]. 2013.
[5] Landgrebe D. A survey of decision tree classifier methodology[J]. IEEE
UISN-UD 0.9367 0.9475 0.9053 0.8813 Transactions on Systems Man & Cybernetics, 2002, 21(3):660-674.
[6] Malhotra A, Totti L, Meira W, et al. Studying User Footprints in
BSNA 0.9812 0.9806 0.9694 0.9218 Different Online Social Networks[C]// Ieee/acm International
Conference on Advances in Social Networks Analysis and Mining.
IEEE, 2013:1065-1070.
1 [7] Caliano D, Fersini E, Manchanda P, et al. Uni Mi B: Entity Linking in
0.9 Tweets using Jaro-Winkler Distance, Popularity and Coherence[C]//
WWW. 2016.
0.8
[8] Korula N, Lattanzi S. An efficient algorithm for social networks[J].
0.7 Proceedings of the VLDB Endowment, 2014, 7(5):377-388.
precision@1

0.6 [9] Zhou X, Liang X, Zhang H, et al. Cross-Platform Identification of

Anonymous Identical Users in Multiple Social Media Networks[J].
0.5 IEEE Transactions on Knowledge & Data Engineering, 2016,
0.4 28(2):411-424.
0.3 [10] Zhang Y, Tang J, Yang Z, et al. Cosnet: Connecting heterogeneous
social networks with local and global consistency[C]//Proceedings of the
0.2 21th ACM SIGKDD International Conference on Knowledge Discovery
0.1 and Data Mining. ACM, 2015: 1485-1494.
[11] Wu Zheng, Yu Hongtao, Liu Shuxin, et al. Cross-social network user
0
identification method based on information entropy [J]. Computer
MOBIUS UISN-UD BSNA Applications, 2017, 37(8):2374-2380.
language number normal
[12] Zafarani R , Liu H . Connecting users across social media sites: a
Fig. 5. Analysis of results on synthetic data set behavioral-modeling approach[C]// Proceedings of the 19th ACM
SIGKDD international conference on Knowledge discovery and data
mining. ACM, 2013.
VI. CONCLUSION
[13] Li Y, Peng Y, Zhang Z, et al. Understanding the User Display Names
In this paper, a BP neural network mapping learning model across Social Networks[C]// International Conference on World Wide
BSNA is introduced to align social network accounts. The Web Companion. International World Wide Web Conferences Steering
Committee, 2017.
accuracy is improved by about 4% on several social network
datasets, which shows that the proposed method is superior to [14] Li Y, Peng Y, Zhang Z, et al. Matching user accounts across social
networks based on username and display name[J]. World Wide Web,
the existing work. BSNA can effectively solve social network 2019.
alignment: usernames on different social networks usually [15] Liu J, Zhang F, Song X, et al. What's in a name? an unsupervised
contain a lot of information redundancy, and this kind of data approach to link users across communities[C]// ACM International
does not involve personal privacy and is easy to access, so it Conference on Web Search and Data Mining. ACM, 2013:495-504.
can avoid the problem of obtaining rich social network user [16] Ukil A. Support Vector Machine[J]. Computer Science, 2002, 1(4):1-28.
information based on as few attribute profiles as possible; By [17] Dong Liu, Wu Quanyuan, Han Weihong, et al. Identification of user
using BP neural network, the problem is transformed into the identity based on username features [J]. chinese journal of computers,
mapping problem in different social networks, which 2015,38(10):2028-2040.
minimizes the distance of username feature vectors and solves [18] Man T, Shen H, Liu S, et al. Predict Anchor Links across Social
the dependence on marked user pairs and training iterations to Networks via an Embedding Approach[C]. International Joint
Conference on Artificial Intelligence, 2016: 1823-1829.
a certain extent. The future research direction will enhance the
[19] Zhou F, Liu L, Zhang K, et al. DeepLink: A Deep Learning Approach
identifiability of different languages and numbers, improve the for User Identity Linkage[C]. International Conference on Computer
accuracy of username alignment for different languages, and Communications, 2018: 1313-1321.
realize cross-language social network alignment. [20] Guo X , Liu Y , Liu L , et al. User Identity Linkage Across Social
Networks via Community Preserving Network Embedding[M]//
ACKNOWLEDGMENT Information Security and Privacy. 2020.
This work was supported by the National Natural Science [21] Yu X, Efe M O, Kaynak O. A General Backpropagation Algorithm for
Feedforward Neural Networks Learning[J]. IEEE Transactions on
Foundation of China (U1804263), Basic Strengthening Plan Neural Networks, 2002, 13(1): 251-254.
Technology Foundation (2019-JCJQ-JJ-125) and Zhongyuan
[22] Xue Gangcheng. Research on Flame Detection Algorithm Based on
Science and Technology Innovation Leading Talent Project Deep Learning [D]. xidian university, 2019.
(No. 214200510019). [23] Léon Bottou. Stochastic Gradient Descent Tricks[J]. Neural Networks,
2012, 7700: 421-436.
REFERENCES [24] Iofciu T, Fankhauser P, Abel F, et al. Identifying Users Across Social
[1] https://www.globalwebindex.com/reports/social-2019 Tagging Systems[C]. Proceedings of the Fifth International Conference
[2] Vosecky J, Hong D, Shen V Y S. User Identification Across Multiple on Weblogs and Social Media, 2011: 522-525.
Social Networks[C]// International Conference on Networked Digital
Technologies. IEEE, 2009.

42
 2021 the 13th International Conference on Computer and Automation Engineering

Evolutionary Games and Distributed Decision-

Making in a Multi-agent System for Formation
Control
Shitong Wang, Chunyan Zhang, Jianlei Zhang
College of Artificial Intelligence, Nankai University, Tianjin, China, 300350
[email protected]

Abstract—In this work, the formation control based on dy- simultaneously. The results we obtained are valuable to
namic game evolution under the mixed strategies is proposed. investigate the evolution for behavioral strategies in the
Different from the general formation control and collision multi-agent systems.
avoidance, here agents are independent and make decisions
in a distributed way. The motion states of the agent are In summary, our aim is to study the group evolution in for-
determined by strategy selection and evolution. We investigate mation control for multi-agent systems via the experiments
how the agents accomplish the formation with less loss and under the different circumstances. Meanwhile, observing the
fewer steps. Firstly, the fixation probabilities and proportions experimental results to discuss how to make the system ac-
of different strategies under the different circumstances are an- complish the formation mission better in diverse conditions.
alyzed. Then, we discuss the evolution processes and formation
completions which vary with the collision cost according to the
II. T HEORETICAL M ODEL AND A NALYSIS
simulation results. Moreover, we extend our model to a larger
scale, that is increasing the intensity of population in a fixed Considering a multi-agent system consisting of N
region, to study a more complex system. In our framework, discrete-time agents in a finite region[8, 9]. The set of
we analyze how evolutionary stable strategy emerges and how
formation control finishes better in the population. Our results
population is expressed as Ni , i = 1, 2, ..., N . The number
provide some innovative ideas on achieving formation control of type for strategy probability in the population are T ,
and collision avoidance based on evolutionary game in the real which are described as S = {St |t = 1, 2, ..., P T }. And the
world. T
frequencies of the strategy St is xt . Therefore, t=1 xt = 1
Keywords-evolutionary game theory; distributed decision could be got.
making; formation control; There are two definite strategies in the game, A and B.
Each agent Ni in the population chooses strategy A or B as
I. I NTRODUCTION her current strategy Ci with a certain probability φi in each
Considering that the swarm intelligence can drive the round of the game[10]. All the agents play in a two-player
evolution of behavior[1, 2], we assume that the agents two-strategy game which the payoff matrix is represented as
in a formation mission may adjust their own strategies follows:  
or behaviors according to themselves and the surrounding a1 a2
G= . (1)
environment[3, 4]. That is, all the agents in the system are a3 a4
distributed, they control their behaviors on their owns, but
The quantitative relationship between the elements in the
not from a central control node. The character or personality
matrix G is a1 < a3 < a4 < a2 . When an agent play a
of the agent would determine her strategy, and also the
game with her opponent, she would choose A or B as her
attitude of the opponent does. However, the character of an
current strategy. If she chooses A and her opponent chooses
individual is not invariable. It can be influenced by her own
A at the same time, then they both get a collision cost of
state, other individuals, or the environment[5, 6]. If she is
a1 . If she chooses A, but her opponent chooses B, then she
modest, then she may choose to avoid in the formation tasks.
would get a2 , and her opponent gets a3 . When they both
Otherwise, she is rash and prefer to go straight although
choose B, they get a4 as their collision cost.
she may hit someone. By this way, the formation control
Assuming that when the average strategy probabilities of
can be abstracted as a model which is solved by dynamic
some agents in the population is α, and the average strategy
evolutionary game theory.
probabilities of the other agents is γ, then, the average payoff
Simulation experiments aimed at explaining the forma-
of the α-agents could be indicated by
tion control[7] and group evolution problem in the multi- (
1 −a2 )+N a2 −a1
agent systems. We proposed an algorithm which combining πAα = N (αxα +γ−γxαN)(a −1
evolutionary game theory with intelligence control. In such α )(a3 −a4 )
, (2)
πBα = N (αxα +γ−γx
N −1 + a4
algorithm, agents complete formation tasks in a specific
game which the payoff matrices are predefined. Thence, we in which xα represents the frequency of α-agents. πAα
could observe the processes for population evolution from a denotes the average payoff of α-agents when all the α-agents
new perspective, and analyze the corresponding outcomes choose A-strategy with a certain probability as their current

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 43

 1 1 1 1

0.8 0.8 0.8 0.8

Fixation Probability

Fixation Probability

Fixation Probability

Fixation Probability
0.1 0.1 0.1 0.1
0.2 0.2 0.2 0.2
0.3 0.3 0.3 0.3
0.6 0.4 0.6 0.4 0.6 0.4 0.6 0.4
0.5 0.5 0.5 0.5
0.6 0.6 0.6 0.6
0.4 0.7 0.4 0.7 0.4 (c)ρ ( cc=10) 0.7 0.4 (d)ρ ( cc=10) 0.7
0.8 0.8 α 0.8 γα 0.8
0.9 0.9 0.9 0.9
1 1 1 1
0.2 0.2 0.2 0.2

0 (a)ρα( cc=1) 0 (b)ργα( cc=1) 0 0

0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50
Initial Number of Agent Initial Number of Agent Initial Number of Agent Initial Number of Agent

Figure 1: Figures show the fixation probability of different strategy probabilities When N = 50, and the collision cost cc are
1 and 10, respectively. The subplot (a) and (c) display the fixation probabilities of α when cc = 1 and cc = 10, respectively.
The other two subplots (b) and (d) display that when the a portion of agents adopt α, the fixation probability of γ varies
with the number of individuals. Each line represents the average strategy probability for the different agents, plotted by
initial number of agent on the horizontal axis and fixation probability on the vertical.

1 1 1 1

0.8 0.8 0.8 0.8

Fixation Probability

Fixation Probability

Fixation Probability

Fixation Probability
0.1 0.1 0.1 0.1
0.2 0.2 0.2 0.2
0.3 0.3 0.3 0.3
0.6 0.4 0.6 0.4 0.6 0.4 0.6 0.4
0.5 0.5 0.5 0.5
0.6 0.6 0.6 0.6
0.4 (a)ρα( cc=100) 0.7 0.4 (b)ργα( cc=100) 0.7 0.4 (c)ρα( cc=1000) 0.7 0.4 (d)ργα( cc=1000) 0.7
0.8 0.8 0.8 0.8
0.9 0.9 0.9 0.9
1 1 1 1
0.2 0.2 0.2 0.2

0 0 0 0
0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50
Initial Number of Agent Initial Number of Agent Initial Number of Agent Initial Number of Agent

Figure 2: Figures show the fixation probability of different strategy probabilities When N = 50, and the collision cost cc are
100 and 1000, respectively. The subplot (a) and (c) display the fixation probabilities of α when cc = 100 and cc = 1000,
respectively. The other two subplots (b) and (d) display that when the a portion of agents adopt α, the fixation probability
of γ varies with the number of individuals. Each line represents the average strategy probability for the different agents,
plotted by initial number of agent on the horizontal axis and fixation probability on the vertical.

strategy. Similarly, πBα denotes the average payoff of α- are 

agents if all the α-agents choose B-strategy as their current  Pm,m+1 = Px
N
A N FA
(1 − xA )
Pi=1 xi N F i
strategy. Besides, when γ-agents choose A-strategy or B- N
i=1,Ci 6=A xi N Fi
, (5)
 Pm,m−1 = xA
strategy, the average payoff of γ-agents would be written
PN
i=1 xi N Fi

respectively by
respectively. Thus, Pm,m = 1 − Pm,m+1 − Pm,m−1 repre-
sents m maintains the same value for the next time step.
(
πAγ = γN (a1 −aN2−1
)+N a2 −a1

γN (a3 −a4 )+N a4 −a4 . (3) Then the ratio between m minus to m − 1 and m adds to
πBγ = N −1 m + 1 is
When the agent chooses A-strategy or B-strategy as her PN
Pm,m−1 FA k=1,Ck =A Fk
current strategy in a round of game, the fitness of the agent = = PN . (6)
is expressed as Pm,m+1 FB k=1,Ck =B Fk
F = 1 − ω − ωπ, (4) Thus, the fixation probability for A-strategy could be indi-
where ω represents intensity of selection[11]. When ω → 0, cated by
1
it means that F → 1, other agents would learn A-strategy or ρA = PN −1 Qk FA . (7)
B-strategy with a great probability. Conversely, they might 1 + k=1 i=1 FB
imitate her strategy with a little probability if ω → ∞.
Furthermore, the ratio of A-strategy and B-strategy would
Assuming that there are m agents whose strategies are A. be expressed as
In addition, the fitness of A-strategy is FA . Then, the state k
of the population can just change by one. The transition ρA Y FA
= . (8)
probabilities that form m to m + 1 and from m to m − 1 ρB i=1 B
F

44
 In Fig.1(a), when the collision cost cc = 1, a part
of population take strategy probability α, and the other
agents adopt strategy probability γ. It shows that the fixation
probability of strategy probability α approach 1 when α
is close to 1. Furthermore, when the strategy probability
is greater, it is easier for strategy probability to fix. And
the strategy would dominate in the population, other agents
might imitate the predominant strategy in order to make their (a) Initial state for the formation (b) State of 10 steps for the for-
task. All the agents start to move mation task. A part of agents
payoff become higher. On the contrary, the strategy with from the initial positions (green arrive at the terminal positions
a little probability is hard to dominate among the agents. points) to the terminal positions (red points). When two agents
That is, the number of the agent who adopt these strategies (yellow points). encounter, they may choose to
avoid or collide in order to reduce
would descend. Few agents learn their strategies, so that loss and finish the task within
those strategies would be towards extinction at last. Fig.1(b) short time.
reveals the fixation probability of strategy probability γ
when a portion of agents adopt α and cc = 1. When
the value of α is small, γ-agents might fix with a greater
probability, whereas it is hard for α to reach the fixation
as time goes by. If the value of cc is larger, that implies
the dominant strategies could fix faster. And also the agents
in the population would like to learn those strategies which
are moderate and more likely to avoid obstacles. Reversely, (c) State of 20 steps for the for- (d) Finished state for the forma-
if some agents with a great probability occupy an important mation task. Most agents finish tion task. All of the agents finish
status in the population, thus, it is difficult for other agents to the task, and mark the terminal the task.
positions as red points.
fix their own strategies. In addition, γ-strategy would reach
fixation in the fewer time steps. The simulation results are Figure 3: Some extracts for a round of formation.
shown in Fig.1(c) and (d), respectively, when cc = 10. The
similar conditions could be interpreted in Fig.2 when the
value of cc are 100 and 1000, separately. for a position, she would choose to avoid or collide with
III. S IMULATION AND R ESULTS a certain probability which depends on her own character
and her obtained payoff. If the agent chooses to avoid, it
In this work, a certain scale of individuals take motions indicates she may change the direction of movement to
in a two-dimensional fixed grid area which the region size is bypass the gaming conflict site, and gets a loss of time at
20 × 20. System randomly initializes the starting coordinate the same time. Conversely, the agent would not turn around,
positions Pist , i = 1, ..., N for each agent, and ensures that and just goes ahead. By this time, if the opponent chooses
the positions of everyone are unique and different from each to go straight as well, then two players collide continuously
other. The target locations Pitg , i = 1, ..., N for each one are until one of them update the strategy to make a detour. Thus,
set up simultaneously, and also are not coincident comparing the game between two players finishes. That is, when two
to other locations. individuals meet and choose to go straight, they collide and
When a formation task begins, all agents start at position get a collision cost cc. The formation finishes until all agents
Pist , and then move forward one step on the grid in each reach the target points. Subsequently, all the individuals
time. Fig.3 shows some extracts for a round of formation. study another strategy of agent stochastically and update
The subfigures display some of extracts for the formation, their own strategies Ci utilizing Fermi Function, which is
which is initial state, states of 10 steps, 20 steps and finished described as
state, separately. The green points denotes starting positions,
1 1
and yellow points represents destination points. Besides, θi = = , (9)
1+e −β(πIi −πIj ) 1 + e−β∆π
cyan points signifies the paths of movement, and red points
indicates that agents have arrived at the terminal. More- in which a randomly chosen individual Ii compares its
over, game conflict points are expressed by black points. payoff πIi with a randomly chosen individual Ij . If Ij has
Each agent move towards the destination point Pitg . We higher payoff, then Ii switches strategy with probability
consider four circumstances, in which the payoff matrices θi > 1/2. Otherwise, it switches with θi < 1/2. Assuming
of the game are [1, 0; 1, 0.5], [10, 0; 1, 0.5], [100, 0; 1, 0.5], that the probability of switching is given by Fermi distribu-
and [1000, 0; 1, 0.5], respectively. The value of cc could be tion, and the inverse temperature β determines the intensity
1, 10, 100 and 1000. Once an agent plays a game with an of selection. β  1 means the payoff makes unconsidered
opponent at a certain moment resulting from scrambling contribution to the fitness. However, when β  ∞, the

45
 50 0−0.1 50 0−0.1 50 0−0.1 50 0−0.1
0.1−0.2 0.1−0.2 0.1−0.2 0.1−0.2
(a) cc= 1 0.2−0.3
(b) cc= 10 0.2−0.3
(c) cc= 100 0.2−0.3
(d) cc= 1000 0.2−0.3
40 0.3−0.4 40 0.3−0.4 40 0.3−0.4 40 0.3−0.4
Number of Agent

Number of Agent

Number of Agent

Number of Agent
0.4−0.5 0.4−0.5 0.4−0.5 0.4−0.5
0.5−0.6 0.5−0.6 0.5−0.6 0.5−0.6
0.6−0.7 0.6−0.7 0.6−0.7 0.6−0.7
30 0.7−0.8 30 0.7−0.8 30 0.7−0.8 30 0.7−0.8
0.8−0.9 0.8−0.9 0.8−0.9 0.8−0.9
0.9−1 0.9−1 0.9−1 0.9−1
20 20 20 20

10 10 10 10

0 0 0 0
0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500
Time Step Time Step Time Step Time Step

Figure 4: Each subplot denotes the evolution process over the time steps under the different collision cost, 1, 10, 100 and
1000, when N = 50. Each legend represents the probability of collision. The higher the value, the more likely the individual
would collide, and vice versa.

100 0−0.1
100 0−0.1
100 0−0.1
100 0−0.1
(a) cc= 1 0.1−0.2 (b) cc= 10 0.1−0.2 (c) cc= 100 0.1−0.2 (d) cc= 1000 0.1−0.2
0.2−0.3 0.2−0.3 0.2−0.3 0.2−0.3
80 0.3−0.4 80 0.3−0.4 80 0.3−0.4 80 0.3−0.4
Number of Agent

Number of Agent

Number of Agent

Number of Agent
0.4−0.5 0.4−0.5 0.4−0.5 0.4−0.5
0.5−0.6 0.5−0.6 0.5−0.6 0.5−0.6
0.6−0.7 0.6−0.7 0.6−0.7 0.6−0.7
60 0.7−0.8
60 0.7−0.8
60 0.7−0.8
60 0.7−0.8
0.8−0.9 0.8−0.9 0.8−0.9 0.8−0.9
0.9−1 0.9−1 0.9−1 0.9−1
40 40 40 40

20 20 20 20

0 0 0 0
0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500 0 100 200 300 400 500
Time Step Time Step Time Step Time Step

Figure 5: Each subplot denotes the evolution process over the time steps under the different collision cost, 1, 10, 100 and
1000, when N = 100. Each legend represents the probability of collision. The higher the value, the more likely the individual
would collide, and vice versa.

probability for replacement of the less fit agent is close to 1. goes by.
And then, the population started a new round of formation.
When N = 100, the evolution results are similar to the
When N = 50, the more the collision cost, the fewer the situation when N = 50. Furthermore, if the collision cost
times of collision. On the contrary, the growing times of is small, such as cc = 1, it implies that when two agents
avoidance reveal that in order to increase payoff, the agents collide, they just lose a little, but could reach the destination
reduce loss and learn or imitate the better strategies. Fig.4 as soon as possible. Therefore, the agents are more likely to
shows that when the collision cost is small (e.g. cc = 1), become radical, and would collide with a larger possibility.
the agents are likely to evolve to be uncooperative, and Thus, the population evolves to be willing to collide when
when the population reach to a stable state, the frequencies they encounter and play a game. The population evolves
of cooperation for agents are between 0.4 and 0.7. When to become more uncooperative finally with the probabilities
the collision cost is 10, the probabilities of cooperation between 0.4 and 0.6( Fig.5(a) ). However, the probabilities
for most agents situate between 0.1 and 0.4, and a little of cooperation decrease with the increasing in collision cost.
bit part of agents belong to (0.4, 0.6). When the value of While the collision cost is larger, that means the agents
cc is 100, the cost of collision is too much, so that the might lose a lot, the payoff of them descend rapidly. In order
agents lose a lot, as well as are unwilling to go straight and to reduce loss or arrive at the destination fast, they might
collide with the opponents. Thence, the population evolves to imitate or study some dominant strategies. Under these
become more cooperative, and would rather choose to avoid circumstances, some moderate agents could be dominant
when faced with the opponents. As a consequence, most of in the population, as they lose little and are willing to
individuals evolve to be more moderate, and the frequencies avoid with each other. And finally, the population evolves
mostly belong to the interval of (0, 0.3). However, the to be more likely to avoid or become cooperative. The
situation for cc = 1000 is more extreme. Because of the probabilities are concentrated in (0, 0.3), (0, 0.2), when the
exorbitant collision cost, agents in the population become collision cost are 10, 100, respectively ( Fig.5(b) and (c) ).
more and more reluctant to collide when meet. So that, the In Fig.5(d), most of the agents evolve to the states that the
probabilities of strategies concentrate in (0, 0.2) with time probabilities to collide are between 0 and 0.1.

46
 According to the simulation results we get, we can con- [2] B. Allen, G. Lippner, Y.-T. Chen, B. Fotouhi, N. Mo-
clude the strategy which colliding with a great probability meni, S.-T. Yau, and M. A. Nowak, “Evolutionary
in the system fixes to nearly 1 when collision cost is small. dynamics on any population structure,” Nature, vol.
In addition, when N is larger, such as N = 100, strategy 544, no. 7649, pp. 227–230, 2017.
may fix in shorter time steps. Reversely, strategy which [3] L. Hindersin, B. Wu, A. Traulsen, and J. Garcı́a,
avoiding with a great probability would dominate in the “Computation and Simulation of Evolutionary Game
system when collision cost is much, like cc = 100, 1000. Dynamics in Finite Populations,” Sci. Rep., vol. 9, 12
Thus, the simulation results coincide with the theoretical 2019.
analysis and numerical calculation we obtain in Section 2. [4] X. Gao, J. Wang, and D. Yang, “Stability and stabiliza-
tion of evolutionary games with time delays via matrix
IV. CONCLUSION method,” Asian Journal of Control, vol. 21, no. 6, pp.
A formation control and collision avoidance based on 2587–2595, jun 2019.
dynamic evolutionary game theory and distributed decision- [5] X. Ding, H. Li, and F. E. Alsaadi, “Regulation of
making is studied here. We discuss behavioral evolution game result for n -person random evolutionary Boolean
of the population via the interactions between agents in games,” Asian Journal of Control, may 2019.
formation missions. All of the agents take mixed strategies. [6] G. Ichinose and T. Kurisaku, “Positive and negative
Either the agent is moderate, and more likely to avoid the effects of social impact on evolutionary vaccination
collision, or she is impulsive, and prefer to go on a rampage. game in networks,” Physica A: Statistical Mechanics
The agents take these strategies with a certain probability. and its Applications, vol. 468, pp. 84–90, 2017.
Through theoretical analysis and simulation, we study the [7] D. W. Djamari, “Distributed position estimation ap-
impact of collision cost on the evolution of strategy in the proach for multiagent formation with size scaling,”
population. Besides, the simulation results here confirm our Asian Journal of Control, sep 2020.
numerical calculation. Results show that excessive collision [8] J. Zhang, Z. Li, Z. Xu, and C. Zhang, “Evolutionary
cost makes individuals tend to choose an avoidance strategy, Dynamics of Strategies without Complete Information
so as to decrease the loss of payoff. In addition, if the agents on Complex Networks,” Asian Journal of Control,
are more likely to avoid and make a detour, the finishing path vol. 22, no. 1, pp. 362–372, aug 2018. [Online].
for the agents would increase, which means the individuals Available: https://doi.org/10.1002%2Fasjc.1874
would reach to the destination more slowly and accomplish [9] Taroni and Andrea, “Evolutionary game theory: Metric
the formation task with long period. of cooperation,” Nat. Phys., vol. 14, no. 1, pp. 7–7,
Besides, the scale of population can also make an influ- 2018.
ence on the efficiency of formation. A higher density of [10] Z. Xu, J. Zhang, C. Zhang, and Z. Chen, “Fixation of
population makes the area more crowded. The possibility strategies driven by switching probabilities in evolu-
of the interaction between agents increases. To some extent, tionary games,” EPL (Europhysics Letters), vol. 116,
it can cause the rising of average time steps for finishing no. 5, p. 58002, dec 2016. [Online]. Available: https:
a formation task. Moreover, as the evolutionary velocity //doi.org/10.1209%2F0295-5075%2F116%2F58002
accelerates, the system would evolve stably and complete [11] L. A. Imhof and M. A. Nowak, “Evolutionary game
convergence in less time. All the results proposed here offer dynamics in a Wright-Fisher process,” J. Math. Biol.,
us a novel approach to investigate formation control and vol. 52, no. 5, pp. 667–681, 2006.
interaction between individuals, and also make us discuss
the evolution of population in a new light. By this way,
we could study the evolutionary proceeding from a more
realistic aspect.
ACKNOWLEDGMENT
This work was supported by the National Natural Science
Foundation of China (Grant Nos. 62073174, 62073175 and
91848203) and the Tianjin Natural Science Foundation of
China (Grant No. 18JCYBJC18600).
REFERENCES
[1] J. Chen, B. Liu, and M. Umar, “Evolutionary dynamics
games based on edge diversity in complex networks,”
in 2020 International Conference on Networking and
Network Applications (NaNA), 2020, pp. 254–258.

47
 2021 the 13th International Conference on Computer and Automation Engineering

STGRN: A Spatial-Temporal Graph Resonance

Network for Social Connection Inference
Shengjie Min Jing Peng Guangchun Luo
University of Electronic Sichuan Provincial University of Electronic Science
Science and Technology of China Public Security Department and Technology of China
Chengdu, China Chengdu, China Chengdu, China
[email protected] [email protected] [email protected]

Zhan Gao Bo Fang Dingyuan Rao

Sichuan University ChinaCloud Information ChinaCloud Information
Chengdu, China Technology Co., Ltd. Technology Co., Ltd.
[email protected] Chengdu, China Chengdu, China
[email protected] [email protected]

Abstract—Social Connection Inference (SCI), as one of the

most important social network analysis techniques, has been a
popular research direction for decades. A lot of studies have
been utilizing spatial-temporal co-occurrence events for mining
latent social relations. The previous studies mainly look into the
characteristics of the point-to-point meetup events. Moreover,
some methods employ the social neighborhood of two target
objects for better prediction. However, we have rarely seen
research utilizing the network structure formed by meetup
events for better SCI performance. In this paper, we propose
a graph neural network framework, namely Spatial-Temporal
Graph Resonance Network (STGRN), for social connection in-
ference enhancement. Firstly, we calculate co-occurrence events’
characteristics, including two novel features we designed: Geo-
Spread and ActM-Rate. Secondly, we build a network embedding Fig. 1: Social Connection Inference Approaches. (1) Approach
based on meetup relations. Thirdly, we combine the meetup
events, social network, and co-occurrence network for the final 1 only utilizes the features of meetup events. (2) Approach 2
prediction. In the end, we conduct extensive experiments on employs a partially known social network structure for better
public datasets to prove our method’s effectiveness. performance. (3) Approach 3 (Our method) combines the
Index Terms—Trajectory Data Mining, Graph Neural Net- features of meetup events, social and co-occurrence network
work, Social Connection Inference, Social Network Analysis structures for further improvement.
I. I NTRODUCTION
Social Network Analysis (SNA) [1] has been a popular Trajectory, as a series of spatial-temporal data points, is
research direction for years. Various relationships, such as an indicator of people’s geographic activities. Co-occurrence,
phone calls, instant messages, meetup events, etc., form differ- indicating two people occurring together simultaneously, is
ent social networks, which are inevitably incomplete (missing the main event we look into for mining social connection.
nodes and links). Therefore, Numerous recommendation sys- The research on location-based social networks has intensively
tems intend to find potential links between users to recommend studied the meetup events, which have been increasingly useful
friends or products. In our case, we mainly study criminal for intelligence agencies. Unlike some direct relations, such
networks, which are incomplete due to various reasons: lack of as family ties, telecommunications (phone calls or messages),
intelligence, partially captured information, and anti-detection financial transactions, etc., the social links discovered on geo-
strategies from criminals. Unlike failing to make a friend graphic movement are far more implicit, providing a different
recommendation or delivering an inappropriate advertisement, angle for us to look at the relations between objects.
some missing connections in the criminal network may cause Group pattern discovering methods, such as Traveling com-
severe consequences. Therefore, predicting social links and panion [2], Gathering [3], Platoon [4], etc., study the moving
having the criminal network reflect the real-world relations as patterns of objects and divide spatiotemporally nearby objects
much as possible have been one of the most critical analytical into groups. Further, there are numerous methods of group
tasks in the criminology field. pattern mining predicting social relations over co-occurrence

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 48

events, such as EBM [5], PGT [6], SCI+ [7] and Resonance
[8], etc. As shown in Figure 1, Approach 1 mainly looks at
the characteristics of the co-occurrence events between objects.
The potential connections may have different strengths as the
co-location events vary. For instance, two strangers bouncing
into each other in a bus station is completely different from
two acquaintances meeting up in a private club.
From a spatial perspective, there are some key co-location
factors: Count, Popularity, Diversity, and Geo-Spread, etc.
Count indicates the number of commonplaces two objects
meet up. Popularity shows how popular the meetup place is
from both public and private perspectives. Diversity demon-
strates how diverse the co-location places vary, while Geo-
Spread is a crucial factor capturing the magnitude of co-
occurrence spread. From a temporal perspective, the key
factors are Gap, Stability, Duration, etc. Gap is an indicator Fig. 2: Architecture of STGRN
reflecting the time interval between two consecutive events.
Stability indicates the intensity of the two objects’ meetups,
STGRN (Approach 3 in Figure 1), for social connection
while Duration indicates how long two objects have been
inference enhancement.
meeting. By looking at the co-occurrence events’ different
In our study, our main contributions are as follows:
factors, we can distinguish the coincidental and organized
(i). We propose two novel factors, Geo-Spread and
meetups.
ActiveMovement-Rate (ActM-R). Geo-Spread captures the
As a lot of studies have noticed, the check-in trajectory
magnitude of co-occurrence spread while ActM-R penalizes
data can be very imbalanced. Some active objects may have
the co-location cases that people have too many trajectory
hundreds of trajectory points, while some inactive ones may
points.
only have a few. And we found that the same phenomenon
(ii). We build a network on co-occurrence events: resonance
in the real-world datasets. The intuition is that when a co-
network, and utilize the graph neural network approach to
occurrence event happens on two objects with hundreds of
model it, which provides valuable insight into latent social
trajectory points a day while a meetup occurs on two people
relationships from a spatial-temporal perspective.
with only a few trajectory points a day, the latter case has
(iii). To the best of our knowledge, our method is the first
a more significant probability of being friends. In this work,
attempt that leverages the structural information in a spatial-
we propose a novel factor: Active Movement Rate, namely
temporal meetup network to improve the effectiveness of the
ActM-R, to capture it.
social connection prediction tasks.
The popularity of deep learning has been rapidly growing
(iv). We conduct extensive experiments on Gowalla and
in recent years. For graph-related problems, the traditional
Brightkite public datasets with social connection inference
machine learning methods require in-depth domain knowledge
tasks. The results show that our method outperforms the state-
and intensive-labor feature engineering. GNN (Graph Neural
of-the-art baselines significantly.
Network), inspired by previous methods such as CNNs and
RNNs, has become an effective deep learning methodology II. M ETHODOLOGY
for network analysis. It has been proven that methods such In this paper, we propose a novel framework to infer so-
as DeepInf [9], SEAL [10], GCN [11], GraphSage [12], cial connections from spatial-temporal trajectories. Firstly, we
GAT [13], CTDNE [14], LBSN2Vec [15], etc., provide ef- calculate the co-occurrence events to predict the social links
ficient ways of transforming graphs into low-dimensional using multiple features, including two proposed factors: Geo-
representations for better down-stream machine learning tasks. Spread and ActM-R. Secondly, we build a spatial-temporal
As Approach 2 shown in Figure 1, deep learning methods resonance network based on the co-occurrence events. In the
empower us to aggregate the social neighborhood information end, we utilize the graph neural network method to model
for further prediction tasks. Motivated by the above-mentioned the network’s topological structure, combined with co-location
deep learning frameworks, we take the graph neural network events and social networks, to make further social connection
path to tackle our social connection inference problems. predictions.
The existing studies mainly focus on point-to-point co-
occurrence events when predicting social connections, which A. Co-occurrence Event Characteristics
generally neglect the fact that the network structure can be We first calculate the co-occurrence events based on check-
influential. Further, we have not seen research building net- in data points. Then, we design three modules for the down-
works over co-occurrence relations and utilizing its topology stream social connection inference task: event, social, and
information to improve social connection inference tasks’ spatial-temporal (resonance). The event module is mainly cap-
performance. Therefore, we propose a graph neural network: turing the characteristics of the co-occurrence events between

49
objects. Our study utilizes seven popular features: Frequency, B. Two Novel Factors: Geo-Spread and ActM-R
Personal Popularity, Global Popularity, Diversity, Temporal
Along with the features above-mentioned, we design two
Gap, Temporal Duration, and Temporal Stability.
new novel factors to capture more latent characteristics of
1) Frequency is the number of the co-locations as |Ri,j |. co-occurrence events: Geo-Spread and ActM-R. Geo-Spread
2) Personal Popularity Gpij indicates the probability of an represents the total distance between the meeting locations
individual user visiting a location [6]. of two objects. Although previous research used features
like diversity to capture meeting locations’ uncertainty [7],



X exp −cd · dist lock , lociq
ρ (i, lock ) = it failed to capture the geographic characteristics. Experience
|Si |
(lociq ,tiq )∈Si in public security investigation tells us that the further the
(1) meeting places spread, the closer the two objects are related.
p
wij (ek ) = − log (ρ (i, lock ) · ρ (j, lock )) (2) To describe the magnitude of the meeting locations spread, we
define Geo-Spread as the standard deviation of the distances
Gpij = max wij p

(ek ) × |Eij | (3)
ek ∈Eij between all meeting locations and their center point. The
center point’s position is the arithmetic average of latitudes
3) Global Popularity Ggij captures the popularity of a
and longitudes of all meetup locations.
location to the general public, which can be calculated
Suppose there are N meeting locations between objects i
using Shannon entropy [5] [6].
X and j. The mean latitude is ϕi,j and mean longtitude is λi,j as
Hl = − Pi,l log Pi,l (4) shown in Equation (16). The center of all locations is (ϕi,j ,
i:Pil 6=0 λi,j ).
X PN N
P
Ggij = exp(−Hl ) (5) ϕk λk
ij,l ϕi,j = k=1 ; λi,j = k=1 (16)
N N
4) Diversity Dij evaluates the uncertainty of meeting lo-
cations by Shannon Entropy [6] [7]. In Equation (17), Haversine Formula [16] is used to calcu-
late the distance. The distance between the kth location and
l
|Rij |
l
Pi,j = (6)
the center point is notedras ∆dk . r is the radius of the earth.
|Rij | ϕk − ϕi,j
sin2 (


∆dk = 2r arcsin ) + cos ϕi,j
S
X 2
Hij =− Pijl log Pijl (7) ! (17)
2 λk − λi,j
l
S
cos (ϕk ) sin ( )
Dij = exp(Hij ) (8) 2
In the end, we calculate the standard deviation of the
5) Temporal Gap T Gij penalize the weight of correlated
distances, noted as GSi,j in Equation (18).
meeting events [6]. v
u N
uP 2
lT (ek , ep) = exp(−ct · |tk − tp |) (9) u (∆dk )
GSi,j = k=1
t
( (18)
t 1 − lT (ek , ep), k > 1 N
wij (ek ) = (10) Most previous studies handled the imbalance from a lo-
1, k=1
cation perspective, without considering the impact of the
people’s trajectory sparsity. SCI+ [7] mentioned the imbalance
X
t
T Gij = wij (ek ) (11)
ek ∈Eij of user activity, but it just filtered out all users with less than
ten total check-ins. In the public security area, however, the
6) Temporal Duration wδ (x, y) indicates how long two long tail of the inactive objects is non-negligible, especially for
objects have been meeting [7]. criminals with anti-tracking skills. In the criminology field, we
δx,y = max(tix,y ) − min(tix,y ) (12) found that abnormally rare activities sometimes can be more
suspicious. Therefore, we propose a new factor named ActM-
δx,y R to highlight the hard-to-capture observations and to penalize
wδ (x, y) = (13)
maxu∈U (δx,y ) the easy-to-capture ones.
7) Temporal Stability ws (x, y) represents the intensity of We define the importance of object i to object j as the
the two objects’ meetups [7]. ratio of all meetings between i and j to the total number of
P i observations of i, as shown in Equation (19). ActM-R of the
δx,y object i and j is the square root of Ii,j i j
and Ii,j ’s product,
µx,y = (14)
|Ψ| denoted as AM Ri,j in Equation (20).
s
1 X i |Ri,j |
ws (x, y) = δ − µx,y (15) i
Ii,j =
|ψx,y | i x,y |Ti |
(19)

50
 III. E XPERIMENT AND E VALUATION
|Ri,j |
q
AM Ri,j = i · Ij
Ii,j =p (20) A. Datasets and Metrics
i,j
|Ti | · |Tj | This section presents a comprehensive performance study of
the proposed framework and two novel features on two public
C. Social and Co-occurrence Network Embedding datasets: Gowalla and Brightkite from SNAP [17]. They are
The social module captures the social topology of tar- both location-based social networking websites where users
geted objects by embedding their neighborhood information. share their locations by checking-in. Gowalla dataset contains
Adopting the GraphSAGE [12] approach, we aggregate the 196591 objects, 950327 relationships, and 6442890 trajectory
neighborhood features to the target node using Equation (21). points collected from Feb. 2009 to Oct. 2010. Brightkite
has 58228 users, 214078 connections, and 4491143 trajectory
k−1
" #!
points from Apr. 2008 to Oct. 2010. We use the AUC (the area
P
s j∈N (i) hj k−1
hi =ReLU Wk  
N(i)  , Bk hi , ∀k ∈ {1, 2, ...K} under the curve) score to evaluate our method’s performance
and compare it to the state-of-the-art methods, and we also
(21)
plot the AUC score histogram to illustrate the comparison.
B. Experiment Setup
hsi,j = hsi , hsj , hci,j = hci , hcj
   
(22)
Gowalla and Brightkite contain all users’ check-ins over the
hsi denotes the social network embedding of the ith node, whole world. It is not very meaningful to infer all these users’
and k indicates the number of layers for the neighborhood relations, considering the relations mainly exist between two
aggregation. N(i) is node i’s neighbors within K hubs while users from the same city. Therefore, we conduct experiments
hk−1 and hk−1 define the feature embeddings of neighbor j in three areas: California, Texas, and Stockholm, respectively.
j i
and i, respectively. Wk and Bk are two training parameters. We group all the check-ins to the cities based on their
We aggregate all neighbors layer by layer until we reach the longitudes and latitudes. Further, we select the users whose
target — node i. By concatenating hsi and hsj , we have a check-ins are usually located in the city and filter out the
vector: hsi,j representing i and j’s social topology. Meanwhile, users with less than 10% of their total check-ins. The subset
hci and hcj represent the co-occurrence network embeddings of selection strategy above-mentioned is commonly used by prior
node i and j, respectively. hci,j implies the spatial-temporal studies [18] [15] [19]. In the end, we randomly split the dataset
neighborhood information of i and j’s over co-occurrence into two parts, 60% for training and 40% for evaluation.tex
network. C. Baselines
Our study utilizes the co-occurrence network, a network The baselines we select are: EBM [5], PGT [6], SCI+ [7],
built upon the meetups, as a key part of the framework. Resonance [8], GCN [11], LBSN2Vec [15], and GraphSAGE
We create a link between any pairs having meetup events [12].
within one hour. It is worth pointing out that there might 1) EBM [5]: EBM makes use of Entropy to analyze how
be better ways of making co-occurrence networks, such as the diversity of the co-occurrence events affects the
setting up a meetup threshold, calculating co-locations via strength of social connections.
zone segmentation, etc. We decide to leave it as the future 2) PGT [6]: PGT utlizes three factors: personal, global,
research direction and focus on whether the co-occurrence temporal in co-occurrence events to predict the social
network brings any improvement to the social connection connection between two mobile users.
inference tasks. We aggregate the spatial-temporal neighbors 3) SCI+ [7]: SCI+ takes features: frequency, popularity,
using a similar approach to the social network embedding. diversity, stability and duration into consideration to
In our case, we also aggregate meetup neighbors within two distinguish real friends’ meetings from strangers’ coin-
layers. cidental meetings.
4) Resonance [8]: Resonance introduces Geo-Spread,
D. Final Fully-Connected Layer which captures the extent of the co-location spread, as
a novel factor to produce better SCI prediction.
As shown in Equation (23), we combine the event module,
5) Resonance+ : Resonance+ is a version of Resonance
social network embedding, and spatial-temporal network em-
with the novel feature we propose in this work: ActM-
bedding using a fully-connected neural network. It aggregates
Rate.
the three components by a direct concatenation, while sum-
6) GCN [11]: As a typical spectral-based approach (like
ming them up can also be an option. In the end, hˆi,j is the
CNN), GCN learns the target node’s features by per-
representation considers node i and j’s meetup event, social
forming convolutional operations on the neighboring
network structure, spatial-temporal topology all together for
cells.
the down-stream social connection inference prediction.
7) LBSN2Vec [15]: LBSN2Vec is a hyper-graph embed-
ding approach designed specifically for LBSN data for
hˆi,j = hei,j , hsi,j , hci,j
 
(23) automatic feature learning.

51
 8) GraphSAGE [12]: GraphSAGE is a general induc-
tive embedding framework that effectively aggregates
features from a node’s local neighbors and generates
embeddings for previously unseen data. In the exper-
iments, we apply this method to aggregate only the
social neighbors, while our method employs social and
meetup neighbors. For aggregation strategy, we adopt
Mean (average), MeanPool (element-wise mean), and
MaxPool (element-wise max).

D. Effectiveness of Our Method (a) AUC Score of Algorithms (California)

To evaluate the effectiveness of the proposed framework, we
compare our method to the popular methods: EBM [5], PGT
[6], SCI+ [7], Resonance [8], GCN [11], LBSN2Vec [15], and
GraphSAGE [12]. As shown in Table I and Figure 3, we can
make the following obversations:

TABLE I. The AUC Scores of Baselines

Brightkite Gowalla
Baseline
CA TX STO CA TX STO
EBM 0.8272 0.8771 0.7187 0.8827 0.8613 0.9098
(b) AUC Score of Algorithms (Texas)
PGT 0.7414 0.7492 0.6150 0.6413 0.6777 0.6185
SCI+ 0.8856 0.9259 0.9212 0.9425 0.9474 0.9512
Resonance 0.9031 0.9395 0.8975 0.9502 0.9485 0.9601
Resonance+ 0.9122 0.9421 0.9108 0.9596 0.9573 0.9703
GCN 0.7472 0.6796 0.6829 0.8833 0.8676 0.7562
LBSN2Vec 0.9074 0.9456 0.9047 0.9577 0.9717 0.9354
GraphSAGE 0.9265 0.9521 0.9345 0.9771 0.9784 0.9666
STGRN 0.9341 0.9647 0.9415 0.9834 0.9898 0.9713

1) STGRN produces the best overall performance

among all baselines. STGRN has the best overall
performance on all datasets and outperforms all the
baselines by a significant margin. As shown in Figure (c) AUC Score of Algorithms (Stockholm)
3, STGRN produces the best AUC scores: 0.9341,
Fig. 3: AUC Scores of Methods
0.9647, 0.9415 on Brightkite, 0.9834, 0.9898, 0.9713
on Gowalla. The results prove that our method has
a powerful capability to infer social connections by
combining co-location events, social networks, and co- pointing out that the only difference between Resonance
occurrence networks. In particular, the proposed co- and Resonance+ is one feature: ActM-R. Therefore, the
occurrence network embedding does bring extra valu- results indicate that ActM-R is a crucial factor of the
able information and helps our framework produce better meetup events regarding social connection inference.
prediction performance. The intuition is that our frame- 3) Deep learning approaches generally outperforms the
work knows an additional piece of information between feature-based methods. By looking at Figure 3, we
two users: spatial-temporal friends. This extra piece of notice that deep learning frameworks: LBSN2Vec [15],
knowledge makes a significant contribution to the final GraphSAGE [12], and STGRN (our method), generally
social connection inference. perform better than feature-based methods: EBM [5],
2) ActM-R proposed in this work is a useful feature for PGT [6], SCI+ [7], and Resonance [8]. In particular,
inferring social connections based on co-occurrence STGRN and GraphSAGE take all the first and second
events. The feature of Geo-Spread’s effectiveness has places in all cases. It indicates that, compared to feature-
been proven in our prior work [8], and we find the based approaches, GNNs (Graph Neural Network) have
same phenomena in our experiments as well. As shown more robust capabilities of capturing the latent relation-
in Figure 3, Resonance wins all the cases against ships within location-based social networks. It is also
other feature-based methods: EBM [5], PGT [6], and noticeable that GCN performs very poorly in all cases.
SCI+ [7]. Furthermore, an important finding is that: We suspect the sparsity of the neighborhood causes
Resonance+ beats Resonance in all cases. It is worth trouble to the spectral-based convolutional method. In

52
 contrast, the network embedding methods can create [11] Thomas N. Kipf and Max Welling. Semi-supervised classification with
better low-dimensional representation for this type of graph convolutional networks. 5th International Conference on Learning
Representations, ICLR 2017 - Conference Track Proceedings, pages 1–
network. 14, 2019.
It is worth pointing out that we also applied the same ex- [12] Nesreen K. Ahmed, Ryan A. Rossi, Rong Zhou, John Boaz Lee,
Xiangnan Kong, Theodore L. Willke, and Hoda Eldardiry. Inductive
periments on real-world surveillance datasets and verified the Representation Learning in Large Attributed Graphs. arXiv preprint
predictions against the criminal networks labeled by the police arXiv:1710.09471, pages 1–11, 2017.
department. The results demonstrate the same phenomenon as [13] Petar Veličković, Arantxa Casanova, Pietro Liò, Guillem Cucurull,
Adriana Romero, and Yoshua Bengio. Graph attention networks. 6th
what we have observed on public datasets. Especially when International Conference on Learning Representations, ICLR 2018 -
social and meetup relations are both more sparse in the real- Conference Track Proceedings, pages 1–12, 2018.
world, the margins between our method and baselines are more [14] Giang Hoang Nguyen, John Boaz Lee, Ryan A. Rossi, Nesreen K.
Ahmed, Eunyee Koh, and Sungchul Kim. Continuous-Time Dynamic
significant. However, due to the data and privacy protection Network Embeddings. The Web Conference 2018 - Companion of the
regulations, we can not reveal the data in any form from the World Wide Web Conference, WWW 2018, pages 969–976, 2018.
law enforcement agencies. Therefore, we only share the results [15] Dingqi Yang, Bingqing Qu, Jie Yang, and Philippe Cudre-Mauroux. Re-
visiting user mobility and social relationships in LBSNs: A hypergraph
on the public datasets to illustrate the effectiveness of our embedding approach. The Web Conference 2019 - Proceedings of the
method. World Wide Web Conference, WWW 2019, pages 2147–2157, 2019.
[16] James Inman. Navigation and Nautical Astronomy for the Use of British
IV. CONCLUSION Seamen. C. and J.Rivington, 2010.
[17] Eunjoon Cho. Friendship and Mobility : User Movement In Location-
This paper proposes STGRN, a graph neural network frame- Based Social Networks. Proceedings of the 17th ACM SIGKDD
work for social connection inference over spatial-temporal international conference on Knowledge discovery and data mining,
2011.
trajectories. The framework firstly extracts multiple features on [18] Wei Zhang, Jianyong Wang, and Wei Feng. Combining latent factor
co-occurrence events, including Geo-Spread and ActM-R pro- model with location features for event-based group recommendation.
posed in this work. Then, it uses graph embedding techniques Proceedings of the ACM SIGKDD International Conference on Knowl-
edge Discovery and Data Mining, Part F128815:910–918, 2013.
to transform both social and co-occurrence networks into [19] Wei Zhang, Xin Lai, and Jianyong Wang. Social Link Inference via
low-dimensional vectors. In the end, it aggregates the three Multiview Matching Network From Spatiotemporal Trajectories. IEEE
modules representing meetup events, social and co-occurrence Transactions on Neural Networks and Learning Systems, pages 1–12,
2020.
networks to predict social connections. Experiments on public
datasets prove the effectiveness of our framework.

R EFERENCES
[1] Stanley Wasserman and Katherine Faust. Social Network Analysis.
Cambridge University Press, nov 1994.
[2] Lu An Tang, Yu Zheng, Jing Yuan, Jiawei Han, Alice Leung, Chih Chieh
Hung, and Wen Chih Peng. On discovery of traveling companions from
streaming trajectories. Proceedings - International Conference on Data
Engineering, 1(3):186–197, 2012.
[3] Yu Zheng, Xiaofang Zhou, Kai Zheng, Nicholas J. Yuan, and Shuo
Shang. Online Discovery of Gathering Patterns over Trajectories. IEEE
Transactions on Knowledge and Data Engineering, 26(8):1974–1988,
2013.
[4] Yuxuan Li, James Bailey, and Lars Kulik. Efficient mining of platoon
patterns in trajectory databases. Data and Knowledge Engineering,
100:167–187, 2015.
[5] Huy Pham, Cyrus Shahabi, and Yan Liu. EBM - An Entropy-Based
Model to Infer Social Strength from Spatiotemporal Data. Proceedings
of the 2013 international conference on Management of data - SIGMOD
’13, page 265, 2013.
[6] Hongjian Wang, Zhenhui Li, and Wang Chien Lee. PGT: Measuring
Mobility Relationship Using Personal, Global and Temporal Factors.
Proceedings - IEEE International Conference on Data Mining, ICDM,
2015-Janua(January):570–579, 2015.
[7] Gunarto Sindoro Njoo, Kuo-wei Hsu, and Wen-chih Peng. Distinguish-
ing Friends from Strangers in Location-based Social Networks using
Co-location. Pervasive and Mobile Computing, 2018.
[8] S. Min, G. Luo, Z. Gao, J. Peng, and K. Qin. Resonance - an intelligence
analysis framework for social connection inference via mining co-
occurrence patterns over multiplex trajectories. IEEE Access, 8:24535–
24548, 2020.
[9] Jiezhong Qiu, Jian Tang, Hao Ma, Yuxiao Dong, Kuansan Wang, and
Jie Tang. DeepInf: Social influence prediction with deep learning. Pro-
ceedings of the ACM SIGKDD International Conference on Knowledge
Discovery and Data Mining, pages 2110–2119, 2018.
[10] Muhan Zhang and Yixin Chen. Link prediction based on graph neural
networks. Advances in Neural Information Processing Systems, 2018-
Decem(Nips):5165–5175, 2018.

53
 2021 the 13th International Conference on Computer and Automation Engineering

Classifying Solving Behavior by Handwriting on

Tablets
Ryosuke Goshima Fumiko Harada Hiromitsu Shimakawa
Ritsumeikan University Connect Dot Ltd Ritsumeikan University
Shiga, Japan Kyoto, Japan Shiga, Japan
[email protected] [email protected] [email protected]

Abstract—In this thesis, we propose a method to estimate the understand how well the knowledge and skills they need to
learner's solving behavior during the question exercises from the acquire are in the students. Teachers need to understand how
handwriting situation on the tablet. The model of random forest students are using their knowledge and skills in these situations.
is constructed to estimate the solving behavior. The constructed
model classifies exactly not only the state in which the hands are Konishi et al. [2] proposed a method for estimating learning
stopped and the state in which the hands are moving but also the behavior based on psychological evaluation from the learner's
state of answering correctly and the state of answering biological information. Biological information is brain waves,
incorrectly. pulse rate, heart rate, blood flow, and so on. Learner's irregular
learning behavior is detected by analyzing biological
Keywords—handwriting, solving behavior, tablets, electronic information. However, specialized equipment is required for
pen, support seeking analysis. Learners feel a high load. Grafsgaard et al. [3]
proposed a method for predicting learning effects from
I. INTRODUCTION nonverbal behaviors and task actions. Nonverbal behaviors are
It is important to give appropriate guidance according to the facial expressions or postures. Facial expressions and postures
level of understanding of the learner. In the conventional are automatically detected using image processing. This study
educational field, teachers often look around the learner while shows that multimodal data including nonverbal behavior and
the learner works on the question exercises. In this way, task actions are effective for prediction. However, the
teachers grasp the level of understanding of the learner. The classification accuracy is not that high. A highly accurate
load on the teachers is high when a small number of teachers estimation method is required. It should not place a load on the
teach a large number of learners. Many people use paper and learner.
pens to study. We are used to handwriting learning. In this
research, we focus on tablets from the viewpoint of enabling B. Recognition of Understanding Status from Handwriting
handwriting learning when used in combination with an Under certain circumstances, learning with pen-based
electronic pen. tablets is as effective as learning with paper and pencil [4, 5, 6].
Learning methods using pen-based tablets are studied
In this study, we propose a method to estimate the learner's positively. A method to support learning a foreign language
solving behavior by analyzing the handwriting situation on the using Pen-based tablets was proposed. Yu et al. [7, 8] showed
tablet. This method estimates learner's solving behavior from that the learner's cognitive load is estimated using only the
handwriting logs and supports seeking logs. The handwriting features obtained from the online handwriting logs. This study
log is the pen information acquired when the electronic pen is shows that the three writing features are effective as indicators
touched to the tablet. Pen information consists of the following: of cognitive load. The three writing features are writing
position coordinates, pressure, altitude, and azimuth. Support pressure, writing speed, and pen orientation. In particular, the
seeking log is the record of checking hints and explanations local maximum writing pressure and the local minimum
during the question exercise. This method creates a random writing speed are effective indicators of cognitive load. These
forest model to estimate the solving behavior. Aurélien [1] features tend to occur at the beginning and end of a stroke. The
said ”Random Forests work by training many Decision Trees model that combines pen pressure and pen orientation can be
on random subsets of the features, then averaging out their estimated with the highest accuracy.
predictions. Building a model on top of many other models is
called Ensemble Learning, and it is often a great way to push III. ESTIMATION OF SOLVING BEHAVIOR BY HANDWRITING
ML algorithms even further.” Teachers can give guidance
according to the situation. A. Solving Behavior on Tablet
In this study, we propose a method to estimate the learner's
II. ESTIMATION OF THE SOLVING BEHAVIOR DURING solving behavior by analyzing the handwriting situation on the
LEARNING tablet. Many people use paper and pens to study. we are used
A. Estimation of the Learning State to handwriting learning. Handwriting learning is effective in
the contents of working memory are stored for a long time [9,
Classes at cram schools and schools aim to equip learners 10]. In this study, we consider a method to estimate the
with knowledge and skills. After teaching knowledge and skills, learner's solving behavior by acquiring the writing pressure,
teachers allow students to apply them. Teachers need to writing speed, and pen orientation as handwriting logs. This

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 54

method analyzes the learner's handwriting data to estimate the Studying state is the state in which learners are answering
state of the learner's solving behavior. Written logs are questions using hints and explanations. It is assumed that
acquired when the learner engages in question exercises on a learners are trying to acquire knowledge by reading the
tablet. One of the purposes of the question exercise is to be explanation. Misunderstanding state is the state in which
able to solve typical questions smoothly. In this study, we learners are answering questions incorrectly. Learner's hands
assume question exercises to acquire definitions and formulas are moving, but they keep writing the wrong answer. Learners
as knowledge. continue to answer without noticing what is wrong. If this state
continues, the learning efficiency deteriorates. Knowledge less
state is the state in which learners do not know what to do. It is
assumed that they cannot understand the contents even if they
read the hints and explanations. Thus, their hands were stopped
during learning. Teachers are required to provide prompt
support to learners in this state.
C. Acquisition of Handwriting Log
When the electronic pen moves on the tablet screen, the
time, the position coordinates, the pressure, the altitude, and
Fig. 1. Method overview the azimuth are a sampled is acquired every 10 msec. In this
research, Apple Pencil is used as an electronic pen. The tip of
Figure 1 illustrates the method overview. In this research, the Apple Pencil is equipped with a special sensor packed with
handwriting learning is possible by using an electronic pen and the latest technology. The sensor detects position, pressure, and
a tablet device. Learners write freely when they solve questions. tilt. Pen movements during learning can be expressed by
Learners can answer while freely selecting three drawing tools. acquiring samples continuously. Position coordinates indicate
Drawing tools are black, red, and eraser. Learners can check where the tablet terminal is grounded. The position coordinates
hints and explanations when they judge that they cannot solve are based on the upper left of the tablet, with the x-axis to the
it by themselves. This is to prevent the learner from giving up right and the y-axis to the bottom. The pressure is the value of
the answer to the question. Handwriting data during the force that pushes the screen of the tablet. The average
handwriting learning can be acquired in detail by using an pressure is 1. The weaker you write, the closer the pressure
electronic pen and a tablet device. approaches 0. Altitude is the vertical angle between the
Writing data is two types: handwriting log and support electronic pen and the tablet. Altitude is 0 when the electronic
seeking log. The handwriting log is the pen information pen is laid flat on the surface of the tablet. Altitude is π/2 when
acquired when the electronic pen is touched to the tablet. Pen the electronic pen stands straight with a point on the tablet.
information is four types: position coordinates, pressure, Azimuth is the direction in which the electronic pen is tilted.
altitude, and azimuth. It is possible to express how the pen is Azimuth is obtained in radians. Azimuth is 0 when the
moving during learning by acquiring pen information each electronic pen is tilted to the side in the x-axis direction from
time the electronic pen moves on the tablet. Support seeking the center. Azimuth increases as it is rotated clockwise and
log is the record of checking hints and explanations during the decreases as it is rotated counterclockwise. The positive and
question exercise. Learner's solving behavior is estimated from negative values are inverted at the position rotated 180 degrees
these writing data. from the x-axis direction.

B. Characteristics in Each Solving Behavior Writing pressure, writing speed, and pen orientation are
effective indicators of cognitive load. Writing pressure is
calculated from the pressure. Writing speed is calculated from
the position coordinates and time. Pen orientation is expressed
from altitude and azimuth. Learner's solving behavior is
estimated from the three handwriting features that are effective
as indicators of cognitive load.
D. Acquisition of Support Seeking Log
Learners aim to be able to solve typical questions smoothly
through problem exercises. However, learner's hands may stop
during learning because of their lack of knowledge. In this
method, learners can check hints and explanations freely when
Fig. 2. Solving behavior they judge that they cannot solve it by themselves. If they still
do not understand, they check explanations and prepare to
In this study, we define solving behavior into 4 types. solve the next question. Learners can proceed with the question
Figure 2 shows the four types of solving behavior. Solving exercises without giving up.
behavior consists of two axes: smoothness and comprehension. The timing to check hints and explanations differs
The acquired state is the state in which learners are answering depending on the learner's understanding levels. In some cases,
questions correctly. It is assumed that the learners have learners finish the answer without checking hints. How learners
established the knowledge necessary to solve the question. seek support is important in estimating the learner's

55
understanding levels. In this method, we record the situation IV. ANALYSIS OF HANDWRITING SITUATION ON TABLET
that learners check hints and explanations. The state of
checking hints and the state of checking explanations are A. The System to Acquire Handwriting Data
recorded as different states. Hints and explanations are In this study, we implemented a learning application to
displayed in pop-ups. Learners cannot solve questions while acquire the handwriting data of learners working on math
browsing the hints and explanations. Learners need to close the question exercises. The application was created with swift. In
hints and explanations to return to the answer window. For this study, iPad and Apple Pencil are used to acquire the
example, if they want to dead copy the answer, they need to learner's handwritten log. Considering effective learning,
open explanations many times. learners should not be overloaded. In this study, the iPad has a
screen size of 12.9 inches, which is close to A4 paper. Learners
E. Estimation of Solving Behavior do not have the problem that the screen is small and difficult to
This method estimates learner's solving behavior from write. In this study, a paper-like protective film is attached to
handwriting logs and supports seeking logs. Learners behave the iPad screen. Learners can learn as if they were using paper
differently in each solving behavior. It is assumed that and a pen.
handwriting characteristics are also different. Handwriting
characteristics are writing pressure, writing speed, and pen
orientation However, these values vary from learner to learner.
The mean value and standard deviation are calculated and
standardized for each learner. Azimuth differs greatly between
right-handed and left-handed. The azimuth of left-handed
learners is converted to eliminate the influence of the dominant
hand. The analysis may consider individual differences in
handwriting logs.
In this method, the learner's solving behavior is labeled at
1-second intervals. In schools and cram schools, teachers often
walk around the class and check how learners are doing.
Teachers give guidance adapted to the individual learning
situation by looking at the learner's activities. It is easy to judge (a) Select question page (b) Answer page
the solving behavior by paying close attention to the learners. Fig. 3. Learning application
Learners are captured on videotape during the question
exercises to label their solving behavior. A screen of the tablet Figure 3 shows the implemented learning application. The
is recorded during the question exercises. This record shows application consists of two screens, the select question page,
what learners are writing during the question exercise. and the answer page. The answering page consists of three
Learner's solving behavior is judged by using these video data. parts: the exercise part, the button part, and the handwriting
part. Questions are displayed in the display question part. Some
Learner's solving behavior is estimated from the
buttons are installed in the button operation part. Some buttons
handwriting log and support seeking log for the past 30
are the submission button, timer, change tool button, hint
seconds using the moving window. Learner's solving behavior
button, and explanation button. At the end of the answer,
is predicted every second. This method creates a random forest
learners tap the submission button located to the left of the
model to estimate the solving behavior. Explanatory variables
timer. The handwriting data acquired during question exercises
are as follows.
and the image of the answer are saved on the tablet. Learners
Pressure Average: Average of writing pressure. can answer while freely selecting three drawing tools. Drawing
Pressure Maximum: Maximum of writing pressure. tools are black, red, and eraser. Learners can change the
drawing tools by tapping each button. The two buttons located
Writing Speed: Average of writing speed. on the right side of the button part show hints and explanations
Writing Time: Total time when the electronic pen is touched on pop-up windows Learners can write freely in the
Altitude: Average of altitude. handwriting answer part. Handwriting data is acquired only
when writing with Apple Pencil in the handwritten answer part.
Azimuth: Average of azimuth. Writing data is acquired when the Apple Pencil touches the
Hint Time: Total time when learners check hint. iPad, is moving on the iPad, and is away from the iPad. The
Explanation Time: Total time when learners check writing data acquired are 6 points: time, position coordinates,
explanation. pressure, altitude, azimuth, and selected drawing tool.
All explanatory variables are calculated from the writing B. Experiment Overview
data over the last 30 seconds. Pressure average, pressure An experiment was conducted to confirm whether the
maximum, writing speed, writing time, altitude, and azimuth method estimates the learner's solving behavior. The
are calculated for each drawing tool selected by learners. The experiment aims at collecting Handwriting data of learners
model is trained from 20 explanatory variables. An objective working on the exercises as a dataset and verifying a random
variable is set with a label attached by observing the learner. forest model generated from the dataset.

56
 Subjects are high school students in the second grade. They The datasets are divided into data for the model
worked on question exercises using the learning application construction and the verification. We adapted 4-fold cross-
implemented in this study. Questions are given from the range validation to verify the accuracy of the model. Table I shows
of the triangle ratio learned in the first grade of high school. the classification results by random forest. The Accuracy is
Subjects review the range that has already been learned. They 0.96. The accuracy of the random forest model varies with
were captured on videotape during the question exercises. A hyperparameters. Grid search is used to find the optimum
screen of the tablet was recorded during the question exercises. values of the hyperparameters. The grid search tries all
Subjects worked on 12 questions. The 12 questions consist of 3 hyperparameter combinations to attain the most accurate model.
sutra-copying questions, 1 practice question, 5 exercise The hyperparameters searched in this work are max_depth: the
questions, and 3 confirmation tests. All subjects are new to depth of decision tree, min_samples_leaf: the minimum
handwriting learning using a tablet terminal. They need to get number of samples required to compose a leaf, and
used to the operation. n_estimators: the number of decision trees that make up a
random forest.
First, they worked on three sutra-copying questions. Sutra-
TABLE II. HYPERPARAMETER SEARCH
copying question is a question format in which the learners
copy the displayed sentences. The three sutra-copying Parameters Combinations Value
problems are Japanese sentences, English sentences, and
max_depth [5,9,13,17,21,25,29,33,37] 25
mathematical formulas. Next, subjects worked on a practice
question to understand how to use the learning application. min_samples_leaf [1,2,3,4,5,6,7] 1
When they worked on a practice question, we explained three n_estimatiors [100,200,300,500,1000,2000] 300
methods: how to select a drawing tool, how to display hints
and explanations, and how to submit answers. Once Table II shows the combinations of hyperparameter values
accustomed to the operation, subjects worked on five exercise tried in the grid search and the optimum combinations. The
questions. random forest model provides the most accurate prediction
when the tree depth, the minimum number of samples, and the
The time to work on the exercise questions was set to 40 number of trees are set to 25, 1, and 300, respectively. If trees
minutes. It is necessary for subjects to consider how much time are too deep, the model causes overfitting. The constraint on
they would spend on each question. Three additional questions the depth of trees to 25 prevents overfitting. Similarly, too
are prepared for the subjects who have completed 5 exercise many trees lead the model to overfit. The number of the fewest
questions before 40 minutes have passed. Subjects were samples in each leaf is 1, which is the minimum value. It
instructed to solve five exercises in 40 minutes. The additional would be a result of the small data size used for learning. A
three questions are out of the scope of this analysis. Next, small classifier of this size can be installed on a small computer.
Subjects worked on three confirmation tests. They cannot
check hints and explanations in the confirmation tests. It is V. CONCLUSION
used as an index to measure the learner's understanding levels We proposed a method to estimate the learner's solving
after working on the exercise. A model for estimating the behavior from the writing data working on the exercise on the
learner's solving behavior is constructed from the writing data tablet. Since the learner's solving behavior can be estimated
of five exercise questions. A random forest is used to build the online, teachers can give feedback to the learner quickly.
model and extract the behavioral features. Through experiments to verify this method, the learner's
C. Result solving behavior was estimated with high accuracy. Writing
characteristics in each solving state were extracted by looking
A random forest model was constructed to estimate the at the important variables in a random forest model.
learner's solving behavior. We used the written data from the
five exercise questions to build the model. The writing data of REFERENCES
the five subjects were incomplete. The explanatory variables [1] Aurélien, Géron. "Hands-on machine learning with scikit-learn &
and objective variables used to train the models are twenty tensorflow." Geron Aurelien (2017).
features explained and labeled solving behavior of the learners, [2] R. Konishi, R. Shimmura, K. Nakajima, R. Hayashi, and T. Sato, “A
respectively. The learner's solving behavior is labeled manually study on inconscient learning behaviors and psychological evaluations
based on the learner's state captured on the video camera and using biological data for an image-based vocabulary building
application,” in E-Learn: World Conference on E-Learning in Corporate,
the result of the confirmation tests. Government, Healthcare, and Higher Education. Association for the
TABLE I. RESULT OF SOLVING BEHAVIOR Advancement of Computing in Education (AACE), 2017, pp. 798– 803.
[3] J. Grafsgaard, J. Wiggins, K. E. Boyer, E. Wiebe, and J. Lester,
precision recall f1-score “Predicting learning and affect from multimodal data streams in task-
oriented tutorial dialogue,” in Educational Data Mining 2014, 2014.
Misunderstanding 0.99 0.91 0.95
[4] Herodotou, Christothea. "Young children and tablets: A systematic
Knowledge less 0.91 0.99 0.95 review of effects on learning and development." Journal of Computer
Assisted Learning 34.1 (2018): 1-9
Acquired 0.98 0,98 0.98
[5] Read, Janet C. "A study of the usability of handwriting recognition for
Studying 1.00 0.94 0.97 text entry by children." Interacting with Computers 19.1 (2007): 57-69.

57
[6] Mann, Anne-Marie, Uta Hinrichs, and Aaron Quigley. "Children's [9] Smoker, Timothy J., Carrie E. Murphy, and Alison K. Rockwell.
Creativity Lab: Creating a'Pen of the Future'." Proceedings of the 2014 "Comparing memory for handwriting versus typing." Proceedings of the
Workshops on Advances in Computer Entertainment Conference. 2014. Human Factors and Ergonomics Society Annual Meeting. Vol. 53. No.
[7] K. Yu, J. Epps, and F. Chen, “Cognitive load evaluation of handwriting 22. Sage CA: Los Angeles, CA: SAGE Publications, 2009.
using stroke-level features,” in Proceedings of the 16th international [10] Longcamp, Marieke, Marie-Thérèse Zerbato-Poudou, and Jean-Luc
conference on Intelligent user interfaces, 2011, pp. 423–426. Velay. "The influence of writing practice on letter recognition in
[8] Yu, Kun, Julien Epps, and Fang Chen. "Cognitive load measurement preschool children: A comparison between handwriting and typing."
with pen orientation and pressure." Proceedings of MMCogEmS (2011). Acta psychologica 119.1 (2005): 67-79.

58
 2021 the 13th International Conference on Computer and Automation Engineering

Review on Low Frequency Control of Quasi-Zero

Stiffness
Bu-yun Li1 ,2 Chang-geng Shuai1,2 Wei Xu1,2 Zhao-hao Yang1,2
1 1 1 1
Institute of Noise & vibration, Institute of Noise & vibration, Institute of Noise & vibration, Institute of Noise & vibration,
Naval University of Engineering Naval University of Engineering Naval University of Engineering Naval University of Engineering
2 2 2 2
National Key Laboratory on National Key Laboratory on National Key Laboratory on National Key Laboratory on
Ship Vibration & Noise Ship Vibration & Noise Ship Vibration & Noise Ship Vibration & Noise
Wuhan, PRC Wuhan, PRC Wuhan, PRC Wuhan, PRC
[email protected] [email protected] [email protected] [email protected]

Abstract—Quasi-zero stiffness (QZS) vibration isolation system. As the minimum stiffness of the system does not reach
system is composed of positive stiffness structure and negative zero stiffness, the system is in the state of high static low
stiffness structure. It can obtain high static stiffness and low dynamic stiffness [5].
dynamic stiffness near the static equilibrium position. The
benefits of the QZS technology are to realize the low frequency III. RECENT RESEARCH PROGRESS
vibration control and solve the contradiction of low natural
The concept of QZS was first proposed by P. Alabuzhev in
frequency and small static deformation. In this paper, the
the 1980s. With nearly 40 years of development, the QZS
principle of QZS and the research progress in recent years are
introduced, and the outlooks are also proposed. theory is gradually improved, and a variety of QZS isolators
have been designed and manufactured. However, the bearing
Keywords—Quasi-zero stiffness, Low frequency control, capacity of the most QZS isolators is not adjustable. It is easy
Vibration isolation system, High-static-low-dynamic stiffness for them to lose stability under large-scale excitation. And
most of them can only achieve one-direction vibration isolation.
I. INTRODUCTION In recent years, the researchers focus mainly on the
Researches have shown that it is difficult to further reduce improvement of QZS nonlinear dynamic theory and design
the natural frequency of the vibration isolation system by new QZS isolators that can be adjustable, have the ability to
reducing the stiffness of the vibration isolator under the keep stability under large amplitude excitation and realize
condition of limited space [1]-[3]. The quasi zero stiffness multi-directional vibration isolation.
(QZS) vibration isolation system is an effective and feasible A. Improvement of QZS Nonlinear Dynamics Theory
means to solve the problem. The QZS vibration isolation
The improvement of QZS nonlinear dynamic theory in
system is a kind of nonlinear vibration isolation system with
recent years can be generally classified into two categories: one
positive and negative stiffness, which can effectively solve the
is to enrich the solution methods of QZS nonlinear dynamic
contradiction between extremely low natural frequency and
equation, such as harmonic balance method, average method,
small static deformation [4].
perturbation method, and etc.; the second is to analyze the
II. QUASI-ZERO STIFFNESS THEORY nonlinear characteristics of QZS system, such as stability, jump
phenomenon, bifurcation characteristics, chaos analysis and etc.
Quasi-zero stiffness (QZS) vibration isolation system is a [6]-[11].
kind of positive-passive stiffness parallel structure, which can
obtain high static stiffness and low dynamic stiffness near the
static equilibrium position. It can support large static load with
small static displacement and keep low dynamic stiffness near
the static equilibrium position. QZS is usually achieved by Positive stiffness
connecting elastic elements in the horizontal direction to form structure
a catastrophe structure. Fig. 1 is the basic principle diagram of
QZS realization. Theoretically, the QZS technology can Quasi-zero stiffness realization
achieve vibration isolation of all frequencies and realize low
frequency vibration control for the mechanical system.
However, it is difficult to realize in practical engineering, as
the load is not perfect or the stiffness is not perfect. The
imperfect load means that the static equilibrium point is not the
lowest point of stiffness, while the imperfect stiffness means Negative stiffness
structure
that the negative stiffness only partially offsets the positive
stiffness. These conditions can make the minimum dynamic Fig. 1. The principle of quasi-zero stiffness realization.
stiffness fail to be zero, but still less than the stiffness of linear

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 59

 Carrella [12]-[13] described the dynamic model of QZS according to the load, so that the system can be in an ideal state
isolator with Duffing equation, and used harmonic balance of QZS under static load. Xu [30] developed a buckling plate
method to analyze its transmissibility characteristics. Peng type QZS vibration isolator. The QZS structure consists of an
Xian [14] analyzed the primary resonance response and elastic plate with negative stiffness and an elastic component
stability of the link spring type QZS vibration isolation system with positive stiffness. The elastic plate is nested in an
by L-P perturbation method. Yang [15] used the average adjusting hoop, which can actively preset the QZS state of the
method to analyze the sub-harmonic phenomenon of the QZS system to make it suitable for different vibration isolation
nonlinear system, and evaluated the vibration isolation objects. Shi [31] designed a QZS vibration isolator with
performance of the system through the power flow adjustable magnetorheological materials. The vibration isolator
transmissibility. Kovaci [16]-[17] optimized the structural consists of a slotted disc spring and a magnetorheological
parameters of the three spring QZS vibration isolation system elastomer with smaller stiffness. The magnetorheological and
and studied its bifurcation and chaos characteristics. Han magneto rheological effects of smart materials can be used to
Yanwei [18] established two geometrically QZS nonlinear realize stepless stiffness adjustment. Yang [32] designed a
systems, and studied their complex dynamic characteristics, magnetic QZS vibration isolator with adjustable stiffness. The
such as high dimension bifurcation, global behavior, chaotic rectangular permanent magnet is used to generate negative
characteristics, main resonance response and stability. Hao [19] stiffness, and the air spring provides positive stiffness. Zhou
studied the bifurcation characteristics and multi solution [33] designed a magnetic vibration isolator with adjustable
coexistence phenomenon of QZS vibration isolation system. stiffness by replacing permanent magnet with electromagnet.
Lu [20] researched double-layer high static and low dynamic By adjusting the current, the magnetic strength and polarity of
stiffness vibration isolation system to overcome the problems the electromagnet were changed to adjust the system
of large static deformation and small attenuation rate of high- parameters. The desired low-frequency vibration isolation
frequency vibration. Gatti [21]-[23] studied the dynamic performance was obtained by semi-active control. Zhao [34]
behavior of a two degree of freedom QZS vibration isolation developed a pneumatic spring type QZS vibration isolator with
system under harmonic excitation. Its amplitude frequency adjustable load. The system can be in the optimal vibration
characteristic equation was obtained by using the average isolation state as the bearing mass changes by adjusting the air
method, and the stability of the main resonance response was pressure. Ren [35] selected the air spring as the vibration
analyzed. Wang [24] analyzed the dynamic characteristics of isolation element, designed and studied a new type of air spring
two degree of freedom QZS vibration isolation systems, and QZS vibration isolator with adjustable bearing capacity. The
compared the vibration isolation performance of different system parameters can be adjusted by changing the air pressure
systems by dynamic displacement peak value, transmissibility of the air spring, so that the vibration isolation system can still
peak value and effective vibration isolation frequency band. achieve great QZS characteristics as the bearing mass changes
Navazi [25] analyzed the vibration isolation performance and greatly.
stability of an unbalanced rotor system by the multi-scale
method. Ho [26] analyzed and designed a high static and low
dynamic stiffness vibration isolation system with nonlinear
damping by using the method of output frequency response
function. Tang [27] studied the impact resistance of high static
and low dynamic stiffness vibration isolation system, and
found that under the premise of small impact, the impact
resistance performance is directly proportional to the
nonlinearity of the system, and the excitation amplitude is
inversely proportional.
B. Adjustable QZS Isolators
In the practical application, it is necessary to adjust the Fig. 2. QZS isolator with adjustable mechanical structure.
positive stiffness and negative stiffness to make the system
reach the best QZS state. There are roughly three ways to
adjust the positive stiffness and negative stiffness of QZS
isolator, namely using adjustable mechanical structure (shown
as Fig. 2), adjustable aerodynamic force (shown as Fig. 3) and
adjustable electromagnetic force (shown as Fig. 4).
Shi [28] has designed a multi-dimensional adjustable QZS
vibration isolation platform. The three damping springs at the
upper end and the three damping springs at the lower end are
parallel connected as negative stiffness components and the
vertical springs worked as positive stiffness. The initial
position of the QZS isolators can be adjusted by mechanical
structure to realize perfect QZS characteristics. Cao [29]
designed a kind of vibration isolation device with adjustable
Fig. 3. QZS isolator with tunable electromagnetic force.
negative stiffness. It can flexibly adjust the negative stiffness

60
 to add the vibration isolation dimensions by structural design
(shown as Fig. 8).
Li [40] designed a QZS vibration isolation system
composed of disc spring and European pull rod. The device is
decoupled from each other in vertical and horizontal directions.
It has good vibration isolation ability in two directions. Zhu [41]
established a 6-DOF magnetic levitation QZS vibration
isolation system with magnets. It can achieve good QZS
characteristics at six degrees of freedom. Sun [42] designed a
multi direction QZS vibration isolator based on three springs. It
has good vibration isolation effect in horizontal, vertical and
transverse directions. Zheng [43] designed a six degrees of
Fig. 4. QZS isolator with tunable aerodynamic force. freedom QZS vibration isolation system. It is achieved by
parallel connection of multiple (> 3) three degrees of freedom
C. Stable QZS Isolator Design under Large Excitation
QZS isolators.
The QZS system can lose stability under large excitation.
Therefore, it is quite important to keep the system stable under
great rouse. At present, the structural optimization design for
QZS system is often used to solve the problem. For example,
detached structure is designed to make the negative stiffness
structure and the positive stiffness structure separated under
large amplitude excitation, which makes the system degenerate
into a linear system (shown as Fig. 5) and also self-locking
structure (shown as Fig. 6) is adopted to keep the QZS system
stable under large excitation. Fig. 5. QZS isolator with gear-rack detached structures to avoid instability.
Ledezma Ramirez [36] designed a shock isolation system
with high static and low dynamic stiffness characteristics by
electromagnets and permanent magnets. Xu [37] designed a
QZS system for automobile suspension, which is composed of
negative stiffness structure, air spring and gear rack. The
overall stiffness and natural frequency of the body are reduced
and the ride comfort of the whole vehicle is improved. As the
body amplitude is too large, the rack and pinion do not engage
and the negative stiffness structure stops working, thus
avoiding the instability caused by the negative stiffness
structure. Wen [38] designed a folding QZS vibration isolation Fig. 6. QZS isolator with self-lock structure to avoid instability.
system. The spring provides negative stiffness, and the folding
components provide positive stiffness. The parallel connection
of positive and negative stiffness structure makes the total
stiffness of the system zero, and the displacement amplification
effect of the first folding component and the second folding
component enables the system to enter negative stiffness under
a small displacement excitation. As the excitation is too large
and the folding angle between the upper folding plate and the
lower folding plate reaches a certain value, the system would
be in the self-locking state. Therefore, the folding QZS
vibration isolation system becomes a rigid structure to keep the Fig. 7. Two-dimensional vibration isolator.
system stable. Yang [39] designed a tensile QZS vibration
isolator. The vibration isolator is composed of a negative
stiffness structure in parallel with a positive stiffness main
spring. The negative stiffness structure is composed of a tensile
spring, a connecting rod, a slider and a guide rail, which can
produce negative stiffness in the vertical direction and avoid
the instability under large excitation.
D. Multi-directional QZS Vibration Isolator
In many cases, it is necessary to realize multi-directional
QZS. There are roughly two ways to realize the characteristics,
that is to parallel different QZS isolators (shown as Fig. 7) and Fig. 8. Six-dimensional vibration isolator.

61
 IV. REMAINED PROBLEMS [5] Chai Kai, Study on the dynamic characteristics and control techniques of
the vibration isolation system based on high-static-low-dynamic-
Although many engineering application problems of QZS stiffness, Doctoral Thesis, Naval University of Engineering, 2018.
vibration isolation technology has been solved, the following [6] Q. Cao, M. Wiercigroch, E. Pavlovskaia, C. Grebogi and J. M. T.
problems still need to be further discussed and researched: Thompson, “The limit case response of the archetypal oscillator for
smooth and discontinuous dynamics,” International Journal of Non-
(1) The nonlinear dynamic analysis of QZS technology Linear Mechanics, vol. 43, no. 6, pp. 462-473, January 2008.
should be further improved. As the system parameter is not [7] R. Tian, Q. Cao and S. Yang, “The codimension-two bifurcation for the
matched and the system is excited by complicated forces, the recent proposed SD oscillator,” Nonlinear Dynamics, vol. 59, no. 1-2,
nonlinear dynamic behavior of QZS system should be further pp. 19, April 2010.
investigated. [8] Y. Wang, S. Li, C. Cheng, and X. Jiang, “Dynamic analysis of a high-
static-low-dynamic-stiffness vibration isolator with time-delayed
(2) The QZS isolator should be further studied in feedback control,” Shock and Vibration, vol. 2015(PT.2), pp. 1-19,
miniaturization and compact design to adapt to the narrow March 2015.
space environment, such as ship environment. The negative [9] P. Malatkar, and A.H. Nayfeh, “Calculation of the jump frequencies in
stiffness structures are often helical spring, diagonal link or the response of s.d.o.f. non-linear systems,” Journal of Sound and
buckling beam, which takes up a large space; Vibration, vol. 254, n. 5, pp. 1005-1011, August 2002.
[10] C. Cheng, S. Li, Y. Wang, and X. Jiang, “On the analysis of a high-
(3) The bearing capacity of QZS system need to be static-low-dynamic stiffness vibration isolator with time-delayed cubic
improved. At present, most of the QZS vibration isolation displacement feedback,” Journal of Sound and Vibration, vol. 378, pp.
systems has low bearing capacity due to the limited 76-91, May 2016.
deformation of positive stiffness structure; [11] X. Sun, J. Xu, X. Jing, and L. Cheng, “Beneficial performance of a
quasi-zero-stiffness vibration isolator with time-delayed active control,”
(4) On-line intelligent adjustment technology of positive International Journal of Mechanical Sciences, vol. 82, pp. 32-40, March
and negative stiffness of QZS system should be researched. As 2014.
the environment changes dramatically, such as load changes, [12] A. Carrella, M. Brennan, T. Waters, and V. Lopes, “Force and
the system can not keep good QZS state. It needs on-line displacement transmissibility of a nonlinear isolator with high-static-
intelligent adjustment of the positive and negative stiffness to low-dynamic-stiffness,” International Journal of Mechanical Sciences,
vol. 55, n. 1, pp. 22-29, November 2012.
keep great QZS characteristics.
[13] A. Carrella, Passive vibration isolators with high-static-low-dynamic-
V. CONCLUSION AND OUTLOOKS stiffness, Doctoral Thesis, University of Southampton, 2008.
[14] P. Xian and Z. Shi, “Nonlinear Resonance Response Analysis of a Kind
This paper summarizes the development and research of Passive Isolation Sy stem with Quasi-zero Stiffness,” Journal of
hotspots of QZS in recent years, and puts forward remaining Hunan University (Natural Sciences), vol. 38, n. 8, pp. 21-28, August
problems of QZS researches. Although the QZS vibration 2011.
isolator technology has achieved certain engineering [15] J. Yang, Y. Xiong, and J. Xing, “Dynamics and power flow behavior of
application and realize the low frequency control, but its a nonlinear vibration isolation system with a negative stiffness
application range is not wide due to the remained limits. The mechanism,” Journal of Sound and Vibration, vol. 332, n. 1, pp. 167-
183, August 2012.
next research directions can be
[16] I. Kovacic, M. Brennan, and T. Waters, “A study of a nonlinear
(1) Nonlinear dynamic analysis of QZS system under vibration isolator with a quasi-zero stiffness characteristic,” Journal of
multi-point, multi-frequency excitation and parameter Sound and Vibration, vol. 315, n. 3, pp. 700-711, December 2008.
mismatch situation. [17] I. Kovacic, M. Brennan, and B. Lineton, “Effect of a static force on the
dynamic behaviour of a harmonically excited quasi-zero stiffness
(2) The new QZS system that has the characteristics of system,” Journal of Sound and Vibration, vol. 325, n. 4-5, pp. 870-883,
large load bearing, high reliability, and being suitable for March 2009.
narrow space. [18] Han Yanwei, Nonlinear dynamics of a class of geometrical nonlinear
system and its application, Doctoral Thesis, Harbin Institute of
(3) On-line intelligent adjustable positive-negative stiffness Technology, 2015.
control method. [19] Hao Zhifeng, Research on dynamics and vibration control based upon
high-performance low-frequency isolator, Doctoral Thesis, Harbin
ACKNOWLEDGMENT Institute of Technology, 2016.
[20] Z. Lu, M. Brennan, T. Yang, X. Li, and Z. Liu, “An investigation of a
The work is supported by the National Defense Research
two-stage nonlinear vibration isolation system,” Journal of Sound and
Project of China. (Grant No.HJ2019C020499) Vibration, vol. 332, n. 6, pp. 1456-1464, November 2013.
[21] G. Gatti, I. Kovacic, and M. Brennan, “On the response of a
REFERENCES
harmonically excited two degree-of-freedom system consisting of a
[1] A.H. Nayfeh and D.T. Mook, Nonlinear Oscillations, Wiley, New York, linear and a nonlinear quasi-zero stiffness oscillator,” Journal of Sound
1979. and Vibration, vol. 329, n. 10, pp. 1823-1835, November 2010.
[2] A.H Nayfeh and D.T.Mook, Nonlinear Oscillations, John Wiley and [22] G. Gatti, M.Brennan, and I. Kovacic, “On the interaction of the
Sons, II Edn., 1995. responses at the resonance frequencies of a nonlinear two degrees-of-
[3] A.H. Nayfeh, Perturbation Methods, John Wiley and Sons, 1973. freedom system,” Physica D Nonlinear Phenomena, vol. 239, n. 10, pp.
591-599, January 2010.
[4] P. Alabuzhev and A. Gritchin, Vibration Protecting and Measuring
Systems with Quasi-Zero Stiffness, Hemisphere Publishing, New York, [23] G. Gatti and M. Brennan, “On the effects of system parameters on the
1989. response of a harmonically excited system consisting of weakly coupled
nonlinear and linear oscillators,” Journal of Sound and Vibration, vol.
330, n. 18-19, pp. 4538-4550, April 2011.

62
[24] Y. Wang, S. Li, A. Simon, and J. Zheng, “Comparison of the dynamic [33] N. Zhou, and K. Liu, “A tunable high-static-low-dynamic stiffness
performance of nonlinear one and two degree-of-freedom vibration vibration isolator,” Journal of Sound and Vibration. vol. 329, n. 9, pp.
isolators with quasi-zero stiffness,” Nonlinear Dynamics, December 1254-1273, November 2010.
2017. [34] D. Xu, Z. Zhao and J. Zhou, “Design and analysis of an adjustable
[25] M. Hojjati and H. Navazi, “Nonlinear vibrations and stability analysis of pneumatic vibration isolator with quasi-zero-stiffness characteristics,”
a rotor on high-static-low-dynamic-stiffness supports using method of Journal of Hunan University (Natural Sciences), vol. 40, n. 6, pp. 47-52,
multiple scales,” Aerospace Science and technology, vol. 63, pp. 259- June 2013.
265, January 2017. [35] Ren Xudong, Characteristics analysis and application study of the quasi-
[26] C. Ho, Z. Lang, and S. Billings, “Design of vibration isolators by zero stiffness isolator using air spring, Doctoral Thesis, Academy of
exploiting the beneficial effects of stiffness and damping nonlinearities,” Military Medical Sciences, 2017.
Journal of Sound and Vibration, vol. 333, n. 12, pp. 2489-2504, [36] F. Ledezma, D. Ferguson, M. Brennan, and B. Tang, “An experimental
February 2014. nonlinear low dynamic stiffness device for shock isolation,” Journal of
[27] B. Tang, and M. Brennan, “On the shock performance of a nonlinear Sound and Vibration. vol. 347, pp. 1-13, February 2015.
vibration isolator with high-static-low-dynamic-stiffness,” International [37] Xu Xing, “A quasi zero stiffness vibration isolation system and vehicle,”
Journal of Mechanical Sciences, vol. 81, pp. 207-214, February 2014. China Patent No. 201910227755.8, July 2019.
[28] Shi Peicheng, “Multi-dimensional adjustable quasi zero stiffness [38] Wen Guilin, “A folding quasi zero stiffness vibration isolation device,”
vibration isolation platform,” China Patent No. 201610545258.9, August China Patent No. 201910664265.4, October 2019.
2018.
[39] Yang Xiaoxiang, “Tensile quasi zero stiffness vibration isolator,” China
[29] Cao Hao, “A negative stiffness adjustable zero stiffness vibration Patent No. 201620464907.8, December 2016.
isolation device and its application method,” China Patent No.
[40] Li Yinshuai, “A quasi zero stiffness isolation system,” China Patent No.
201910816951.9, October 2019.
201811494664.2, April 2019.
[30] Xu Daolin, “Buckling plate type quasi zero stiffness vibration isolator,”
[41] T. Zhu, B. Cazzolato, W. Robertson, and A. Zander, “Vibration isolation
China Patent No. 201310330357.1, February 2015.
using six degree-of-freedom quasi-zero stiffness magnetic levitation,”
[31] Shi Xiaotian, “A vibration isolator based on quasi zero stiffness,” China Journal of Sound and Vibration, vol. 358, July 2015.
Patent No. 201811048587.8, December 2019.
[42] X. Sun and X. Jing, “Multi-direction vibration isolation with quasi-zero
[32] Yang Qingchao, “A magnetic quasi zero stiffness isolator with stiffness by employing geometrical nonlinearity,” Mechanical Systems
adjustable stiffness,” China Patent No. 201611039718 .7, November and Signal Processing, vol. 62-63, pp. 149-163, January 2015.
2016.
[43] Zheng Junruoyu, “6-DOF quasi zero stiffness vibration isolation device,”
China Patent No. 201810769687.3, November 2018.

63
 2021 the 13th International Conference on Computer and Automation Engineering

Non-invasive Smartphone Use Monitoring to Assess

Cognitive Impairment
Nguyen Dinh Thang Giles Oatley Andrew Stranieri Darren Walker
Federation University Federation University Federation University Central Queensland University
University Drive, Mt Helen 100 Clyde Road, Berwick University Drive, Mt Helen Cairns Square
VIC, Australia 3350 VIC, Australia 3806 VIC, Australia 3350 Cairns, QLD, Australia 4870
+61411222027 +61351226076 +61353279283 +61740375122
[email protected] [email protected] [email protected] [email protected]
ation.edu.au

Abstract—Background: There are many tests for the early Studies that have recently appeared aim to create
detection of Mild Cognitive Impairment (MCI) to prevent or delay applications for clinical purposes such as the Standardized
the development of dementia, particularly amongst the elderly. Touchscreen Assessment of Cognition (STAC) [5]. This
However, many tests are complex and are required to be application requires patients to complete computerized
performed repeatedly. Cognitive assessment apps for a versions of a test at a healthcare center. This is beneficial to
smartphone have emerged, but like other tests, they require the healthcare professionals because it reduces clinical processing
user to perform complex tasks like drawing time on a clock. Few time in administering a test. Another approach involves users
studies have explored non-invasive ways of tracking and assessing performing a test while they are playing a game such as
MCI without having the user perform specific tests. Objective:
WarCat [6]. This game monitors the loss of players after the
This research ultimately aims to develop an app that runs in the
background and collects smartphone usage data that correlates
game is finished and uses a “lock picking” framework to
well with MCI test results. The focus of this preliminary study was analyze data and carry out a result. However, these
to develop an app that collects usage data and common MCI applications require that users perform tasks on devices and, as
questionnaires to see if usage data between people varied, and to such can be said to be invasive.
establish associations between phone usage and cognitive tests The current study proposes the use of a smartphone to
results. Method: An android application was developed to gather determine the same cognitive assessment non-invasively as
data over three weeks by three volunteers (authors). Usage data
that gathered by formal cognitive assessments. The assumption
collected included: SMS and call log, accelerometer, location, app
is that as smartphones become widely adopted, a feasible
usage, self-report. Cognitive tests implemented were Stroop,
Go/No Go tests and absentmindedness questionnaires. Due to the
indicator of an individual’s cognitive ability could be detected
small sample size and Covid-19 restrictions (October 2020), non-invasively by a change in the way the use of their
location data was not reliable. SMS text was not collected for smartphone changes. In this research, there will be a
privacy reasons. Results: App categories can differentiate people, correlational analysis between cognitive impairment
but the app usage cannot be used to distinguish people. questionnaires and the app usage. The app usage data will be
monitored to see if the questionnaire can be answered and
Keywords—Cognitive impairment, smartphone usage, cognitive there are any changes in the smartphone use which is
assessment, dementia associated with to results of the cognitive impairment
questionnaires. Ultimately data can be collected about the
I. INTRODUCTION patterns of smartphone use that correlate with cognitive
The ageing population is increasing which leads to impairment results thereby realizing a non-invasive test
additional burden on carers and governments as many elderly routinely, without having the person go to a test center or
are cognitively impaired[1]. The quality of life of dementia perform a specific MCI test. An app was developed that
patients will be enhanced and socioeconomic costs reduced gathered relevant information in three categories:
and the progression of dementia is slowed[2]. Therefore, - Device attributes: SMS log, call log, accelerometer,
having an early detection of cognitive decline can lead to location, and App use
interventions that delay the onset of this debilitating disease
and reduce its burden [3]. - Self-report questionnaires. Free text entered by the
participant prompted to describe how they currently felt
Current cognitive assessment tools require that patients
perform tasks that can feel uncomfortable and are tedious to - Two cognitive tests: Go/No go test [7], Stroop test [8]
repeat routinely. The Montreal Cognitive Assessment (MoCA)
Data from the 3 categories were analysed to identify the
requires participants to draw a clock, draw a cube and recall
most important and feasible features that might be gathered on
words [4]. This test costs time and repeating it over days or
smartphone usage that may have predictive validity for
weeks leads to difficulties not only for the patient but also for
cognitive decline.
healthcare professionals administering the test.

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 64

 II. METHODOLOGY Two mindfulness mobile applications were recommended
A. App Design in the app supplied with download links.
The app is built to run on the Android mobile platform, SMS text and Call log would normally not be collected in a
written in Java using the Android Studio IDE. published app because of privacy violations. Also gathering
data constantly in background drains battery, reduces the
1) Device attributes performance of the phone and can consumes cellular data.
SMS and Call Log. The variables collected from SMS log
are phone numbers, type of message (receive or send), seen B. Data Gathering
and read status, receive date time, and sent date time. The The data was collected over three weeks from three
result can be used to build a view of how users communicate to participants who were authors of this article. There were 3
others in term of frequency of messaging. Another purpose is stages of recording. The phone features gathered data in the
to see whether users saw but forgot to read the message. The background for 3 weeks and user answered questionnaires and
variables collected from Call log are phone number, type of completed cognitive tests every day. They were instructed do
call (missed, incoming, outgoing), call date and the duration. this over two sessions: one around 9am and one in the evening.
These variables can help to know the frequency of call activity, Stage 1, 2 and 3 respectively occurred the first, second and third
the most connected contacts, which is related to social week. The purpose of dividing the timeline into 3 stages is to
communication. [9] collect and assess data before, during and after the rehabilitation.
Accelerometer: The x, y and z coordinates are recorded Figure 1 illustrates the data collection schedule.
which help to identify the force of movement. This data in The data was collected from three subjects, labelled with
conjunction with location can predict if a person is moving age (S28, S53 and S61). Data was collected during October
(running or walking) routinely or if they are engaged in other 2020 in Victoria, a time when the Covid-19 restrictions were in
activities. place that restricted geographical movement.
Location: The location and the date time is gathered. There were some difficulties in data gathering because of
Using spatiotemporal data it is possible to develop profiles, for Covid-19 pandemic and technical problems associated with the
instance identifying the most visited places and how long a application. Call Log: The location data was limited due to
person spends at that location. It can be used to predict if a Covid-19 restrictions on movement. The application was not
person’s frequency of social communication is decreasing or compatible with Android 7.1.1 the Color OS version 3.4. Thus,
not Moreover, with the use of accelerometer and location, this S53’s data was not collected reliably.
data can be used to detect the movement and behavior of user
[10] III. DATA ANALYSIS
A. Location
App Usage: This is the most important data that is used to
The location plot is clearly different between the 3 subjects.
categorise people and can link to answers in the self-report
S61 visited the fewest places and had the shortest traveling
questionnaires. The data gathered includes the date time, usage
distance. S28 visited places more than the other two and had
duration and app name. Profiling and prediction uses include
the longest traveling distance. This was consistent with their
measuring any changes in app usage, which could be related to
self-reported movement estimates. S28 travelled by motorcycle
personal lifestyle [11] or cognitive decline.
often between locations around Melbourne, mainly for
Keystroke tracking: This feature tracks the amount of employment. S53 worked from home and travelled to
backspace errors made, and sentiment analysis can also be neighboring subjects for social reasons. S61 was on leave from
carried out as the user is answering a long-answer question. work but stayed mostly at home and walked to his mother’s
Evaluating the use of backspace can help in detecting mood house daily, nearby. The oldest subject traveled the least and
disturbance [12] which can be associated with cognitive the youngest travelled the most.
impairment.
2) Questionnaires and tests
Three sets of questionnaires (General, Absent-mindedness,
and Real-life Absent-mindedness) were implemented in the app
to daily collect the phone use lifestyle and psychological issues
(absent-mindedness). These set of questionnaires and the
parallel app use will be combined to determine if these
questionnaires can be applied in the mobile phone.
The Go/NoGo test requires users to react as quickly as
possible to a positive signal and not at all to a negative signal.
The reaction time is captured by the app. The Stroop test
presents labels of colors that are sometimes dissonant in that the
word GREEN is displayed in a font other than green. The user
is asked to select the color depicted by the label not the font. A
dissonant font interferes with normal cognitive processing so
reaction times and error rates are greater with cognitive decline. Fig. 1. Data gathering schedule, in-app testing and data sample

65
B. Accelerometer E. Questionnaire
The result of the accelerometer data was similar for the Three questionnaire sets are:
three subjects. The amplitude of S61, S53 and S28 are 30, 20
and 80, respectively. Thus, the amplitude is obviously not - General questionnaire (GQ) (SUQ-G1 to SUQ-G10)
related to age. However, the frequency of movement shows - Absent-minded questionnaire (AQ) (SUQ-A1 to SUQ-
that S28 moved the phone more than S53 and S61. A10)
C. App Usage - Real-life absent-minded questionnaire (RAQ) (RH1 to
After 12 days collecting data, we categorized the apps used RH12)
into 4 main categories, Communication, Video, Tools, News S28 has highest average score (6.23/7) compared with S53
and Social as follows: (4.81/7) and S61 (2.88/7) in GQ. The order is the same in AQ
Communication: Samsung Internet, Messenger, Call, with score of 2.73, 2.62 and 2.47 which respectively belongs to
Gmail, Messages, Chrome, Zoom, Whatsapp, Outlook, Teams, S28, S53 and S61. In term of RAQ, it seems that this set of
Skype, Wechat. questions is not related to aging which is scored at 3.42/7 by
S28, 2/7 by S53 and 4.08/7 by S61.
Video player: VLC, Youtube
The average score for each question differed between S28
Tools: Google, Gallery, Clock, Tether, Translate, and S61 and there are some intersections between S61 and S53.
Calculator, Voice, Recorder, My Optus, Flashlight, Photo, Thus, there can be said to be some association between phone
Compass, Gboard usage and absent-minded between S61 and S53
News & magazine: Social: Facebook, Instagram F. Keystroke Test
S61 and S53 spent a higher proportion of their phone use The data result in number of backspaces for S61 had 178
time on Communication and News while S28 mostly used backspaces in total of 443 words (40.18%) and S53 had 65
Tool, Video player and Social apps. backspaces in total of 247 words (26.32%) while S28 made
508 backspaces in total of 551 words (92.19%). The location
Usage in the Communication categories had the greatest when user answer question is mostly at home. As described
variability over time; the app usage difference fluctuated and above, text was not collected so sentiment analysis was not
changed each day, thus there is no evidence that the usage time performed.
is associated with age for our participants. There is no
news/magazines app usage for S28 and the usage time of S53 G. SMS and Call Log
and S61 fluctuated. Due to privacy, SMS and call log data was not completely
A stereotype that young people tend to use social apps recorded. Moreover, the “read” status variable cannot be
more than older adults seemed to be confirmed. One hundred retrieved in the system SMS log. Therefore, we cannot
100% of the S28’s usage time over a 10-day period compared determine whether users forgot to read or skipped messages.
with no social app usage recorded by S61 over the same period.
IV. DISCUSSION
There was no clear difference between the three subjects in
The study demonstrated proof of concept that a mobile app
the tool’s category. The usage time by each sample changed
can provide insights into cognitive performance that
each day and the change seemed random as we cannot see any
discriminated between different users.
values dominated.
Location: Cho, Myers [13] concluded that periodic
There is no video players & editor app usage by S61 in 12
movement is associated with social network and long travel
days. S52 used these apps only in 2 days and it dominated
distance is correlated to social network ties while short
S28’s usage.
distance movement is not related. Thus, according to prior
D. Go/No Go Test and Stroop Test results, the younger users may have more social interaction
than elderly.
All 3 subjects made less than 2% errors for the Go/No. We
can see a slight difference between 3 samples and the Accelerometer: Due to the discontinuity of accelerometer
percentage of correct answers decrease with aging. Errors were data, we could not predict the type movements when user is
broadly associated with age. Subject S61 made a high number performing action or resting. However, the results can show
of errors and much more than S53 and S28. that S28 is more active than S53 and S61 as S28’s frequency of
movement is highest.
The average time taken to respond when the word matched
the color (i.e. the word GREEN was in green font). The Usage: The app categories can differentiate people, but the
interference score was calculated as the difference between the app usage cannot obviously be used to predict cognitive
average speed when the word matched the color, and when the impairment. This result can illustrate that young people have
word did not match the color. From the literature, we expect more phone usage than elderly and this decreases with age.
differences in value of Go/No go and Stroop test such as the
average speed should be roughly 2 secs. The difference in Go/NoGo and Stroop: The result is unexpected due to the
interference values cannot tell the difference between samples design of the test on smartphone. User must spend time on
as S61 score 0.02, S53 score 0.01 and S28 score 0.04. learning; thus, it creates the first difficulty in these tests.

66
Furthermore, the response time is shorter than the expected difference of culture and literacy [15]. Another limitation is
results pointing to a motor effect. Due to the lack of data, these privacy which personal data such as app usage, location and
two tests cannot be tested the usability in smartphone. accelerometer data are recorded and pushed to server for
human analysis. Future research is planned to deploy Artificial
Questionnaires: This results that young people may have Intelligent to transform and analyze data locally.
more purpose in using the phone than elderly and this phone
usage trend decrease in ageing. An interesting point is that REFERENCES
while spending much time on the phone, there is more absent-
[1] Blanka, K., Mobile Phone Apps in the Management and Assessment of
mindedness. Mild Cognitive Impairment and/or Mild-to-Moderate Dementia: An
Opinion Article on Recent Findings. Frontiers in human neuroscience,
Compared to other variables, app usage is most closely 2017. 11.
associated with the questionnaire results that can answer [2] García-Casal, J.A., et al., ELECTRONIC DEVICES FOR COGNITIVE
almost of the questions in GQ. However, the current phone IMPAIRMENT SCREENING: A SYSTEMATIC LITERATURE
variables in this research is not enough to answer questions in REVIEW. International Journal of Technology Assessment in Health
AQ and RAQ. These questionnaires were collected by Marty- Care, 2017. 33(6): p. 654-673.
Dugas and Ralph [14]. However, it is quite hard to apply these [3] Nestor, P.J., P. Scheltens, and J.R. Hodges, Advances in the early
questions into smartphone as this research results could not detection of Alzheimer's disease. Nature Medicine, 2004. 10(7): p. S34-
S41.
link the phone usage to those questions.
[4] Nasreddine, Z.S., et al., The Montreal Cognitive Assessment, MoCA: A
Keystroke: The result cannot be used to predict the mood Brief Screening Tool For Mild Cognitive Impairment. Journal of the
disturbance because S28 experienced languages barrier and American Geriatrics Society, 2005. 53(4): p. 695-699.
keys tracking function did not work properly. Most of users are [5] Innovations, C., STAC: A Cutting-Edge Cognitive Evaluation Tool.
using autocomplete feature, thus it is hard to analyze the errors 2017.
while typing a message. In this case, the frequency use of [6] Leduc-McNiven, K., et al., Serious games to assess mild cognitive
impairment:‘The game is the assessment’. Research and Review
backspaces is not related to the error of word spelling. Insights, 2018. 2(1).
SMS and Call Logs: There is no “read” variables in [7] Verbruggen, F. and G.D. Logan, Automatic and controlled response
system message app. Thus, we cannot use these data to analyze inhibition: associative learning in the go/no-go and stop-signal
paradigms. J Exp Psychol Gen, 2008. 137(4): p. 649-72.
the correlation between SMS log to cognitive impairment.
[8] Stroop, J.R., Studies of interference in serial verbal reactions. Journal of
Moreover, there is no data for SMS and call log. Therefore, the Experimental Psychology, 1935. 18(6): p. 643-662.
frequency use of message and call app cannot be used to [9] Bennett, D.A., et al., The effect of social networks on the relation
answer the question in questionnaires. However, this can between Alzheimer's disease pathology and level of cognitive function
partially be answered by using the app usage variable. in old people: a longitudinal cohort study. Lancet neurology, 2006. 5(5):
p. 406-412.
V. CONCLUSION [10] Pei, L., et al., Human behavior cognition using smartphone sensors.
Sensors (Basel, Switzerland), 2013. 13(2): p. 1402.
The study offers insights into a very promising area of
study, namely how to start to think about the assessment of [11] Augner, C. and G.W. Hacker, Associations between problematic mobile
phone use and psychological parameters in young adults. International
cognitive impairment using non-invasive smartphone use Journal of Public Health, 2012. 57(2): p. 437-441.
monitoring. [12] Zulueta, J., et al., Predicting Mood Disturbance Severity with Mobile
Several findings are significant, but tempered with the Phone Keystroke Metadata: A BiAffect Digital Phenotyping Study. J
Med Internet Res, 2018. 20(7): p. e241.
limitations of the sample size, with the difficulty to
[13] Cho, E., S. Myers, and J. Leskovec, Friendship and mobility: user
differentiate between population, and correlate the phone movement in location-based social networks. 2011. p. 1082-1090.
variables to the questionnaires. There were also compatibility [14] Marty-Dugas, J., et al., The Relation Between Smartphone Use and
issues, with certain phone OS’s not fully supporting all the app Everyday Inattention. Psychology of Consciousness: Theory, Research,
features. Therefore, S53 had problems in gathering data. and Practice, 2018. 5.
Moreover, due to the Covid-19 restrictions, travelling and [15] Kua, E.H. and S.M. Ko, A questionnaire to screen for cognitive
social communication data were not reliable. impairment among elderly people in developing countries. Acta
Psychiatrica Scandinavica, 1992. 85(2): p. 119-122.
The questionnaires are apply to developed countries and
they may not be applicable to developing countries due to the

67
 2021 the 13th International Conference on Computer and Automation Engineering

Influence Analysis of Joint Effectiveness Partial Loss

on the Motion Capability in the Space Manipulator
Wenqian Xu Gang Chen Yingzhuo Fu Xudong Wang
School of Automation School of Automation School of Automation School of Automation
Beijing University of Posts and Beijing University of Posts and Beijing University of Posts and Beijing University of Posts and
Telecommunications Telecommunications Telecommunications Telecommunications
Bejing, China Bejing, China Bejing, China Bejing, China
[email protected] [email protected] [email protected] [email protected]

Abstract—Aiming at joint effectiveness partial loss failure of manipulator's motion capability is different, and the subsequent
space manipulator, an influence analysis method of motion path planning and control methods are also different. Therefore,
capability is proposed. Firstly, the general model of joint failure it is of great significance to study the influence of joint failure
is constructed; then it is introduced into the kinematic model and on the motion capability of the manipulator.
dynamic model of the manipulator to obtain the model of the
space manipulator with joint effectiveness partial loss failure; The kinematic and dynamic model of the faulty
finally, the manipulability is constructed to characterize the manipulator should be established before the analysis of its
dexterity, and the dynamic load-carrying capacity calculation motion capability. After locking the faulty joint, Xu realized
model is established to calculate the load-carrying capacity of the the kinematics model reconstruction of the manipulator [3]. Jia
manipulator with joint fault. The simulation experiment is used the momentum conservation and Lagrange equation to
carried out to analyze the influence of joint failure characteristics establish the kinematics and dynamics model of the free-swing
(fault degree and location) on the motion capability of the space joint failure space manipulator [4]. Based on the kinematic and
manipulator. dynamic model of the faulty manipulator, scholars have
researched the influence analysis of motion capability.
Keywords—space manipulator, joint effectiveness partial loss,
motion capability Common indexes to evaluate the motion capability of the space
manipulator include manipulability, minimum singular value,
I. INTRODUCTION condition number, etc [5]. Mu pointed out that for the joint-
The space manipulator can complete the tasks of on-orbit locked failure manipulator, it is necessary to establish the
assembly spacecraft, propellant supply, etc [1]. With the reduced Jacobian matrix and reconstruct various motion
establishment of the space station, the space manipulator is capability indexes [6]. Korayem solved the dynamic load-
more and more used in precise on-orbit operation tasks, which carrying capability of the space manipulator considering
puts forward higher requirements for its reliability and safety. attitude disturbance [7]. However, the above research mainly
As the crucial motion unit of the space manipulator, any part of focuses on the fault-tolerant performance analysis of the joint-
the joint will cause joint failure. Besides, once the joint of the locked failure and the free-swing joint failure, there are few
space manipulator fails, it can not be repaired in time, which researches on the influence analysis of joint effectiveness
may lead to the failure of the manipulator to complete the task partial loss on the motion capability in the space manipulator, it
in time. Therefore, it is necessary to analyze the influence of is a novel idea. And the system model of the manipulator with
the motion capability of the failed manipulator to evaluate effectiveness partial loss joint failure can not be obtained by
whether the task can continue to be completed. simply setting the output of the failed joint to zero, the above-
mentioned indexes can not be directly used to evaluate the
Joint failure type mainly includes locked-joint failure, motion capability of the manipulator with effectiveness partial
effectiveness partial loss joint failure, and free-swing joint loss joint failure. Therefore, it is necessary to combine the
failure [2]. Locked-joint failure and free-swing joint failure output characteristics of the failed joint to establish more
correspond to two extreme cases of complete loss of joint comprehensive evaluation indexes of the motion capability of
motion capability and torque output capability respectively. In the manipulator.
a broad sense, if joint motion or torque output capability is
perturbed and can not reach the expected value, it is considered To sum up, this paper proposes a method for analyzing the
that joint failure occurs, that is effectiveness partial loss joint influence of joint failure on the motion capability of the space
failure. Therefore, the analysis of this failure will have stronger manipulator. The general expression of the effectiveness partial
generality and practicability. Joint failure is usually caused by loss joint failure is established, and the mathematical model of
internal factors (e.g, lubrication failure, gap increase, etc.) or the faulty manipulator is reconstructed. Through the analysis of
external factors (e.g, heavy work, harsh environment, etc.), the mathematical model, the indexes of the motion capability
which usually leads to a certain degree of degradation of the are obtained. The influence of joint failure on the motion
motion capability of the manipulator, such as joint speed and capability of the space manipulator is analyzed.
torque output reduction. Also, the degradation degree of the

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 68

 II. MATHEMATICAL MODEL OF SPACE MANIPULATOR WITH If k (t )=e (t )=0 , J k is locked and cannot output motion,
EFFECTIVENESS PARTIAL LOSS JOINT FAILURE the dexterity of the manipulator will degrade.
After the joint failure of the space manipulator, the output
capability of the joint changed, and the original mathematical When the torque output of J k changed, the torque output
model is no longer applicable. Therefore, it is necessary to can be expressed as
reconstruct the kinematic and dynamic model of the failed  k (t )   k (t )   (t )( k (t) 1) k (t )+ e (t) (5)
manipulator. The model of space manipulator with
effectiveness partial loss joint failure is shown in Fig. 1. In this If  k (t )= e (t )=0 , J k occurs free-swing failure and cannot
paper, a series n -degrees redundant space manipulator is taken
output torque. If no measures are taken at this time, the
as the research object in m -dimensional space. manipulator will lose control and threaten itself. Also, the load-
Lk 1 Jk k
Lk
carrying capability of it will degrade.
 k 
J k 1
Jn
J3  k +1 B. The Model of Faulty Manipulator
J k 1 effectiveness  n Ln
L2 3 partial loss joint After the joint failure of the space manipulator, its
kinematic and dynamic model will change, which will affect its
J2 2 motion capability. To analyze the factors that restrict the
L1 motion capability of the faulty manipulator, the model of the
faulty manipulator needs to be constructed.
J1 1
a) Kinematic Model
base
Based on the velocity transfer from joint space to operation
Fig. 1. The model of the manipulator with effectiveness partial loss joint. space, the conversion relationship between joints velocity q(t )
J1 , J 2 , , J n represents the joints from the base to the end effector, and end effector velocity v(t ) can be obtained
L1 , L2 , , Ln represents the links, and 1 , 2 , , n represents the coordinate
system fixed to each link. v (t )  J (q)q(t ) (6)

A. The Model of Effectiveness Partial Loss Joint Failure J (q)  R6n represents the generalized Jacobian matrix of the
Effectiveness partial loss joint failure can be divided into transfer from joint space velocity to operation space velocity,
two types: multiplicative failure and additive failure. In case q(t )   q1 (t ), q2 (t ), , qk (t ), , qn (t )  , qi (t ) represents the
T

that J k occurs multiplicative failure, the output of it can be

i-th joint angle, q(t )   q1 (t ), q2 (t ), , qk (t ), , qn (t )  .
T
expressed as
U k (t )  U k (t )   (t )( k (t) 1)U k (t ) (1) When the motion output of J k changed, the rotational
speed of the joint is qk (t )  qk (t )   (t )( k (t) 1)qk (t )+qe (t ) .
where U k (t ) is the actual output of J k at the moment of t ,
At this time, the joint space velocity of the faulty manipulator
U k (t ) is the expected output of J k at the moment of t , is
0， 0  t  Te

 (t )   is a step function, Te is the time of q(t )   q1 (t ), q2 (t ), , qk (t ), , qn (t )  (7)
T

1， Te  t  T f

fault happens, T f is the end time of the task, and k (t) is the The operation space speed of the manipulator can be
obtained by substituting (7) into (6)
effective rate of the output of J k at the moment of t .
v(t )  J (q)q(t ) (8)
In case that J k occurs additive failure, the output of it can
be expressed as By substituting J (q(t ))  [J 1 , , Jk , , J n ] and (7) into (8)
can obtain
U k (t )  U k (t )  Ue (t ) (2)
v(t )  [ J 1 , , Jk , , J n ][q1 (t ), , qk (t ), , qn (t )]T
where U e (t ) is an unknown additive fault function. (9)
 J1q1 (t )   J k qk (t )  J n qn (t )
Combined with (1) and (2), the general expression of the
effectiveness partial loss joint failure can be obtained b) Dynamic Model
Based on Newton-Euler recursive method, the dynamic
U k (t )  U k (t )   (t )( k (t ) 1)U k (t )+Ue (t ) (3) model of space manipulator is obtained as follows
When the motion output of J k changed, the velocity output H (q)q(t )  C (q, q(t ))   (t ) (10)
can be expressed as
where H (q) is the inertia matrix of the manipulator,
k (t )  k (t )   (t )( k (t )  1)k (t )+e (t ) (4) C (q, q(t )) is centrifugal force and Coriolis force term,

69
 (t )  [1 (t ), , k (t ), , n (t )]T is the torque output of the The reduced Jacobian matrix
manipulator. J (q)  [J1 , , k (t )  J k , , J n ] represents the mapping
relationship between the joint space velocity input q(t ) and
When the output torque of the joint changes, its output is
shown in (5). At this time, the output torque of the faulty the actual operation space velocity v(t ) of the manipulator.
manipulator is Based on the reduced Jacobian matrix, the manipulability of
the faulty manipulator can be constructed
 (t )  [1 (t ), , k (t ), , n (t )]T (11)
By substituting (11) into (10), the dynamic model of the w  det(J (q)J (q)T (15)
manipulator with effectiveness partial loss joint failure is
obtained
H (q)q(t )  C (q, q(t ))   (t ) (12) b) Load-carrying capability
Through the analysis on the reconstruction of kinematics The load capacity carrying capability of the space
and dynamics model of the space manipulator with manipulator will inevitably degrade when the joint of it fails.
effectiveness partial loss joint failure, the model reconstruction That is to say, under the same motion state and constraint, the
can be realized when the joint has any degree of effectiveness mass that the manipulator can load will decrease. Because the
loss failure. load and n-link are usually rigidly connected, the combination
of them can be regarded as a new n-link , the mass, center of
III. THE INFLUENCE ANALYSIS OF EFFECTIVENESS PARTIAL mass, and inertia of the new link are
LOSS JOINT FAILURE ON THE MOTION CAPABILITY OF
MANIPULATOR mn  mn  me
To evaluate whether the fault manipulator can continue to mn  Pn  me  ( Pn  Pe )
Pn  (16)
complete the task, it is necessary to construct the indexes mn  me
which can represent the motion capability of the faulty
manipulator. In this paper, we will consider the actual space I n  I n  I e
operation task and the mathematical model of the fault
manipulator, the indexes that can represent the motion where mn , me and mn are the mass of original n-link , load
capability of the manipulator is combed out, and then establish and new n-link respectively; Pn , Pe and Pn are the center of
the mapping relationship between the fault characteristics and
the motion capability. mass of original n-link , load and new n-link respectively; I n ,
I e and I n are the inertia of the original n-link , load and new
A. The Mathematical Representation of Indexes
n-link respectively. By substituting (16) into (12), the load
Typical on-orbit operation task includes no-load transfer
dynamic model of the manipulator with effectiveness partial
task and load-carrying task. Therefore, this paper will focus on
loss joint failure can be obtained.
the analysis of dexterity and load-carrying capability of the
space manipulator with effectiveness partial loss joint failure. When the space manipulator performs the load operation
a) Dexterity task, it needs to move according to a certain trajectory, that is,
the joints’ angle, velocity, and acceleration are known. When
In this paper, manipulability is used to characterize the
dexterity of the manipulator constructed by the Jacobian matrix J k fails, the torque limit of its output will also change. If there
in each configuration. is no failure of the joint, the output torque constraint is
 k min   k   k max , then the output torque constraint when it
When J k fails, the actual joint space velocity of the
fails is
manipulator is q(t ) , but the control system input is q(t ) .
Making  k min (t )   k min + (t )( k (t)  1) k min + e (t )   k t )
(17)
q (t )  (t )qk (t )  qe (t )   k max   (t )( k (t)  1) k max + e (t ) =  k max (t )
k (t )  k = k
qk (t ) qk (t ) The motion state information is brought into the load
(13)
qe (t ) dynamic model and combined with (17), the joint output torque
  k (t )  ，0  k (t )  1  (t )  [1 (t ), , k (t ), , n (t )] can be calculated. Then the
qk (t )
maximum load calculation model of the faulty manipulator is
where k (t ) represents the degree of k -th joint velocity established at the moment of t is
degradation.
max mt
Substitute (13) into (9) can obtain
s.t  i min   i t )   i max
v (t )  J1q1 (t )   k (t )  J k qk (t )   J n qn (t ) (18)
 k min (t )   k t )   k max (t )
=[J1 , , k (t )  J k , , J n ][q1 (t ), , qk (t ), , qn (t )] (14)
T
i  1, , k  1, k  1, ,n
=[J1 , , k (t )  J k , , J n ]q(t )

70
 The maximum load at any time can form a set

M  m0 , , mt , mtf  (19)

To ensure the stability of the faulty manipulator in the

execution of tasks, its maximum load should not be greater
than any element in the load set, so the load-carrying capability
of the faulty manipulator is
mload  min M (20)
B. Influence Analysis of Joint Effectiveness Partial Loss on
the Motion Capability
Taking the 7-DOF space manipulator as a simulation object,
the influence of motion capability is analyzed, the D-H
parameters of it are shown in Table I. Based on D-H
parameters, the kinematics model of space manipulator can be
obtained.

TABLE I. THE D-H PARAMETERS OF SPACE MANIPULATOR

Link i θi () d i (m) ai (m)  i ( )

0 \ \ 0 0
1 0 0.6 0 90
2 90 0.5 0 -90
3 0 0.5 5 0
4 0 0.5 5 0 Fig. 2. The dexterity when a failure occurs with different locations and
degrees.  represents the effective rate of the velocity output of the fault
5 0 0.5 0 90
joint; qe represents the unknown additive fault function of the fault joint; w
6 -90 0.5 0 -90 represents the dexterity of the space manipulator with effectiveness partial
7 0 0.6 \ \ loss joint failure.

a) Analysis of dexterity b) Analysis of load carrying capability

When the joint failure occurs in the configuration of Based on known tasks, the load mass which corresponds to
q  [23,81,  54,75,13,149,137] , the joint space velocity each point of the trajectory, i.e. the load set, then the maximum
is q  [0.01,0.01,0.01,0.01,0.01,0.01,0.01] rad/s, then the load mass of the manipulator can carry in this task will be
calculated. When the fault location and fault degree are
relationship between dexterity and joint failure characteristics
unknown, the load-carrying capability of the manipulator is
(fault degree and location) is shown in Fig. 2. When there is no
shown in Fig. 3. If there is no failure, the maximum load mass
failure, the dexterity of the manipulator is 0.0763, however,
with the increase of multiplicative and additive failure degree, of the manipulator is 6510 kg, and the failure of J 5 , J 6 , J 7
its dexterity gradually decreases. When J 2 and J 6 fail, have little effect on the load-carrying capability of the
manipulator. When the failure degrees of J1 , J 2 , J 4 are less
dexterity degradation is relatively serious. If k (t )  0 , the
than a certain degree, the load-carrying capability will not
dexterity is also zero, the manipulator will fall into a singular
degrade, but with the gradual increase of the failure degree, the
state at this time. Therefore, it is necessary to avoid J 2 and J 6
load-carrying capability will gradually decrease. When J 3
breakdown as much as possible when the space manipulator
fails, the load-carrying capability of the manipulator is
completes the tasks with high dexterity requirements such as
the no-load transfer task. seriously degraded. Therefore, the failure of J 3 should be
avoided as far as possible when the space manipulator
completes the load-carrying task.

71
 IV. CONCLUSION
To evaluate the motion capability of the space manipulator
with effectiveness partial loss joint failure, a general fault
model construction method is proposed in this paper, which
can realize the general characterization of joint motion and
torque output when there are arbitrary changes. By combining
the fault model with the model of the original manipulator, the
kinematic model and dynamic model of the manipulator with
effectiveness partial loss joint failure are obtained. By
analyzing the kinematic model, the expression of the
manipulability which can characterize the dexterity is
constructed. Based on the dynamic model of the faulty
manipulator, the load dynamic model, and the load-carrying
capability calculation model are constructed to realize the
quantitative evaluation of the motion capacity of the fault
manipulator. Finally, the simulation experiment of a 7-DOF
manipulator shows the effectiveness of the proposed method.
ACKNOWLEDGMENT
The project was supported by the National Natural Science
Foundation of China under Grant 51975059 and Grant
61802363.
REFERENCES
[1] A. Flores-Abad, O. Ma, K Pham, and S Ulrich. “A review of space
robotics technologies for on-orbit servicing,” Prog. Aerosp. Sci, vol. 68,
pp. 1–26, July 2014.
Fig. 3. The load-carrying capability when a failure occurs with different [2] A. D. Luca, R. Mattone, “Actuator failure detection and isolation using
locations and degrees.  represents the effective rate of the torque output of generalized momenta,” Proc. IEEE Int. Conf. Rob. Autom, vol. 1, pp.
the fault joint;  e represents the unknown additive fault function of the fault 634–639, September 2003.
joint; M represents the Load Carrying Capability of the space manipulator [3] W. F. Xu, Y. She, Y. S. Xu, “Analytical and semi-analytical inverse
with effectiveness partial loss joint failure. kinematics of SSRMS-type manipulators with single joint locked failure,”
Acta. Astronautica, vol. 105, pp. 201–217, December 2014.
The simulation results show that the proposed method can [4] Q. X. Jia , B. N. Yuan , G. Chen , and Y. Z. Fu, “Kinematic Modelling
of the Free-Floating Space Manipulator with Free-Swinging Joint
solve the degenerate manipulability of the space manipulator Failure,” Int. J. Aerosp. Eng, vol. 2019, pp. 1–22, September 2019.
when the velocity of any joint changes, which can be used to [5] U. Yoji, Y. Kazuya, “Workspace and Manipulability Analysis of Space
characterize the dexterity of the space manipulator at the Manipulator,” Trans. Soc. Ins. Con. Eng, 2009, 26(2): 188–195.
moment of failure. When the output torque of the joint changed, [6] Z. G. Mu, L. Han , W. F. Xu, and B Liang, “Kinematic analysis and
the load-carrying capability of the faulty manipulator can be fault-tolerant trajectory planning of space manipulator under a single
obtained. Combined with the above simulation results, the joint failure,” Robotics Biomim, vol. 3, pp. 16–26, October 2016.
motion capability of the manipulator with the effectiveness [7] M. H. Korayem , H. Ghariblu , A. Basu, “Dynamic load-carrying
partial loss joint failure can be known and it can be used to capacity of mobile-base flexible joint manipulators,” Int J Adv Manuf
Technol, vol. 25, pp. 62–70, January 2005.
provide help for subsequent path planning and control.

72
 2021 the 13th International Conference on Computer and Automation Engineering

Reinforcing Intrinsic Walking Power with Nudge

Reflecting Day-of-Week Pattern
Takuya Mori Hiromitsu Shimakawa Fumiko Harada
Ritsumeikan University Ritsumeikan University Connect Dot Ltd
Shiga, Japan Shiga, Japan Kyoto, Japan
[email protected] [email protected] [email protected]

Abstract—In order to reduce the risk of lifestyle-related shows the target user the nudging message based on the
diseases, continuous exercise is necessary. However, many people periodicity of walking steps each day. This method effect
are too busy to exercise on a consistent basis. Many existing depends on the experiment of variation of walking steps using
approaches do not take into account the individual's environment the periodicity of walking steps and nudging message. This
and are one-sided in their support. Therefore, they fail to study discusses how much the walking steps can be changed by
motivate individuals to exercise essentially. In this study, we the message.
propose an intervention that takes the environment into account
and is tailored to the individual. The proposed method takes into II. LACK OF EXERCISE AND EXSITING METHOD
account the busyness of the subject based on the periodicity of
their walking volume, and nudge-based messages are sent to A. Nudge and Related Research
them. The use of periodicity and nudges enables consideration of Nudge is the method for behavior modification in
the individual's environment and motivation for intrinsic behavioral economics. Nudge was announced by Thaler in
exercise and is expected to lead to sustained exercise. 2008 [2]. Nudge means to poke lightly with an elbow. Nudge
utilizes the characteristics and biases of the target. Therefore, a
Keywords—walking steps, nudge, STL, seasonal decomposition,
nudge that considers a person's habits and characteristics cause
exercise
a person to act ideally without coercion a person.
I. INTRODUCTION Nudge is also expected to contribute to the increase in
The constant The constant level of physical activity reduces constant exercise [3]. Nudge is easier to continue exercising
risk of lifestyle diseases. According to World Health than coercion exercise support. Cherubini provided target users
Organization, definition of physical activity refers to all with monetary rewards and motivational messages. as result,
movement. Physical activity includes a commuting, jobs, etc. target users continued exercise [4]. However, target users'
The constant level of physical activity is recommended in order intrinsic exercise motivation was reduced.
to prevent lifestyle diseases. However, 1/4 adult in the world is
No nudge is valid in every context, and the keywords and
below the constant level of physical activity [1]. Many
methods used in each context are different. The Behavior
countries are getting concerned about improving lack of
Change Wheel (BCW) enables to arrange the relationship
exercise.
between context and effective nudge. BCW is a framework that
The reason for the lack of exercise is roughly two. First summarizes the theory of behavior change [5]. BCW classifies
reason, many people not conscious of exercise. The benefit of the elements of intervention to promote behavior modification
exercise is invisible. Many people realize how important into nine.
exercise is after they get lifestyle diseases. Another reason is
Various solutions using IT are proposed for the lack of
the constraint of context of exercise. For example, A worker
physical activity. The recent popularization of wearable
has a working item that attends the regular sessions some days
devices enables us to measure movements and signals of
in a week.
human bodies without effort [6, 7]. It has brought e-Health,
This paper focuses on the periodicity of busyness each day. which incorporates IT into medical care, to practical use [8].
In some cases, people too busy to exercise by time constraints. The incorporation of IT into medical care is expected to reduce
This problem influences many workers. However, existing IT social costs that come from the health management of various
solutions for health-promoting have ignored the periodicity of populations[9].Another expectation is gamification, which
busyness each day. IT solutions for the health-promoting need applies entertainment elements of games to other fields [10].
integration that motivation and constraint of context for the
B. Seasonal Trend Decomposition
exercise.
Since the busyness varies from day to day, many people
This study handles walking because a busy worker is easily have a periodic increase or decrease in their physical activity. It
possible. In the proposed method, the avatar using in a is dangerous to consider interventions to increase physical
smartphone support improvement of walking steps. The activity by observing only the daily physical activity of a
proposed method estimated the periodicity of walking steps specific person. First, it is necessary to clarify what kind of
each day using seasonal trend decomposition. The avatar

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 73

periodicity exists in the amount of physical activity for each busyness caused by the environment. The target user must be
individual. encouraged to maintain a high level of consciousness of their
own exercise, taking into account the individual's environment
decomposition is a method of removing seasonal for sustained improvement in walking steps.
fluctuations contained in time-series data. For example, the
commodity price of time-series data on crop fluctuates In this method, walking steps is first collected by a
periodically due to the influence of climate. Seasonal trend smartphone or smartwatch. Walking steps are recorded as time-
decomposition is effective when you want to analyze periodic series data with a daily timestamp. Next, in order to consider
price fluctuations or when you want to analyze fluctuations that the individual environment, this method decomposes the time-
cannot be explained by seasonal fluctuations. series data of walking steps into seasonal variation component
and trend component by seasonal component decomposition.
When the seasonal trend decomposition is applied to time In this method, the values of the decomposed seasonal
series data TS, it can be decomposed into three components. component and trend component are associated with the
The formula for seasonal trend decomposition is shown below individual's environmental and exercise consciousness,
(1). respectively.
TS＝S + T + I (1) The constraint that each person has for each day of the
The first component is the seasonal component S, which week is the personal environment. In this study, we consider
represents seasonal fluctuations. The second component is the that the personal environment is reflected in the seasonal
trend component T, which is removed the seasonal component variation component. In this study, we consider that the
from the original time series data. The third element is the personal environment is reflected in the seasonal component.
irregular variation I. This is much of a component that cannot On the other hand, the walking steps that is not inhibited by
represent seasonal and trend elements. external factors represents the original walking steps of the
target user. The present study considers that an individual's
As an example, consider the case where seasonal trend consciousness of exercise appears in the trend component. In
decomposition is applied to time series data of something this study, the trend component is called the intrinsic
prices. Seasonal components show the periodicity of price component. When an individual's health consciousness rises,
declines due to year-end sales. Since the trend component is the intrinsic component is inherently high.
the data excluding the seasonal component, it is possible to
judge the decrease in consumer demand. The irregular In this method, a message containing a nudge is presented
variation component can be read as the effect of a sudden on the target user's smartphone. This message is based on the
recession. In this way, by seasonal trend decomposition, data BCW and encourages walking considering the individual's
can be read from multiple aspects. There are multiple methods environment. The effect of the transmitted message is
for seasonal component decomposition, such as moving measured from the value of the intrinsic component associated
averages and Seasonal and Trend decomposition using Loess with the individual's consciousness of exercise. This method
(STL) using locally weighted smoothing. Rowlands et al. Used repeats the messages that are found to be more effective.
seasonal trend decomposition to confirm that the number of Repetition of the tasks in Figure 2 is expected to improve the
steps was periodic [11]. walking steps.

III. IMPROVEMENT OF WALKING BY NUDGE CONSIDERING B. Application of Seasonal Component Decomposition to

BUSYNESS Walking
Seasonal trend decomposition is a method for extracting
patterns of periodic data such as seasons as seasonal
components from a certain time series data. In this method,
the walking steps is assumed to be periodic for each day of the
week. For example, the target user would typically commute to
work on weekdays and shop on weekends. Walking steps
required for commuting, shopping, etc. is considered to be
constant in most cases. In addition, regular meetings are often
held on certain days of the week. On such days, walking steps
is inevitably small. In order to extract such a component of
periodicity for each day, the width of the component of the
seasonal trend decomposition of the walking steps data was set
Fig. 1. Schematic of the method.
to 7 days in this study.
Seasonal component decomposition using simple moving
A. Context Component and Intrinsic Component averages results in missing data at the edges during the
According to a survey by the Japan Sports Agency, the computation process. To compensate for missing data, there is
reason why people cannot get into the habit of exercising is a method of estimating using only data from either before or
that busyness has a large effect. This method deals with after the edge data. However, this method results in biased
walking, which is easy to introduce even if you are busy. In estimates. To solve these problems, we used STL, which is a
addition, the existing methods do not take into account the method of decomposing seasonal trend components using

74
locally weighted linear regression. STL calculates seasonal and changed with the nudge intervention. The message format
trend components from local regressions. This allows the STL nudge is based on some of the elements of the BCW. This
to decompose the components without loss. paper examines the impact of the intervention by nudges.
Another advantage of using STL is its ability to follow the B. Creating the Message
seasonal component as it changes over time. In general, it is All nudges do not enforce any particular behavior. They
considered that there is a periodicity change in the walking only convey information that is useful to the target user. The
steps over time. The SLT can be optimized to gradually show target user was given the option to choose whether or not to
the effect of any change that has a significant impact on increase the walking steps.
periodicity. For example, even if the mode of commuting
changes, the seasonal component is optimized to show In the method proposed in this paper, the avatar sends a
periodicity as the weeks goes by local regressions. message. Avatars are expressed using emoticons. It is
important to create nudges carefully, keeping in mind the
By decomposing the data with the period set to 7 days, this recipients of the message, and considering whether the nudge
method can calculate the seasonal component, which will feel forced. The avatar was set to live as a health-
represents the periodicity of the walking steps for each day, conscious person. Furthermore, the avatar is not a perfect
and the trend component, which is the original walking steps figure, trying to act ideally but also experiencing failure. In
data minus the seasonal component and the irregular variation other words, the avatar is a person who is hardworking and
component. In this paper, we assume that most of target user ambitious, but who also makes mistakes. In the nudges to
lead a week-long life. Therefore, we consider the seasonal promote simple exercises, the avatars will share their own
component as the day of week component and the trend experiences of working on them and failing. Thus, in thinking
component as intrinsic component. of a message, by setting human-like characteristics for avatars,
C. Messages Using Nudges to Improve Walking Steps the target user of the message will be more likely to empathize.
It is difficult to induce a person who does not exercise V. RESULT
regularly to suddenly engage in strenuous exercise. In order to
achieve continuous exercise, it is physically and mentally A. Results of Statistical Tests of Walking Steps
burdensome for the target user to have others monitor their First, the normality of the data was checked. A display of
behavior and instruct them to exercise. In addition, this method the all-target users walking steps data using q-q plot is shown
is considered to have many problems in social implementation in Figure 2. The red line represents the waveform of the
due to the heavy burden on the vender. walking steps in the case of a normal distribution, and the
This paper proposes a format in which an avatar sends an points plotted in blue represent the data to be treated in this
exercise promotion message to target user. Target user receives study. Figure 2 shows that the edge of the blue point is far from
messages sent by avatars at regular intervals, which increases the red line. Finally, the Shapiro-Wilk test was performed. The
his consciousness of exercise and makes it easier for them to Shapiro-Wilk test is a method to test whether the data are
perform walking. This method is believed that it is important normally distributed, assuming that the population of data
for the target user to agree and sympathize with the proposed sampled for the null hypothesis follows a normal distribution.
method as a basic principle to change the consciousness of As a result, the test statistic is 7.9e-23
exercise. There are many barriers to imposing ideal health (0.00000000000000000000000079), so the null hypothesis is
behaviors on people, regardless of their level of consciousness rejected, and the sampled data is not considered to follow a
of exercise. People who are highly conscious of their health but normal distribution.
are too busy to exercise cannot be expected to change their
behavior even if they are invited to a sports gym. Nudges in
behavioral economics lead to ideal choices by suggesting
solutions to problems from the perspective of the target user. In
this method, the avatar presents a message using nudges while
staying close to the target user. The messages gradually lead
the target user's consciousness of exercise in a positive
direction.
IV. EXPERIMENTS
A. Behavior Change Experiment with Messages
The present experiment was conducted on 13 male college
students. Their ages ranged from 20-24 years. To verify the Fig. 2. Q-Q plot
effectiveness of our method, this method installed avatar on the
target user's smartphone. This experimental is conducted an Since the population does not follow a normal distribution,
experiment using avatars to present message style nudges to a nonparametric test is necessary. In this test, Wilcoxon's
target users. The target users' daily walking steps was signed rank sum test was used for the corresponding data. The
measured using a smart phone. In the experiment, this method unpaired data were tested using the Mann-Whitney U test. The
is examined how walking steps taken by the target users significance level is set at 10%.

75
 TABLE I. TEST STATISTIC OF INTERVENTION IN EACH TARGET USER
subjects A B C D E F G H I J K L M
statics 0.08 0.57 0.95 0.28 0.02 0.64 1.00 0.86 0.23 0.15 0.42 0.12 0.30

regarded as a component that represents the busyness caused

by the environment unique to the target user. The intrinsic
component represents the walking steps per week with the
context component removed. The intrinsic component is the
walking steps due to the target user consciousness of the
exercise. The experiment was conducted to send messages
Fig. 3. Period of time with or without intervention
using nudges on a smart phone.

It was confirmed how much the presence or absence of the By using this method, which takes into account the target
message affected the walking steps. The time period of the data user environment, the target user can be made conscious of the
covered in this analysis is shown in Figure 3. The null need for exercise. In the future, we will apply the proposed
hypothesis was that there was no significant difference method not only to walking motion but also to data of various
between the walking steps during the two weeks before the physical activities. In order to increase the reliability of the
start of the experiment and the walking steps during the two data, we plan to conduct experiments with a large number of
weeks during the experiment when the message was sent. The target users. Long term experiments are needed to see how
test result was 0.127 when performed on all target users. The long the effects of the messages last.
test results for each target user are shown in Table I. In the future, we will apply the proposed method not only
In the overall results, there was no significant difference in to walking steps but also to data of various physical activities.
the walking steps with and without the message. On the other
hand, when examined on an individual basis, significant REFERENCES
differences were found among several target users. [1] World Health Organization. Global action plan on physical activity
2018-2030: more active people for a healthier world. World Health
VI. DISCUSSION Organization, 2019.
[2] Leonard, Thomas C. "Richard H. Thaler, Cass R. Sunstein, Nudge:
A. Effective Messages as Nudges Improving decisions about health, wealth, and happiness." (2008): 356-
360.
This section discusses the results of the interviews for each
[3] Li, Meng, and Gretchen B. Chapman. "Nudge to health: Harnessing
nudge conducted after the experiment. There was a negative decision research to promote health behavior." Social and Personality
opinion about the nudge, which periodically notified the target Psychology Compass 7.3 (2013): 187-198.
user of the days of the week when he was walking less. The [4] Mauro Cherubini, Gabriela Villalobos-Zuñiga, Marc-Olivier Boldi, and
target user was aware of the fact that he was walking less, so Riccardo Bonazzi. 2020. The Unexpected Downside of Paying or
the notification made him disenchanted with walking. In order Sending Messages to People to Make Them Walk: Comparing Tangible
to avoid creating meddlesome nudges, it is necessary to Rewards and Motivational Messages to Improve Physical Activity.ACM
Trans. Comput.-Hum. Interact.27, 2, Article 8 (March 2020), 44 pages.
intervene while taking into account the target user's
[5] Michie, Susan, Maartje M. Van Stralen, and Robert West. "The
consciousness of the walking steps. Nudges do not force a behaviour change wheel: a new method for characterising and designing
behavior on the target user. Nudges that may be perceived as behaviour change interventions." Implementation science 6.1 (2011): 42.
criticizing the target user can have a bad effect if they fail to [6] Yangjian Huang, Junkai Xu, Bo Yu, and Peter B Shull. Validity of fitbit,
suggest the action you want the target user to take. jawbone up, nike+ and other wearable devices for level and stair
walking. Gait & Posture, 48:36–41, 2016.
VII. CONCLUSION [7] S Baskar, P Mohamed Shakeel, R Kumar, MA Burhanuddin, and R
It is important to take into account the target user’s Sampath. A dynamic and interoperable communication framework for
controlling the operations of wearable sensors in smart healthcare
environment and consciousness of exercise when applications. Computer Communications, 149:17–26, 2020.
implementing exercise. In this paper, the method is proposed a [8] Supraja Sankaran, Paul Dendale, and Karin Coninx. Evaluating the
method to improve the walking steps using a smart phone to impact of the hearthab app on motivation, physical activity, quality of
improve inactivity. The proposed method creates messages life, and risk factors of coronary artery disease patients:
based on nudges in order to make the target user motivate to multidisciplinary crossover study. JMIR mHealth and uHealth,
exercise without difficulty. By sending messages using nudges 7(4):e10874, 2019.
to the target user, it is expected to raise consciousness of [9] Farshad Firouzi, Bahar Farahani, Mohamed Ibrahim, and Krishnendu
Chakrabarty. Keynote paper: from eda to iot ehealth: promises,
exercise. challenges, and solutions. IEEE Transactions on Computer-Aided
The proposed method applies the seasonal trend Design of Integrated Circuits and Systems, 37(12):2965–2978, 2018.
decomposition to the walking steps in order to consider the [10] Juho Hamari. Do badges increase user activity? a field experiment on
environment of the target user. The seasonal trend the effects of gamification. Computers in human behavior, 71:469–478,
2017.
decomposition decomposes the walking steps into context
[11] Alex V. Rowlands, Sjaan R. Gomersall, Catrine Tudor-Locke, David R.
component and the intrinsic component. The context Bassett, Minsoo Kang, François Fraysse, Barbara Ainsworth & Tim S.
component is the difference in the walking steps for each day Olds (2015) Introducing novel approaches for examining the variability
of the week. This difference can be attributed to work and of individuals’ physical activity, Journal of Sports Sciences, 33:5, 457-
lifestyle conventions. Therefore, the context component can be 466.

76
 2021 the 13th International Conference on Computer and Automation Engineering

The Effects of Different Artificial LED Light

Compensation for Rice Crops
Analyn N. Yumang, Elvin F. Cruz, Ralph Joshua N. Riel
School of Electrical, Electronics and Computer Engineering, Mapúa University
Muralla Street, Intramuros, Manila 1002 Philippines
[email protected], [email protected], [email protected]

Abstract—Flexible light-emitting diodes serve their purpose

as an alternative source of energy for the rice crops. Three
different colors have been applied for light compensation. These
are white, red, and blue colors. This study aims to apply the
Fuzzy Logic algorithm for illuminating the LEDs for the rice
crops and compare the growth of the rice crops between a
compensated and non-compensated rice crops in terms of light.
The system is active 16 hours a day. LEDs automatically turn
ON gradually in the absence of sunlight and vice versa. Data Fig. 1. Isaac Newton’s Experiment on Visible Light
shows that in terms of leaf growth, the white LED is the best
artificial light for compensation with an average length of 103.53 In the experiment performed by Isaac Newton on visible
cm and in terms of weight of grains, red LED came up with the light [6], each color has a different wavelength. Thus, each
highest yield with a value of 102 grams. With a significant level color of light has different effects on plants. According to a
of a = 0.1, the white LED helped the plants in terms of plant recent study [7], [14] the wavelengths of blue and red light are
height with a t-statistic value of 1.83 and a probability value of absorbed by plants in a greater amount than other wavelengths
0.08. Whereas the red LED obtained a t-statistic value of 2.02 in the spectrum for photosynthesis. White light on the other
and the probability of 0.06 which helped the crops increase the hand contains all wavelengths of visible light similar to
grains harvested. sunlight. Another study shows the use of blue and purple
colors, the results showed the height and weight of the plants
Keywords—LED Light Compensation, Oryza Sativa, Fuzzy
Logic, Urban Farming, T-test
were affected by the purple light [8].
The fuzzy logic algorithm is a type of extended set of
I. INTRODUCTION Boolean Logic wherein it can operate in a set of values [9].
Agriculture has been a huge contributor to food production These values are truth values between what is called
for our daily survival. It is by means one of the largest “completely true and completely false”, or 1 and 0. Fuzzy
contributors to the Philippine economy having 40% of Filipino Logic can predict and determine any values between 1 and 0.
citizens working in the field of agriculture [1]. The stability of Thus, allowing it to have multifarious options. Hence, these
the agricultural sector is in the hands of our farmers. One of multifarious options belonging to the set of values between 0
the largest population of farmers in the agricultural sector is and 1 can be in a form of low, medium, all the way to high.
rice farmers. Rice crops are known to be the most important The set of degrees that belongs between 0 and 1 may depend
crops worldwide. Here in Asia, the most common rice crops on the objective, thus giving it numerous applications. In
known to farmers is called Oryza Sativa, or also known as processing the set of degrees, to gain a better understanding of
Asian rice. The source of energy for the rice crops is light the control system for Fuzzy Logic, it can be implemented
wherein light is used for photosynthesis [2]. The light intensity using the “If-then” rules.
in the environment constitutes the development of the This study aims to develop a system that will be able to
structural form of the rice crop. A light deficiency would affect monitor the effects of different LED compensation for rice
the (1) growth, (2) yield and quality (3), and the formation of crops. The objective the study aims to accomplish is (1) to
grain [3]. develop a system that will utilize a light sensor for detecting
the illuminance value and real-time clock module for
There are many wide-range land areas for farming that are
monitoring the time, (2) to apply the fuzzy logic algorithm for
instead being developed for non-agricultural use [4]. The
controlling the illumination of the LEDs, and (3) to compare
demand for land conversion for urban development in the the growth of the rice crops between the crops with no LED
metropolitan region is growing rapidly [5]. Thus, one of the compensation system and those with LED compensation
factors that farmers (whose lands are converted) will encounter system in terms of plant height and weight of the grains
is the knowledge on growing plants effectively in an urbanized harvested. Hence, this study will cover three different
area and the effects of artificial light on the growth of rice wavelengths (red, blue, and white) for the LED light
crops. compensation for photosynthesis of the rice crops for 16 hours
during the absence of sunlight.

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 77

 II. METHODOLOGY Considering the surface area of the platform, the total
Stated in this section are the procedures that have been amount of Watts that will be used is 13.5W. In relation to
followed to complete the system requirements. Implementing monochromatic light flow [11], luminous flux is determined
and analyzing the methodology results for accurate considering the luminous efficacy of LED lights be 90 lm/W:
conclusions. 𝜑𝑉(𝑙𝑚) = 𝑃(𝑊) × 𝜂(𝑙𝑚)
𝑊

where,
𝜑𝑉(𝑙𝑚) = Luminous Flux in Lumens

𝑃(𝑊) = Watts

𝜂(𝑙𝑚) = Lumens per Watt

𝑊
Fig. 2. Conceptual Framework
𝜑𝑉(𝑙𝑚) = 13.5𝑊 × 90 𝑙𝑚⁄𝑊
In Figure 2, the conceptual framework of the system shows
the three different consecutive approach for system analysis. 𝜑𝑉 = 1215 𝑙𝑢𝑚𝑒𝑛𝑠
For the first approach, we have the input wherein it serves as
the first type of information being accepted by the system. In Approximately 1215 lumens are effused by the LED strip
this case, we have the light sensor readings (illuminance level lights on each side of the trough. This value of lumens is what
of the surroundings) and time recorded by the sensors. The the system will deploy in complete darkness, meaning this
second approach is the process, the process of information amount of light is triggered during the absence of sunlight. The
from the input is being converted into the output. In this case, relationship of lumens from the sunlight to the number of
it initiates the fuzzy logic algorithm and determines if the lumens generated by the LED is inversely proportional to one
system has been running for 16 hours. The last approach is the another.
output, the information processed by the system are reflected
during this state, wherein this case it uses the LED as the 1
output based on the input illuminance value, then the
illuminance value and time is displayed digitally using LCD.
A. Developed Lighting System for Rice Crops 2
The following will be the discussion of the developed light
system for rice crops: 2

Fig. 4. Experimental Setup of the LED Light Compensation for Rice Crops

The figure shown above shows the experimental setup of

the LED light compensation on rice crops. The trough has a
wall-like structure that will serve as a platform that supports
the LED strip lights. The light sensor is placed in a position
where it is higher than the transparent screen which supports
the LED strip lights and it has a curved cap to protect the light
sensor from the light rays of the LED strip lights. When the
strength or amount of light is produced by the sun approaches
low light intensity, the light sensor (1) will detect it. The utility
Fig. 3. Measurements of the Trough box (2) will serve as the storage for the microcontroller, power
supply, and other electronics that will keep it safe from
The LED strip lights are placed on a platform that would unwanted weather conditions.
light up the rice crops below. The platform is a 0.6x0.89 m
transparent screen that has a 0.534m2 surface area and is B. Software Development
attached to each side of the trough (Figure 3). The length of Shown in Figure 5 is the flowchart of the main system. It
the LED strip light is based on the amount of planted rice will start by initializing the system. The timer is set to 16
crops which would enable the researchers to determine the hours since it is the maximum hours of exposure to light by
maximum amount of illuminance LEDs must effuse. The rice crops. Next, it will obtain the readings coming from the
amount of rice crops planted on the trough consists of 8 rice light sensor that will be evaluated using the fuzzy logic
plants wherein each plant contains 3 pre-grown seedlings and algorithm. If the system does not exceed the 16 hours timer, it
having a space between them by 20x20 cm just 5 cm below the will continue obtaining the light sensor readings and then
recommended spacing [10].

78
evaluate it again by fuzzy logic. If the system exceeds the 16- the fuzzy values or crisp inputs in the fuzzification process. It
hour mark, it will turn off the system. will be converted into a linguistic variable that will correspond
to its membership function or the fuzzy rules. First, the
membership value (input) of the illuminance obtained by the
light sensor will be determined. Afterward, the process is
called the inference. This process involves the formulation and
evaluation of fuzzy rules which will be the basis of how the
fuzzy logic system will infer an output that is based on the
input variables [12]. The next process is the aggregation of the
output, wherein this combines the membership functions of all
fuzzy logic rules to form a single fuzzy set. Then lastly, the
defuzzification produces the output which gives the overall
illuminance value of the LEDs.
C. Fuzzy Logic Rules
The fuzzy rule base consists of a collection of fuzzy IF-
THEN rules. In this paper, the rules are summarized in the
enumeration below. The rules stated as follows:
1. If the illuminance level of sunlight is very high, the
luminous flux of the LED is very low.
2. If the illuminance level of sunlight is high, the
luminous flux of the LED is low.
3. If the illuminance level of sunlight is middle, the
luminous flux of the LED is middle.
4. If the illuminance level of sunlight is low, the luminous
flux of the LED is high.
5. If the illuminance level of sunlight is very low, the
luminous flux of the LED is very high.

Fig. 5. Flowchart of the LED Light Compensation

The Fuzzy Logic algorithm is used by the system to
prevent damage to the photosynthesis machinery of the plants
from the excessive change of light levels of the LED lights.
The rapid change of light can cause stress due to
photosynthetic reaction to the radiation of excess light [15],
[16]. By applying the Fuzzy Logic algorithm, not only will it
protect the plant’s light-harvesting complexes of proteins and
chlorophyll molecules that enlarge when the light is low and
shrinks when the light is bright which is also called the
photosynthesis antennae but could also help save power
consumption of the device.

III. RESULTS AND DISCUSSION

The gathered results of the system were compared using
graphical representations and the two-sample T-test. For Table
I, the growth phases of the rice crops are presented.
TABLE I. GROWTH PHASES OF THE RICE CROPS WITH ARTIFICIAL LIGHTING
Growth Phase Description

During this phase, the rice crops are called

seedlings wherein roots are starting to
develop and the leaves will start to grow
as well for the next 14 to 35 days until it
reaches the next stage.
Fig. 6. Flowchart of the Fuzzy Logic Algorithm

Shown in Figure 6 is the flowchart for the pre-defined

function “Fuzzy Logic Algorithm”. Fuzzy logic will be used to
control the brightness of the LED when illuminating the rice
crops. The readings measured by the light sensor will become

79
 After the seedlings have fully grown, it
C. Comparison of the Data Gathered
enters the tillering stage or also known as The figures shown below are the graphical representation
the vegetative phase. During this phase, of the data collected. Figure 7 shows the comparison of plant
tillers will start to develop and have more height, wherein white light is leading. Then in figure 8 in
leaves. Thus, there is an increase in plant
height. terms of the weight of grains, red light is leading.

The next phase is called the reproductive

phase. The rice crops will start to show
bulging of leaf stems, this is the sign that
the plant is getting ready for flowering.
This can be noticed when the tip of the
panicle from the stem emerges and
continues to grow.

Fig. 7. Comparison of Plant Height

During this phase, the plant is getting
ready to be harvested. This phase is called
the ripening phase, wherein the flowers
and fruits are approaching the maturity
stage.

A. Data Gathering for Leaf Height of the Rice Crop

Table II illustrates the plant height per trough during the 5-
month growth of the rice crops. The measurement was taken
from the tallest leaf from each plant from June to October Fig. 8. Comparison of Weight of Grains
2020. Each trough (T) represents the light compensation of the
plants; (A) white light, (B) red light, (C) blue light, and (D) D. Statistical Treatment of Data Using T-test
without compensation.
The statistical treatment that will be applied to the data
TABLE II. AVERAGE LEAF HEIGHT OF THE RICE CROP PER TROUGH collected in the study is a two-sample T-test. One study [13]
uses the t-test to compare the difference of the grain yield
Plant Height (cm) between the system developed to the actual average grain yield
Month 5 having a significance level of 0.05. In this study, four troughs
T P1 P2 P3 P4 P5 P6 P7 P8 Ave will be used: troughs A, B, C, and D. To get the significant
A 102.2 112.8 87.3 96.9 111.5 94.1 101.4 122 103.53 difference in the plant height and weight of grains between the
B 96.1 104.7 96.8 98 93 109.6 90.2 107 99.43 troughs, the null hypothesis (𝐻0 ) is there is no significant
C 89.3 93.6 95.8 96.2 93.8 87.7 95.2 84.2 91.98
difference between the length of the leaves and weight of
grains of the troughs with system and without the LED light
D 103 82.6 90.4 102.3 100.8 93.4 99.1 85 94.58
compensation. The null statement is stated mathematically as:
B. Data Gathering for Grain Weight Harvested
𝐻0 : ̅̅̅
𝑥𝑎 = ̅̅̅
𝑥𝑏
Table III illustrates the weight of grains in terms of grams.
Each seedling in the trough is measured then the amount of The alternative hypothesis (𝐻𝑎 ) will be there would be a
weight per trough is gathered by computing the total grams of significant difference in the plant height and weight of grains
the grains. between the troughs with system and without the LED light
TABLE III. WEIGHT OF GRAINS PER TROUGH
compensation. The alternative statement is stated
mathematically as:
Weight (g)
Tr P1 P2 P3 P4 P5 P6 P7 P8 To 𝐻𝑎 : ̅̅̅
𝑥𝑎 ≠ ̅̅̅
𝑥𝑏
A 13 9 13 15 12 11 12 14 99
B 14 12 14 16 11 13 12 10 102
The significance level (a) that will be used for determining
if the null hypothesis is accepted or rejected will be a = 0.10.
C 9 7 10 13 8 12 9 10 78
The degree of freedom (DF) will be obtained by getting the
D 12 8 13 15 8 11 7 9 83 smaller value of the two samples then subtract one. To

80
determine the probability value that will be compared to the compensation system does not affect the leaf height and the
significance level, T distribution will be used with the test weight of the grains.
statistics connected and the degrees of freedom.
V. RECOMMENDATION
TABLE IV. TROUGH A V. TROUGH D
The researchers were able to complete the study. However,
Height Weight due to the minimum resources from the current pandemic
T-test Analysis situation, there are things that researchers would suggest and
A D A D recommend for future works. Improvements can be done to the
Mean (𝒙
̅) 103.53 94.58 12.38 10.38 design setup of the system. For better accumulation of natural
light, the light sensors can be redesigned for protective
Standard Deviation (s) 11.30 7.96 1.51 2.72 measures and prevent damage to the system. The hardware
Number Sample (n) 8 8 8 8 components can be separated from the troughs to prevent the
hardware from being wet. The troughs can be redesigned
P-value of T-test 0.08 0.09
depending on the set so that they would be weatherproof and
allow the rice crops to grow freely without any disruption from
TABLE V. TROUGH B V. TROUGH D
natural calamities and take proper measurements to prevent
Height Weight pest infestations without using any pesticide. The researchers
T-test Analysis may include an irrigation system for rice crops. In addition,
B D B D future researchers can include a system that would automate
Mean (𝒙
̅) 99.43 94.58 12.75 10.38 the measurement of the plant height for precise monitoring of
the plant growth.
Standard Deviation (s) 6.92 7.96 1.91 2.72
Number Sample (n) 8 8 8 8 REFERENCES
[1] FAO (Food and Agriculture Organization of the United Nations).
P-value of T-test 0.21 0.06 (2005). Fact Sheet Philippines: Women in Agriculture and Rural
Production. [Online]. Available:
TABLE VI. TROUGH C V. TROUGH D http://www.fao.org/3/ae946e/ae946e03.htm.
[2] C. C. Chen, M. Y. Huang, K. H. Lin, etc. (2014). Effects of Light
Height Weight Quality on the Growth, Development, and Metabolism of Rice
T-test Analysis Seedlings (Oryza sativa L.). Research Journal of Biotechnology.
C D C D Volume 9.
Mean (𝒙
̅) 91.98 94.58 9.75 10.38 [3] C. Liu, X. Wu, B. C. Chen, J. Q. Ma, J. Gao (2014). Effects of Low
Light on Agronomic and Physiological Characteristics of Rice
Standard Deviation (s) 1.55 7.96 1.83 2.72 Including Grain Yield and Quality. Rice Science. Volume 21, pp. 243-
251.
Number Sample (n) 8 8 8 8
[4] Kaisahan (2017) Solidarity Towards Agrarian Reform and Rural
P-value of T-test 0.43 0.59 Development. [Online]. Available:
https://kaisahan.com.ph/posts/conflict-interest-villar-committee-
leadership
IV. CONCLUSION [5] I. R. Malaque, and M. Yokohari (2007). Urbanization process and the
As for the design of the light compensation system for the changing agricultural landscape pattern in the urban fringe of Metro
rice crops, the system was able to achieve a 16 hour of Manila, Philippines. Environment and Urbanization, 19(1), pp. 191–206.
operation to give the plants support for energy consumption in [6] Tour of the Electromagnetic Spectrum “Visible Light” (2010) [Online]
a form of artificial light. The materials necessary for the Available: https://science.nasa.gov/ems/09_visiblelight
monitoring system was successfully built and tested for better [7] M. V. Caya, W. Chung, N. B. Linsangan, J. L. Torres, and J. D. A.
Villarama (2019). Development of Automated LED Light
results. After the hardware components have been tested, the Compensation System for Lycopersicon Esculentum. 2019 IEEE
fuzzy logic algorithm was applied in a form of a PWM system International Conference on Consumer Electronics - Asia (ICCE-Asia).
of the microcontroller. The PWM allowed the LED lights to Volume 2019, pp. 180-184.
change their illuminance value at a specific light level obtained [8] A. M. V. Pagarigan and M. V. Caya, “Development of LED light
by the light sensor. In five months, in terms of leaf height, compensation system for agricultural application,” 2018 IEEE 10th Int.
Trough A has a test statistic value of 1.83 and a probability Conf. Humanoid, Nanotechnology, Inf. Technol. Commun. Control.
Environ. Manag. HNICEM 2018, pp. 1–6, 2019.
value of 0.08, which is less than the significance level that
[9] C. H. Cheng and Y. Bai (2012). Using Fuzzy Logic and Light-Sensor
rejects the null hypothesis and accepts the alternative for Automatic Adjustment of Backlight Brightness in a Mobile
hypothesis. In terms of the weight of the grains harvested, Computer. Graduate Institute of Applied Science and Engineering, Fu-
Trough A has a test statistic value of 1.83 and a probability Jen Catholic University, Taiwan. pp. 1-5.
value of 0.08 while Trough B has t statistic value of 2.02 and a [10] D. Dinesh, A. Baskar, D. Jinger, etc. (2019). System of Rice
probability value of 0.06 which makes both significant since Intensification-a boon to Indian Farmers: A Review. Ann. Agric. Res.
the probability value is less than the significance level. In New Series. Volume 40 (2), pp. 123-139.
conclusion, white-colored LED light compensation system [11] E. Spunei, I. Piroi, C. P. Chioncel (2016). The Experimental
affects the leaf height and the weight of the grains in a rice Determination of the Luminous Flux Emitted by a Few Types of
Lighting Sources. IOP Conference Series: Materials Science and
crop, red-colored LED light compensation system affects only Engineering. Volume 163.
the weight of the grains, and blue colored LED light

81
[12] K. Choeychuen (2018). Fuzzy Membership Function Optimization for Efficiency of Three Ornamental Pot Plants. Frontiers in Plant Science,
Smart LED lamp using Particle Swarm Optimization. 2018 8, pp. 1–12.
International Workshop on Advanced Image Technology, IWAIT 2018, [15] A. Tendler, B. C. Wolf, V. Tiwari, U. Alon, and A. Danon (2018). Fold-
pp. 1–4. change Response of Photosynthesis to Step Increases of Light Level.
[13] A. N. Yumang, G. O. Avendano, J. C. D. Cruz, A. Ballado, L. E. IScience, 8, pp. 126–137.
Agustin, D. M. Del Mundo, M. V. Caya (2017). Microcontroller-based [16] G. Zervoudakis, G. Salahas, G. Kaspiris, and E. Konstantopoulou
Fertilizer Dispenser for Rice Crop. Proceedings - 6th IEEE (2012). Influence of light intensity on growth and physiological
International Conference on Control System, Computing and characteristics of common sage (Salvia officinalis L.). Brazilian
Engineering, ICCSCE 2016, pp. 464–467. Archives of Biology and Technology, 55(1), pp. 89–95.
[14] L. Zheng, M.-C. Van Labeke (2017). Long-Term Effects of Red- and
Blue-Light Emitting Diodes on Leaf Anatomy and Photosynthetic

82
 2021 the 13th International Conference on Computer and Automation Engineering

VDPred: Predicting Voltage Droop for Power-

Effient 3D Multi-core Processor Design
He Xiao Monodeep Kar Saibal Mukhopadhyay Sudhakar Yalamanchili
Cadence Design Systems, Inc. Georgia Institute of Technology Georgia Institute of Technology Georgia Institute of Technology
San Jose, CA, United States Atlanta, GA, United States Atlanta, GA, United States Atlanta, GA, United States
[email protected]

Abstract—Modern applications present rapid, time-varying to modify the design of on-chip voltage regulator leading to
demands to the power supply systems. Designing for worst-case more power efficient processor operation.
voltage droop leads to inefficient operations. These inefficiencies This paper proposes VDPred, a learning-based voltage
are amplified in 3D multi-core processors. Minimizing this voltage
variation is key to improving power efficiency. One widely regulator with high accuracy in predicting voltage droops for
accepted method is to apply droop compensation into the voltage the above purposes. VDPred also implements a misprediction
regulators. However, as the design complexity rises, the response handling unit to minimize the consequence of occasional
time of the droop compensation fails to keep pace with the speed prediction errors. We evaluate the performance of VDPred
of load current switching in the modern processors. using a cycle-level full system simulator integrated with a
In this paper, we propose VDPred, a droop prediction system
multi-physics library for power/energy modeling, which is
integrated with a reactive control loop inside a typical on-
chip buck regulator of the 3D processor. VDPred predicts and executed in conjunction with a circuit-level design of a on-
eliminates voltage droops that produce worst-case operating chip voltage regulator. We evaluate VDPred in the context of
voltage margins. It constructs an offline learning model for power a 3D multi-core processor that represents an important design
prediction based on the pipeline and cache microarchitectural point for power efficient processors. Over eight benchmarks
events. When VDPred predicts a hazardous droop, the circuitry
that include both memory and compute bound benchmarks,
warms up the regulator and pulls up the supply voltage against
a voltage loss before the droop happens. VDPred can deal with VDPred realizes an average power reductions of 3.5% (up
very fast droop events, on the order of nanoseconds, and thus to 7%), in which the power of VDPred is also considered.
the voltage guard-band and power consumption of the processor Importantly, we explore the relationships between learning
can be further reduced compared to existing state-of-the-art algorithms and substantive power consuming microarchitec-
techniques. We evaluate VDPred in both single- and four-core
ture events. Further, the design of VDPred reflects a co-
configurations of a 3D processor with the cycle-accurate full-
system simulator. Results show a significant reduction in voltage design approach of circuit and microarchitecture that produces
guard-band and up to 7% runtime power of the overall system. a refined regulator design coupled with a predictive model that
Keywords—3D processors, IVR, voltage droop, machine learn- is sensitive to microarchitecture events. We argue that such
ing & prediction, decision tree coordinated explorations across circuit and microarchitecture
abstraction boundaries are going to be necessary to improve
I. I NTRODUCTION the energy efficiency of future processors.
II. BACKGROUND AND M OTIVATION
The end of Dennard scaling has led to unsustainable growth
in power consumption and the focus on power efficiency in A. Integrated Voltage Regulator
processor design. Such advances are central to the effective The operation of the power delivery system has a significant
operation of all modern processors in platforms ranging from impact on the power/energy efficiency of processors. One key
mobile devices to data centers and to high-performance com- component is the voltage regulator, as shown in Figure 1,
puting (HPC) machines that drive national initiatives in key which delivers the power from an input source to the processor
areas such as science, finance, and defense. Consequently, circuits. Recently, voltage regulators in modern processors
multiple efforts at various levels of abstraction have been have advanced to an on-chip implementation [1] in anticipation
developed for the power efficient design and management of of its capability for fast voltage switching and fine-grained
multi-core processors. A particular challenge is to balance voltage control. In this paper, we report on developments with
the needs of computational performance against the needs an on-chip integrated voltage regulator (IVR) implemented
of power efficient computing, especially critical to the 3D with an inductor-based switching [2]. The switching circuit
processors, where the power consumption is a key design of the voltage regulator converts a DC voltage to a lower
factor given the limited thermal headroom. In this paper we voltage with the same polarity by alternately connecting and
present a cross-layer technique for coordinating power delivery disconnecting the source to an output inductor through pulse
and power consumption to realize gains in power efficiency width modulation (PWM) control [3].
with no impact on perfomance. Power consumption behavior An illustrative IVR is used with the following parameters:
at the microarchitecture level is utilized in a predictive manner L=3.3nH, C=25nF and FSW=400M Hz. The IVR outputs a

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 83

 VIN current sharing [9]. In this paper, we investigate the opportu-
dutyP nity of power reduction from voltage smoothing.
CLKSW Digital Buf M1
Unlike prior research that focused on circuits to improve
PWM dutyN
M2
transient responses, our approach predicts the worst-case volt-
ECOMP Buf
age droop by looking into the high-level microarchitectural
Digital
behaviors in the multi-core processor. By promoting a reliable
Compensator prediction of load transients in voltage regulation, we intent to
VOUT L
bridge the gap between existing techniques for voltage regu-
COUT
lation and the ideal scenario to minimize the required voltage
Err
ADC +- Processor
guard-band and the consequent power/energy consumption.
VREF
III. VOLTAGE D ROOP P REDICTION
Fig. 1. An On-chip Voltage Regulator Model Using a Buck Converter
Due to the complexity of power prediction , estimating
runtime power consumption in a multi-core processor is chal-
lenging. Many research efforts have delved into learning-based
methods to improve the accuracy of power prediction [10] [11]
[12] over a predefined linear power model. One interesting ob-
servation is the strong relationship between microarchitectural
events and voltage variations [13] - a relationship we explore
and utilize in this paper.
In this section, we propose a learning-based prediction
system to estimate near future power consumption for droop
compensation based on microarchitectural events. The predic-
tion system takes advantage of regression analysis between
microarchitectural events and processor power using several
alternative machine learning algorithms and proposes an effi-
cient approach for voltage droop prediction.

A. Significance of Microarchitectural Events

To diagnose the power behaviors, we identify critical mi-
croarchitectural events responsible for significant load tran-
sients. In fact, the pipeline stall and recovery contributes to
one scenario of this transient. The tested applications are
selected from the Splash-2 benchmark [14]. Figure 3 depicts
Fig. 2. IVR Matlab Model: a) Frequency Domain Response of Illustrative
IVR Design and b) Droop For Different Time Resolution of PWL Current two classes of pipeline activities that cause a significant power
increase when a single core executes raytrace.
In the first scenario, the occupancy of the last level cache
constant 0.85V voltage supply to the processor regardless of (LLC) miss status handling register (MSHR) reduces from 9
load transients, and has a phase margin of 43.5◦ and a unity to 7 at cycle 3630, indicating that the data from two LLC
gain bandwidth of 36M Hz in Figure 2 (a). The time resolution misses have been returned from memory. The data reach the
of the IVR control loop is 10ns, as shown in Figure 2 (b). pipeline in the next 20 cycles and the execution unit in the
pipeline resumes to work. Thus, the pipeline power increases
B. Motivation from 0.25W to 4.3W at cycle 3650. In the second scenario,
To prevent timing violations on critical paths experiencing a we track down instruction dependencies in the reorder buffer
voltage droop, circuit designers employ a voltage guard-band (ROB). Starting from cycle 7370, the instruction dependency
based on worst-case voltage droop and operate at correspond- decreases significantly. Instructions retire during this period
ing higher voltage. Such a conservative scheme of designing and pipeline can proceed to fetch and allocate instructions to
the power delivery systems exacts power and energy efficiency ROB in the next few cycles. Power of the pipeline frontend
costs. As device feature size continues to shrink, voltage increases and total power ramps up from 0.26W to 3.5W .
droops are getting worse [4]. These effects have a greater Consequently, we are motivated to develop a prediction
impact in 3D processors, where the dark silicon effect is more system that infers power consumption from pipeline events
pronounced. Consequently, to improve power efficiency there and cache activities. The new current demands are calculated
has been considerable effort on reducing voltage noise and from the predicted power. Note that several types of microar-
supply [5] [6] [7] [8]. Several voltage droop management chitectural events are also related and thus may be equivalent
circuits uses reactive control to mitigate large droops through in their predictive capabilities.

84
 5 4
4

Power (Watt)

Power (Watt)
3
3
2
2
1
1
0 0

3540 3560 3580 3600 3620 3640 3660 800 7300 7320 7340 7360 7380 7400 7420

Instruction Dependency
Cycle Cycle
9 750
LLC MSHR

700
8 650
600
7 550
500
6 450
400
3540 3560 3580 3600 3620 3640 3660 7300 7320 7340 7360 7380 7400 7420
Cycle Cycle
a b

Fig. 3. Power Snapshots of the Raytrace Application in Terms of a) LLC MSHR and b) instruction dependencies in ROB

a b c

Fig. 4. Validation Comparison of Learning Models in Terms of a) Mean Square Error, b) Mean Absolute Error and c) R2 Score

B. VDPred Learning Scheme TABLE I.

ACTIVITY C OUNTERS IN D ROOP P REDICTION
The prediction system utilizes occurrences of microarchi-
tectural events available from performance counters in the Feature Description
inst fetch number of instructions fetched per sampling interval
processor model to predict future power consumption. The
inst retire number of instructions retired per sampling interval
sampling interval sets to 10ns according to the bandwidth of inst alu number of ALU instructions in the ROB
our on-chip voltage regulator model in Figure 2 (b). rob occ ROB occupancy
A summary of the performance counters in VDPred is listed rob age sum of each instruction age in ROB
in Table I. Group I records pipeline information. We extend rob dep sum of each instruction dependency in the ROB
cache occ L1/LLC occupancy
the ROB to collect the on-the-fly instruction information cache miss missed requests in L1/LLC per sampling interval
that implies current application status including instruction cache mshr L1/LLC MSHR occupancy
types, age and dependencies. The instruction age records the llp2llc number of L1 requests to local LLC
average cycle of current instructions resided in the ROB llp2peers number of L1 requests to L1 peers
after allocation. Group II records the cache activities. We all2llp number of responses to L1
llc2mem number of LLC requests to memory
maintain separate performance counters in L1 and LLC to llc2peer number of LLC requests to LLC peers
track different types of cache requests and responses. mem2llc number of memory responses to LLC
To capture the relationship between performance counters
and power, we conduct offline regression analysis using five
regression models provided by the machine learning library Partial Least Square (PLS): The model finds a linear rela-
scikit-learn [15], as follows: tionship between two multivariate datasets. We reduce feature
Linear Regression (LR): The model fits a linear model with dimension to 5 and capture the feature correlation.
a minimized residual sum of squares. The R2 score among Multi-layer Perceptron (MLP): The model constructs a
applications ranges from 0.28 to 0.77 and LR is fairly accurate 2-layer network as a non-linear function approximator. We
for memory bounded applications. LR does not work well for restrict the total number of perceptrons to 50 in the first layer
compute bound applications. and 15 in the second layer.

85
K-nearest Neighbors (KNB): The model derives from the B. Misprediction Analysis
k-means algorithm using eight neighboring points. In our VDPred generates the enable signal for the compensation
experiments, KNB excels other models, especially for compute circuit from a comparison between present power consumption
bound applications (R2 score 0.91 in radiosity). It indicates and predicted future power, which achieves the upper bound
a strong clustering structure in performance counters. of power reduction with perfect prediction.
Decision Tree (DT): The model utilizes a non-parametric When an error takes place in the droop prediction, it pe-
method to predict power via simple decision rules. It works nalizes VDPred’s performance in power reduction. Substitute
well (comparable to KNB) in CPU-bounded applications that the predicted current change ∆i in Equation (1) to ∆i + i (i
have a strong non-linear relationship between performance is a prediction error), and the propagated error to the output
counters and power consumption. voltage becomes βi , as in Equation (2).
C. Power Inference Scheme
Vnew 0 = Vold (i) + α(Vref + β∆i + i − Vold (i))
The power model utilizes the collected performance coun- (2)
ters to infer processor power in the next 10ns with current → Vref + βi
performance counters. We validate the trained power models When the trained model contains noticeable prediction error,
using k-folds cross-validation [16] to estimate prediction errors the power consumption would rise. For example, the power
by plugging the model into the simulator for 500ms execution. consumption with LR predictor in ocean-c increases by over
The prediction results are shown in Figure 4. Both KNB 7% in the experiment. The total power increases for two
and DT show good prediction in the power consumption reasons. First, a misprediction is a false negative, in which
of the processor. Besides, the hardware implementation of VDPred does not predict a worst-case droop. As a result,
the decision tree is fairly simple compared to other models. the voltage guard-band is not reduced and the pull-up circuit
Therefore, we utilize the decision tree logic as the basic block contributes to the extra power. Second, a misprediction is false
in VDPred hardware to predict the voltage droop based on the positive, in which VDPred sends a false signal to the pull-up
microachitectural events. circuit. In this case, the pull-up circuit raises the supply voltage
and increases the processor power when unnecessary.
IV. VDP RED S YSTEM D ESIGN
VDPred is a voltage noise smoothing system built on top V. R ESULTS AND A NALYSIS
of the microachitecture-level droop predictor described in We demonstrate simulation results of VDPred on a 4-core
Section III. The droop predictor sits in the control loop of 3D processor using the Mainfold simulation framework [17].
the voltage regulator to predict the worst case runtime voltage Baseline implements a baseline voltage regulation without
and can effectively minimize the processor power. droop compensation. DT implements a voltage prediction
system based on decision tree with infinite depth. React
A. Droop Compensation
implements a reactive control scheme for droop compensation
The voltage drop during a transient is proportion to the via load current sharing in parallel operations. React enables
increase in current demand. Before a droop occurs, VDPred a bypass circuit to inject current to the processor when the
predicts a current increase in the processor and compensates output of the voltage regulator is below a preset threshold.
this possible upcoming voltage droop by a compensation cir- Ideal implements a perfect power prediction to compensate
cuit. The compensation circuit temporarily raises the reference voltage droops in the voltage regulator.
voltage in the feedback loop of a voltage regulator when
A. Hardware Characterization
current increases. Equation (1) shows the mechanism of droop
compensation in VDPred, where α is a design parameter of We characterize the compensation circuit in VDPred with
a voltage regulator and β = t/C is a parameter depending the step current signals in Figure 5 (a). We compare the droops
on the output network of the voltage regulator. Initially, the between VDPred and the baseline regulator. Results show that
feedback loop corrects the next voltage Vnew to Vref based the maximum droop reduces by over 0.1V (> 11% reduction
on the difference between reference voltage Vref and Vold , in a 0.85V reference) across the magnitudes of the current
given that processor current remains constant. This mechanism jump from 0.1A to 1.8A.
will fail if the current changes abruptly between control The implementation of the decision tree logic is based on
intervals and thus voltage regulation will be out of control. [18]. Figure 5 (b) gives the relationship between prediction
However, predicting the current change ∆i, we can add a accuracy and the depth of the decision tree. When the tree
droop compensation component β∆i to the feedback signal depth is increased, prediction accuracy is increased, but the
and update Vnew (i + ∆i) to Vref . hardware latency and power go up as well. We restrict the max
tree depth to 40 in our offline training so that the prediction
will be completed within 10ns when the circuit frequency
Vnew (i) = Vold (i) + α(Vref − Vold (i)) → Vref
set to 4GHz. The power of decision tree is determined by
Vnew (i + ∆i) = Vnew (i) − β∆i → Vref − β∆i the total bits processed per cycle multiply the energy per bit
(1)
Vnew (i + ∆i) = Vold (i) + α(Vref + β∆i − Vold (i)) (assume 2pj/bit in recent technology). The data width is 8bit
→ Vref (256 levels), and the total power of decision tree is 0.384W .

86
 1.6
baseline
with compensation
1.4

1.2

Mean Square Error

1.0

0.8

0.6

0.4

0.2

0.0

0 20 40 60 80 100 120

Tree Depth
a b

Fig. 5. a) Characterization of the Droop Compensation System w.r.t. Step Current and b) Prediction Accuracy w.r.t. Depth of the Decision Tree

DT

VDPred

1.0

R2 Score

0.8

0.6

0.4

barnes fmm lu-nc ocean-c radiosity radix raytrace water-ns

a b

Fig. 6. Comparison of a) Mean Square Error and b) R2 Score Between DT and VDPred

16
baseline react DT VDPred Ideal

14

12
Voltage Noise (%)

Normalized Power

10

barnes fmm lu-nc ocean-c radiosity radix raytrace water-ns

a b

Fig. 7. Comparison of a) Voltage Guard-Band and b) Power Reduction

87
B. Prediction Accuracy R EFERENCES
Figure 6 depicts prediction accuracy between DT and [1] E. A. Burton, G. Schrom, F. Paillet, J. Douglas, W. J. Lambert,
K. Radhakrishnan, and M. J. Hill, “Fivr x2014; fully integrated voltage
VDPred in terms of MSE and R2 score. VDPred outperforms regulators on 4th generation intel x00ae; core x2122; socs,” in 2014
DT in 6 out of 8 applications, indicating the effectiveness IEEE Applied Power Electronics Conference and Exposition - APEC
of prediction error handing in the system. For barnes and 2014, pp. 432–439, March 2014.
[2] M. Kar, A. Singh, S. Mathew, A. Rajan, V. De, and S. Mukhopadhyay,
raytrace, VDPred performs rather worse than DT , since “Exploiting fully integrated inductive voltage regulators to improve side
VDPred cannot capture abrupt power increase as counters channel resistance of encryption engines.,” in ISLPED, pp. 130–135,
changes within the 10ns window significantly. 2016.
[3] M. Kar, A. Singh, A. Rajan, V. De, and S. Mukhopadhyay, “An all-
digital fully integrated inductive buck regulator with a 250-mhz multi-
C. Power Reduction sampled compensator and a lightweight auto-tuner in 130-nm cmos,”
IEEE Journal of Solid-State Circuits, 2017.
Figure 7 (a) compares the results of voltage droop reduction. [4] V. J. Reddi, S. Kanev, W. Kim, S. Campanoni, M. D. Smith, G. Y.
VDPred achieves an average of 2.6% reduction in voltage Wei, and D. Brooks, “Voltage smoothing: Characterizing and mitigat-
guard-band compared to baseline. With mispredicting han- ing voltage noise in production processors via software-guided thread
scheduling,” in 2010 43rd Annual IEEE/ACM International Symposium
dling, VDPred outperforms React and DT by up to 3%. on Microarchitecture, pp. 77–88, Dec 2010.
For f mm, VDPred and DT have a relative high prediction [5] R. Bertran, A. Buyuktosunoglu, P. Bose, T. J. Slegel, G. Salem, S. Carey,
accuracy and obtain a roughly 5% improvement in droop R. F. Rizzolo, and T. Strach, “Voltage noise in multi-core processors:
Empirical characterization and optimization opportunities,” in 2014
reduction compared to baseline. VDPred also outperforms 47th Annual IEEE/ACM International Symposium on Microarchitecture,
React in most applications. For radiosity and raytrace, pp. 368–380, Dec 2014.
React outperforms VDPred by 1%-2% as VDPred has a [6] K. P. Sozański, “Modification of reactive power control algorithm,” in
Signal Processing Algorithms, Architectures, Arrangements, and Appli-
poor prediction performance due to abrupt changes in the cations SPA 2010, pp. 34–39, Sept 2010.
performance counter in 10ns. [7] Y. Kuai and P. L. Chapman, “Comprehensive and practical optimiza-
Figure 7 (b) presents the results of power reduction. The tion of voltage regulator modules,” in 2011 IEEE Power and Energy
Conference at Illinois, pp. 1–6, Feb 2011.
power reported in VDPred consists of the power consumed [8] P. J. Liu, Y. M. Lai, P. C. Lee, and H. S. Chen, “Fast-transient dc
in the processor and in the prediction hardware (i.e., roughly 8211;dc converter with hysteresis prediction voltage control,” IET Power
2% overhead). The average power saved in VDPred is 2.5% Electronics, vol. 10, no. 3, pp. 271–278, 2017.
[9] M. Ueno, K. Ito, Y. Ishizuka, and H. Matsuo, “A current compensation
compared to 1.2% in React. Specifically, VDPred obtains circuit for vrm in the sudden change of the load,” in 2005 European
a maximum power reduction of 9% in f mm even with Conference on Power Electronics and Applications, pp. 9 pp.–P.9, Sept
prediction power overhead. This is because VDPred manages 2005.
[10] B. Li, L. Peng, and B. Ramadass, “Accurate and efficient processor
to predict the worst-case droops when the processor executes performance prediction via regression tree based modeling,” J. Syst.
the f mm and reduces a 5% noise margin. Archit., vol. 55, pp. 457–467, Oct. 2009.
[11] A. Kerr, E. Anger, G. Hendry, and S. Yalamanchili, “Eiger: A framework
VI. C ONCLUSION for the automated synthesis of statistical performance models,” in 2012
19th International Conference on High Performance Computing, pp. 1–
In this paper, we propose a droop compensation system 6, Dec 2012.
[12] C. Zhang, A. Ravindran, K. Datta, A. Mukherjee, and B. Joshi, “A
VDPred to minimize power consumption by predicting the machine learning approach to modeling power and performance of
runtime voltage droop based on the microarchitectural events chip multiprocessors,” in 2011 IEEE 29th International Conference on
of the 3D processor. We analyzed the prediction performance Computer Design (ICCD), pp. 45–50, Oct 2011.
[13] V. J. Reddi, M. Gupta, G. Holloway, M. D. Smith, G. Y. Wei, and
of several regression models constructed by offline training D. Brooks, “Predicting voltage droops using recurring program and
and explored the design space of the prediction system. microarchitectural event activity,” IEEE Micro, vol. 30, pp. 110–110,
Unlike prior research focused on circuit events and improved Jan 2010.
[14] C. Bienia, S. Kumar, and K. Li, “Parsec vs. splash-2: A quantitative com-
transient responses, our approach integrates the high-level parison of two multithreaded benchmark suites on chip-multiprocessors,”
droop prediction system to the comprehensive IVR control in 2008 IEEE International Symposium on Workload Characterization,
loop. We demonstrate that our approach effectively reduces pp. 47–56, Sept 2008.
[15] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion,
the required voltage guard-band, and consequently, improves O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, et al.,
the overall power and energy efficiency of the processor. In “Scikit-learn: Machine learning in python,” Journal of Machine Learning
particular we are interested in understanding why some learn- Research, vol. 12, no. Oct, pp. 2825–2830, 2011.
[16] E. G. M. de Lacerda, A. C. P. L. F. de Carvalho, and T. B. Ludermir,
ing models create better predictors and thereby gain a deeper “A study of cross-validation and bootstrap as objective functions for
understanding of the power consumption behaviors of the genetic algorithms,” in VII Brazilian Symposium on Neural Networks,
multi-core processor. The combination of microarchitecture 2002. SBRN 2002. Proceedings., pp. 118–123, 2002.
[17] J. Wang, J. Beu, R. Bheda, T. Conte, Z. Dong, C. Kersey, M. Rasquinha,
and circuit models forms a co-design paradigm between circuit G. Riley, W. Song, H. Xiao, P. Xu, and S. Yalamanchili, “Manifold: A
and microarchitecture design for processor voltage regulator parallel simulation framework for multicore systems,” in 2014 IEEE
modules to achieve power efficient processors by (1) en- International Symposium on Performance Analysis of Systems and
Software (ISPASS), pp. 106–115, March 2014.
hancing droop predictions with learning based algorithms, (2) [18] J. R. Struharik, “Implementing decision trees in hardware,” in 2011 IEEE
designing control logic based on regulator characterization and 9th International Symposium on Intelligent Systems and Informatics,
(3) analysis of the design space of voltage-droop prediction for pp. 41–46, Sept 2011.
on-chip voltage regulators.

88
 2021 the 13th International Conference on Computer and Automation Engineering

Robotic Arm Motion Planning Based on Residual

Reinforcement Learning
Dongxu Zhou Ruiqing Jia Haifeng Yao Mingzuo Xie
School of Mechanical Electronic School of Mechanical Electronic School of Mechanical Electronic School of Mechanical Electronic
& Information Engineering & Information Engineering & Information Engineering & Information Engineering
China University of Mining and China University of Mining and China University of Mining and China University of Mining and
Technology (Beijing) Technology (Beijing) Technology (Beijing) Technology (Beijing)
Beijing, China Beijing, China Beijing, China Beijing, China
[email protected] [email protected] [email protected] [email protected]

Abstract—The application of reinforcement learning attractive and repulsive potential fields at different positions.
algorithms to motion planning is a research hotspot in robotics in This method is easy to implement, but the constructed potential
recent years. However, training reinforcement learning agents field has a local minimum problem. Algorithms based on
from scratch has low training efficiency and difficulty in sampling mainly include Probabilistic Road Map Method
convergence. In this paper, a robot motion planning method (PRM) [6] and Rapidly Expanding Random Tree Method
based on residual reinforcement learning is proposed. This (RRT) [7]. The sampling-based algorithm does not need to
method divides the agent's policy of motion planning into initial construct obstacles in the configuration space but only needs to
policy and residual policy. The initial policy is composed of a detect whether the sampled configuration positions collide in
neural network motion planner responsible for guiding the
the workspace. Therefore, their calculation speed is relatively
training direction of residual policy. The residual policy is
fast, and they are currently the primary method used in robotic
composed of Proximal Policy Optimization (PPO) algorithm in
reinforcement learning. A motion planning experiment is carried arm motion planning. Algorithms based on sampling have
out in a simulation environment, and the result shows that the considerable randomness. In recent years, improved algorithms
method can successfully perform motion planning. The of RRT have appeared one after another, such as RRT* [8],
comparison experiment between PPO and the proposed Informed RRT* [9].
algorithm demonstrates that the proposed algorithm has better The application of deep reinforcement learning algorithms
motion planning performance.
to motion planning is a new trend that has emerged in recent
Keywords—robotic arm, motion planning, PPO algorithm,
years. Compared with traditional motion planning algorithms,
residual policy methods based on machine learning have strong generalization
ability and robustness. Yuansheng Dong et al. applied the
I. INTRODUCTION DDPG algorithm to the path planning of mobile robots [10].
Richard Meyes et al. applied the Q learning method to the
The motion planning [1] of the robotic arm is to calculate
motion planning of industrial robots [11]. However, applying
the trajectory that can connect the starting point and the target
the deep reinforcement learning algorithm to the robotic arm's
point under a given environmental obstacle. In traditional
motion planning problem still encounters strenuous training.
industrial applications, the motion planning of the robotic arm
The training process of reinforcement learning is a process in
is often achieved through demonstration [2]. For application
which agents frequently interact with the environment and try
scenarios with changeable working environments, the
and make mistakes. It is difficult for some complex tasks and
traditional demonstration method has become increasingly
takes a long time to train the reinforcement learning agent from
challenging to meet users' needs to control robotic arms
scratch to convergence state.
quickly. Therefore, the motion planning of robotic arm has
become a hot research topic. In order to solve this problem and improve the training
efficiency of the reinforcement learning algorithm, this paper
Motion planning is also called path planning in the field of
proposes a motion planning method based on residual
mobile robots. The path planning of the mobile robot is carried
reinforcement learning. This method divides the policy of the
out in a two-dimensional or three-dimensional configuration
agent into an initial policy and a residual policy. The initial
space. In contrast, the motion planning of the robotic arm is
policy is composed of a neural network, and the residual policy
carried out in higher-dimensional configuration space. This
is composed of a deep reinforcement learning algorithm. The
difference in configuration space dimensions makes some
initial policy is used to guide the reinforcement learning
mobile robot motion planning algorithms unable to be applied
algorithm to train in a roughly correct direction. The
to robotic arms. Early classic path planning algorithms include
reinforcement learning algorithm's training process has
the A* algorithm [3] and Dijkstra algorithm [4]. Although they
changed from the original random trial and error to the
can be applied to robotic arms in theory, the algorithm
directional training process guided by the initial policy, which
calculations will increase exponentially. The artificial potential
significantly reduces the difficulty of training and improves
field method [5] guides the robotic arm's motion by setting
training efficiency.

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 89

 II. RELATION WORKS sampling principle to improve sample data's utilization
efficiency.
A. Markov Decision Process
The Markov decision process is a mathematical model of The objective function of the PG algorithm is shown in (2).
reinforcement learning. In reinforcement learning algorithms, It is expressed as the expected value of the product of the
objects that interact with the environment and make a series of policy action's log function and the advantage function At. The
decisions are called agents. The external objects that interact advantage function is responsible for indicating how good or
with the agent are called the environment. Fig. 1 shows the bad the execution of an action is relative to other actions in a
principle of the interaction process between the agent and the particular state.
environment. At each discrete time t = 0, 1, 2, 3..., the agent 𝐽𝑃𝐺 (𝜃) = 𝐸[𝑙𝑜𝑔𝜋𝜃 (𝑎𝑡 |𝑠𝑡 )𝐴𝑡 ] (2)
decides according to the observed environmental state St,
selects the action at to execute. After the action is executed, the The objective function of the PPO algorithm is shown in
agent receives a numerical instant reward Rt+1. At the same (3). It introduces importance sampling based on PG so that the
time, the environment enters the next new state St+1 due to the updated policy can still use the data sampled by the original
influence of the agent's behavior. policy for training. However, a constraint needs to be added to
the updated policy so that the new policy cannot be too
different from the old policy at the time of sampling.
𝜋𝜃 (𝑎𝑡 |𝑠𝑡 )
𝐽𝑃𝑃𝑂 (𝜃) = 𝐸 [ (𝑎𝑡 |𝑠𝑡 )
𝐴𝑡 ] − 𝜎𝐾𝐿(𝜃, 𝜃𝑜𝑙𝑑 ) (3)
𝜋𝜃
𝑜𝑙𝑑

The Trust Region Policy Optimization (TRPO) [14]

algorithm, the predecessor of the PPO algorithm, puts the
condition that the difference between the new and old
strategies cannot be too large outside the objective function
expression:
𝐾𝐿(𝜃, 𝜃𝑜𝑙𝑑 ) < 𝜀 (4)
Fig. 1. Schematic diagram of Markov decision process
In (4), KL represents the Kullback-Leibler divergence
The Markov decision's dynamic process can be expressed between the two strategies, and 𝜀 represents the set threshold of
as a sequence, as shown in Fig. 2. The agent observes the the degree of difference. However, the TRPO algorithm lists
initial environment state S0 and then decides to choose an the constraints separately, which is not conducive to actual
action a0 to execute. The environment switches to the next state calculations. In order to avoid this shortcoming, the PPO
S1 according to the transition probability p(s1,r1|s0,a0), and gets algorithm puts the constraints between the old and new
a reward r1. Then the agent and the environment continuously strategies directly into the objective function.
cyclically interact with the above process. Finally, a sequence
(s0, a0, r1, s1, a1, r2, s2, a2, r3...) is obtained. In this process, the III. RESIDUAL POLICY MOTION PLANNING
expected total return is expressed as: We first focus on a robot control method called residual
𝑅𝑡 = 𝑟𝑡+1 + 𝛾𝑟𝑡+2 + 𝛾 2 𝑟𝑡+3 + ⋯ = ∑∞ 𝑘
(1) reinforcement learning (RRL) [15]. In RRL, a robotic arm
𝑘=0 𝛾 𝑟𝑡+𝑘−1
needs to place a cube block in the narrow gap between the
other two cube blocks and ensure that the other two cube
blocks cannot be knocked over during the placement process.
The RRL method uses the dynamic controller designed by
manual modeling as the initial policy to move the small cube to
the target position. Then the behavior policy learned during the
Fig. 2. The Markov decision's dynamic process interaction between the reinforcement learning method and the
other two blocks is used as the residual policy. The controller's
B. Proximal Policy Optimization (PPO) Algorithm behavior action output is the superposition action of the initial
The PPO algorithm [12] is a reinforcement learning policy and the residual policy. In the end, RRL allowed the
algorithm based on the policy gradient method proposed by robotic arm to learn how to insert small objects into the narrow
Schulman of the OpenAI in 2017. Generally speaking, gaps that were not wide enough.
reinforcement learning algorithms are susceptible to the Inspired by the idea of superimposing the output of two
hyperparameters set. Besides, for the traditional policy gradient
policies in the RPL theory, this paper decomposes the robotic
(PG) [13] method, its training is challenging. This is because
arm's motion planning problem into the superposition of the
the PG algorithm is on-policy, and it is difficult for the agent to
action output of the initial policy and the residual policy action
obtain enough data conducive to training. The emergence of
output formed by the PPO algorithm. This method is called
the PPO algorithm effectively solves the above problems. On
residual policy motion planning (RMP).
the one hand, it reduces the sensitivity of the algorithm to
hyperparameters. On the other hand, it adopts the importance If the initial policy is 𝜋0 , for the input state 𝑠:

90
 𝑎0 = 𝜋0 (𝑠) (5) initial policy is perfect and when the initial policy is far from
optimal. The figure assumes that there are three action outputs.
If the residual policy is 𝜋𝑅 , for the same input state 𝑠:
A. The Initial Policy of Motion Planning Based on Neural
𝑎𝑅 = 𝜋𝑅 (𝑠) (6)
Network
Then the final behavior policy is: In the RMP method proposed in this paper, the initial
𝑎 = 𝑎0 + 𝑎𝑅 = 𝜋0 (𝑠) + 𝜋𝑅 (𝑠) (7) policy is used to guide the residual policy training composed of
reinforcement learning. For selecting the initial policy, we
Fig. 3 shows a schematic diagram of the initial policy and notice a method called Motion Planning Networks (MPNet)
the residual policy superimposed to output the complete [16]. This method utilizes the nonlinear fitting ability of the
motion planning policy. neural network for motion planning. The MPNet method
consists of two neural networks. The first neural network is
used to reduce the dimensionality of obstacle data. It receives
the obstacle point cloud data from the depth camera and
outputs the dimensionality reduction feature value. The second
neural network receives the obstacle feature value output by
the first network and the robotic arm's joint configuration angle
value in the current state. It then outputs the predicted joint
configuration angle value of the robotic arm in the next state.
For the training of two neural networks, MPNet uses a
supervised learning method. It uses the traditional motion
Fig. 3. The superposition principle of initial policy and residual policy planning algorithm RRT* to generate training trajectories and
train the neural network end-to-end.
In this initial-residual policy method framework, the initial
policy can initially guide the residual policy training The MPNet is essentially imitation learning. The RRT*
constituted by the reinforcement learning algorithm, turning algorithm cannot generate training trajectories very close to the
the once random trial and error into a particular directional edge of the obstacle, which will lead to the problem of data
training process. The guidance of the initial policy significantly sparseness of the imitation learning agent at the edge of the
shortens the training process of reinforcement learning. The obstacle. Moreover, MPNet needs to train two neural networks,
policy of the reinforcement learning part can provide the especially the first neural network that performs dimensionality
necessary policy supplements for the initial policy part. The reduction coding on obstacles is challenging to train. In this
initial policy often lacks self-adaptation and generalization, paper, the simplified MPNet method is used as the initial
which is the characteristic of the reinforcement learning policy, and only the second neural network used to predict the
algorithm. next robot arm configuration is retained. For the abstract
description of obstacles, we propose an obstacle abstract sphere
If an initial policy itself is close to perfect, then the concept. In actual operation, the obstacle is abstracted as a
superimposed residual policy output can be trained as an sphere circumscribed to its outer surface. Fig. 5 shows the
additional correction action close to zero output. Conversely, if process of abstracting a can obstacle as a sphere obstacle.
the initial policy is far from the optimal control policy, the
residual policy after training will have a robust corrective
effect on the initial policy. The initial policy only serves as a
directional guide for the residual policy training provided by
the reinforcement learning, which is used to simplify the
difficulty of the convergence of the reinforcement learning
algorithm and avoid the difficulties faced by training the agent
from scratch.

Fig. 5. The process of abstracting a can obstacle as a sphere obstacle

The position and radius of the abstract sphere and the

robotic arm's current configuration position are input to the
neural network. The neural network outputs the predicted next
Fig. 4. The schematic diagram of the residual action's role and the initial action configuration position and then uses the trajectory generated by
in the final output action the RRT* algorithm for training. Fig. 6 shows the training
process of the initial policy of motion planning composed of
Fig. 4 shows a schematic diagram of the residual action's neural networks.
role and the initial action in the final output action when the

91
 state. The critic part is responsible for estimating the state
value function V(s) of the current state. The initial policy in
RMP theory can guide PPO training in the right direction
before the residual policy has converged. However, the Critic
part parameters at the beginning of training are entirely
random values after initialization. This means that even if the
agent shows relatively good behavior under the initial policy's
guidance, it is difficult for the Critic part of the PPO algorithm
to give a correct evaluation. To solve this problem, we need to
pre-train the Critic part of the neural network in the PPO
algorithm before the complete training. The specific method is
Fig. 6. The training process of the initial policy of motion planning composed to let the agent train under the complete initial policy and
of neural networks residual policy, but keep the Actor part of the network
parameters unchanged and only train the Critic part. During
The abstracted obstacle sphere is not the circumscribed pre-training, the loss function of the Critic part of the neural
sphere of the actual obstacle but an inscribed sphere located network is monitored. When the temporal difference error of
inside the obstacle. Such an obstacle approximation method the value function estimated by the Critic part is less than the
will cause a part of the sphere's obstacles to be not considered. threshold y, it indicates that the Critic part can accurately
However, this method of simplifying the obstacles of estimate the state value function. The pre-training is stopped at
manipulator motion planning can quickly digitize the obstacle this time. After the pre-training is over, the Critic part's neural
features and then input them into the neural network. The RMP network parameters are retained, and then complete training is
method proposed in this paper allows the initial policy formed performed.
by the neural network to guide the residual policy formed by
the PPO algorithm in the right direction. Therefore, if there are C. Algorithm Design
unconsidered parts in the obstacle model simplified in the In the actual implementation, we divide the RMP method
initial policy part of the neural network motion planning, it can into three primary steps and then subdivide small steps in each
be supplemented by the residual policy of the reinforcement primary step.
learning part in the "trial and error" process of its training. This
principle is shown in Fig. 7. 1) Initial policy stage
S1. Initialize the neural network of the initial policy.
S2. Use RRT* algorithm to generate 500 motion planning
trajectories in a virtual environment for motion planning.
S3. Use the joint configuration sequence generated by
RRT* to train the neural network.
2) Pre-training stage
S1. Initialize the Actor and Critic parts of the PPO
algorithm. Set the last layer parameter of the Actor part to 0.
Fig. 7. The obstacle is considered to be avoided by the initial policy and the S2. Keep the Actor parameters of PPO unchanged. Train
residual policy separately. the comprehensive policy consisting of the initial policy and
the residual policy.
B. The Residual Policy Composed of PPO Algorithm
To enable the PPO algorithm's output actions to be S3. Detect the TD error of the Critic part. Stop pre-training
combined with the output actions of the initial policy formed when the TD error is less than the threshold y.
by the neural network motion planning, some modifications 3) Initial policy and residual policy superimposed output
need to be made to the PPO algorithm. training stage
1) Initialize the output layer parameter of the actor Train the comprehensive policy consisting of the initial
network to 0: If the initial policy itself is perfect, then the PPO policy and the residual policy.
algorithm's policy superimposed with the output must ensure
that the already perfect initial policy cannot deteriorate after IV. EXPERIMENTS AND RESULTS
superposition. The most extreme case is that the initial policy The purpose of the experiment is to verify the RMP
is perfect, and the output action of the PPO policy should be 0 method's effect in the robotic arm's motion planning. The
output. For the PPO algorithm to output 0 at the beginning of virtual experimental environment used is V-REP [18], and the
training, the parameters of the last layer of the actor network robotic arm is a desktop 6-DOF robot named Mirobot [19].
need to be initialized to 0. The virtual experimental environment constructed is shown in
Fig. 8. The robotic arm's task is to suck the small green block
2) Pre-training the neural network of the critic part: The
through the end-suction cup and then place it on the red target
PPO algorithm is composed of the Actor-Critic algorithm
position. There are two obstacles to avoid during the
framework[17]. The Actor part is responsible for outputting
movement of the robotic arm.
the probability distribution of actions according to the input

92
 The performance of the RMP method and the single PPO
algorithm in the same virtual task is compared. It can be seen
from Fig. 10 that the average reward value convergence speed
of the algorithm in the training iteration process is significantly
higher than that of the single PPO algorithm.

Fig. 8. Virtual experiment environment in V-REP software

The number of neural network neurons used for the initial

policy is 128*64*64, and the activation function uses ReLu.
The learning rate of the PPO algorithm is 0.01, and the
discount factor is 0.9. The PPO algorithm's input features are
the joint angle value, the coordinate value of the target position,
and the coordinate value of the end position of the robot arm.
The reward function is set as: Fig. 10. The performance of RMP method and single PPO algorithm in the
same virtual task
r1
V. CONCLUSION
   x, y , z    x, y , z  This paper presents RMP, a motion planning method of
 tip goal

 robotic arm based on residual reinforcement learning. This

 if  x, y, z tip   x, y, z  goal  10 (8) method decomposes the policy of motion planning into an
 initial policy and a residual policy. The initial policy is


  x, y, z tip   x, y, z  goal  x, y, z  tip
x, y , z  goal
composed of a neural network used to guide the reinforcement
learning algorithm's training in the residual policy. The residual
 if  x, y, z tip   x, y, z  goal  10
 policy consists of a reinforcement learning algorithm to ensure
the entire algorithm's adaptability and generalization ability.
r2  5, If the robotic arm touches an obstacle The virtual environment's experimental results show that the
r  r1  r2 RMP method's convergence speed is significantly faster than
the PPO algorithm.
In (8), (𝑥, 𝑦, 𝑧)𝑡𝑖𝑝 is the position of the end effector of the
desktop robot arm, (𝑥, 𝑦, 𝑧)𝑔𝑜𝑎𝑙 is the motion planning target REFERENCES
position, (𝑥⃑, 𝑦⃑, 𝑧⃑)𝑡𝑖𝑝 is the unit pose vector of the desktop robot [1] H. M. Choset et al., Principles of robot motion: theory, algorithms, and
implementation. MIT press, 2005.
end effector, and (𝑥⃑, 𝑦⃑, 𝑧⃑)𝑔𝑜𝑎𝑙 is the unit pose vector of the
[2] B. D. Argall, S. Chernova, M. Veloso, and B. Browning, "A survey of
motion planning target. robot learning from demonstration," Robotics and autonomous systems,
vol. 57, no. 5, pp. 469-483, 2009.
As shown in Fig. 9, RMP completes the above tasks
[3] F. Duchoň et al., "Path planning with modified a star algorithm for a
through motion planning in a virtual environment. mobile robot," Procedia Engineering, vol. 96, pp. 59-69, 2014.
[4] S. Broumi, A. Bakal, M. Talea, F. Smarandache, and L. Vladareanu,
"Applying Dijkstra algorithm for solving neutrosophic shortest path
problem," in 2016 International conference on advanced mechatronic
systems (ICAMechS), 2016: IEEE, pp. 412-416.
[5] Y.-b. Chen, G.-c. Luo, Y.-s. Mei, J.-q. Yu, and X.-l. Su, "UAV path
planning using artificial potential field method updated by optimal
control theory," International Journal of Systems Science, vol. 47, no. 6,
pp. 1407-1420, 2016.
[6] R. Geraerts and M. H. Overmars, "A comparative study of probabilistic
roadmap planners," in Algorithmic foundations of robotics V: Springer,
2004, pp. 43-57.
[7] S. Rodriguez, X. Tang, J.-M. Lien, and N. M. Amato, "An obstacle-
based rapidly-exploring random tree," in Proceedings 2006 IEEE
International Conference on Robotics and Automation, 2006. ICRA
2006., 2006: IEEE, pp. 895-900.
[8] F. Islam, J. Nasir, U. Malik, Y. Ayaz, and O. Hasan, "Rrt∗-smart: Rapid
convergence implementation of rrt∗ towards optimal solution," in 2012
Fig. 9. RMP completes the above tasks through motion planning in a virtual IEEE international conference on mechatronics and automation, 2012:
environment. IEEE, pp. 1651-1656.

93
[9] J. D. Gammell, S. S. Srinivasa, and T. D. Barfoot, "Informed RRT*: [15] T. Johannink et al., "Residual reinforcement learning for robot control,"
Optimal sampling-based path planning focused via direct sampling of an in 2019 International Conference on Robotics and Automation (ICRA),
admissible ellipsoidal heuristic," in 2014 IEEE/RSJ International 2019: IEEE, pp. 6023-6029.
Conference on Intelligent Robots and Systems, 2014: IEEE, pp. 2997- [16] A. H. Qureshi, A. Simeonov, M. J. Bency, and M. C. Yip, "Motion
3004. planning networks," in 2019 International Conference on Robotics and
[10] Y. Dong and X. Zou, "Mobile Robot Path Planning Based on Improved Automation (ICRA), 2019: IEEE, pp. 2118-2124.
DDPG Reinforcement Learning Algorithm," in 2020 IEEE 11th [17] D. Bahdanau et al., "An actor-critic algorithm for sequence prediction,"
International Conference on Software Engineering and Service Science arXiv preprint arXiv:1607.07086, 2016.
(ICSESS), 2020: IEEE, pp. 52-56.
[18] E. Rohmer, S. P. Singh, and M. Freese, "V-REP: A versatile and
[11] R. Meyes et al., "Motion planning for industrial robots using scalable robot simulation framework," in 2013 IEEE/RSJ International
reinforcement learning," Procedia CIRP, vol. 63, pp. 107-112, 2017. Conference on Intelligent Robots and Systems, 2013: IEEE, pp. 1321-
[12] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, 1326.
"Proximal policy optimization algorithms," arXiv preprint [19] D. Zhou, M. Xie, P. Xuan, and R. Jia, "A teaching method for the theory
arXiv:1707.06347, 2017. and application of robot kinematics based on MATLAB and V‐REP,"
[13] R. S. Sutton, D. A. McAllester, S. P. Singh, and Y. Mansour, "Policy Computer Applications in Engineering Education, vol. 28, no. 2, pp.
gradient methods for reinforcement learning with function 239-253, 2020.
approximation," in NIPs, 1999, vol. 99: Citeseer, pp. 1057-1063. [20] A. Babinec, L. Jurišica, P. Hubinský, and F. Duchoň, "Visual
[14] J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, "Trust localization of mobile robot using artificial markers," Procedia
region policy optimization," in International conference on machine Engineering, vol. 96, pp. 1-9, 2014.
learning, 2015: PMLR, pp. 1889-1897.

94
 2021 the 13th International Conference on Computer and Automation Engineering

Noncyclic Scheduling of Multi-cluster Tools with

Multi-type Wafers Based on Pareto Optimization
Yuanyuan Yan, Qinghua Tao, Huangang Wang, Wenhui Fan
Department of Automation, Tsinghua University, Beijing 100084, China
[email protected], [email protected], [email protected], [email protected]

Abstract—In semiconductor manufacturing, multi-cluster extended to the dual-armed cluster tool [10]. In [11], the
tools are widely used. Recently, the variety of wafer types has scheduling of dual-armed cluster tool with multi-type wafers
attracted increasing interests due to the demand of diversification was discusses, in which identical wafer flow patterns but
and efficiency in industrial production. In this paper, we focus on different process time were considered. They proved that the
the noncyclic scheduling problem of multi-type wafers in multi- conventional swap sequence is also optimal as for single-type
cluster tools. Firstly, a universal model is constructed to describe wafers. However, these method all consider cyclic scheduling
such problem for both single-armed and dual-armed robots. and specify for single-cluster tool. Wang et al. [12] studied the
Correspondingly, an improved dynamic programming algorithm dual-armed multi-cluster tools scheduling problem with multi-
is proposed based on Pareto optimization, in which a renumber
type wafers, involving transient operations and residency
rule and a normalization rule are introduced to merge redundant
states and further increase the searching efficiency. Theoretical
constraints. They established a non-linear mix integer
analysis of computational efficiency is also discussed. Numerical programming model and proposed a heuristic algorithm based
experiments verify the effectiveness of the proposed algorithm in on swap strategy, but the optimality of the solution cannot be
minimizing makespan on the noncyclic scheduling with multi- guaranteed. For noncyclic scheduling, Wikborg et al.
type wafers. employed Pareto optimization and proposed a dynamic
program approach based on the resource ready time for single-
Keywords—Noncyclic Scheduling, Multi-Cluster Tools, Multi- cluster tools with single-armed robots [13], in which the
Type Wafers Pareto Optimization minimum makespace was verified.
I. INTRODUCTION In this paper, beyond the existing works, we consider
In semiconductor manufacturing, multi-cluster tools are extending the scheduling of multi-type wafers to the more
widely used. Recently, with the development of wafer complex scenarios of the noncyclic scheduling of multi-cluster
customization and the increase of wafer size, different wafer tools with both single-type wafers and dual-armed robots [14].
flows can exist in practical production due to demands of Motivated by the efficiency and effectiveness of Pareto
different wafer types. To increase utilization of multi-cluster optimization, we further utilize Pareto optimization to resolve
tools, multiple types of wafers, which have similar but the multi-type wafers. In noncyclic scheduling problem of
different recipes, wafer flow patterns, and process time, are multi-cluster tools with multi-type wafers, there are more
concurrently processed in a tool [1]. challenging and difficult problems requiring to solve than that
with single-type wafers. Firstly, multi-type wafers can have
There have been sufficient works on the multi-cluster tools different wafer flows, which need to be distinguished during
scheduling, where most of them are based on single-type the modeling. Secondly, the stage of decision making in
wafers. Yi et al. decomposed multi-cluster tools problem into noncyclic scheduling of multi-type wafers is significantly
multiple independent single-cluster tool problems [2]. Chan et larger than that in single-type wafers, making the problem
al. analytically derived closed-form formulations for the intractable when directly adopting the existing methods in
minimal cycle time of two-cluster tools [3] and tree-like single-type wafers. Considering these difficulties, we make the
cluster tools [4], in which a resource-based algorithm was following contributions in this paper. We first build a
proposed with polynomial-time complexity. In [5], an universal model to describe the noncyclic scheduling problem
improved backward strategy was studied to present the of multi-type wafers. Then, in order to improve the efficiency
optimal scheduling for multi-cluster tools based on, residency of searching the solution, we introduce a renumbering rule and
constraints [6] and hybrid-armed robots [7]. In [8], the the normalization to reduce the redundancy in the searching
scheduling strategies for buffer modules in single-armed process. We also prove that the proposed method is capable of
multi-cluster tool can reach the lower bound of cycle time with obtaining the optimal solution in noncyclic scheduling
considering the residency time constraints. problem of multi-type wafers, and briefly analyze the
computing complexity of the proposed method.
With the multi-type wafers, there have been some works.
Wang et al. studied the steady-state scheduling of single- The remainder of this paper is organized as follows. The
armed cluster tool with multi-type wafers and residency time problem description is given in Section 2. Section 3 presents
constraints, and introduced a novel description for the multi- the details of modeling and corresponding algorithm. Section 4
type wafers, and presented an efficient algorithm to find the gives numerical experiment. Section 5 concludes the paper.
periodic steady-state schedule [9]. This method can also be

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 95

 II. PROBLEM DESCRIPTION k {1, 2} . P0 ( LLA) is defined as the starting process, and
A. Schematic of Multi-cluster Tools Pm1 ( LLB) as the ending process;
Pg ( PM i , j / BM i ,k / LL) denotes the process exist in whole
wafers, which is the integration ranking of Ph ,l considering the
process sharing;
pth ,l denotes the processing time of Ph ,l ;
rth ,l denotes the processing time of Ph ,l .

III. NONCYCLIC SCHEDULING BASED ON PARETO

Fig. 1. L-cluster tools. OPTIMIZATION
An L-cluster tool is composing of L single-cluster tools, A. Modeling for Multi-type Wafers
shown in Fig. 1. There are commonly 4 to 6 processing Inspired by [12], we can regard BM as a special PM, and
modules (PM), with a single-armed or dual-armed robot in different BM configurations can be considered as serial,
each single-cluster tool, where wafers can be transferred parallel or reentrant respectively. Since the wafer transfers
between the adjacent cluster tools by buffer modules (BM). between two processed require both unloading and loading
The first cluster tool has extra two load locks (LL), one for actions and the wafer may be hold in the robot for a period of
storing the wafers to be processed (LLA), and the other for time in dual-armed cases, the wafer in the robot can also be
storing the wafers finished (LLB). Similar to [12], variables of regarded as a step inserted between processes. Because the
the mentioned L-cluster are defined as follows: robot is responsible for the transmission of multiple processes
Ci denotes the i -th cluster tool, i  1,2,, L , where L is and the wafer needs to pass through the same robot many
the number of cluster tools; times, the robot can be regarded as a reentrant process. In this
Ri denotes the robot in Ci , Ri {1, 2} , which represents paper, we gives the description on the h-th type wafer flow as
single-armed or dual-armed robot respectively; follows:
PM i , j denotes the j -th PM in Ci , i  1,2,, L , Ph,0  R1  Ph,1  Ph,mh  R1  Ph,mh 1 .
j  1, 2, ; To describe the states, a vector s  ( z1 , z2 ,) is defined,
BM i ,k denotes the k -th BM which is connecting Ci and where zi denotes the process step of the i -th wafer and the
Ci 1 , i  1,2,, L  1 , k {1, 2} . N
dimension of s is the total number of wafers, i.e.  nh . For
h 1

B. Recipes of Multi-type Wafers example, z1 denotes the process step of the first wafer, z1  0
denotes the first wafer is in LLA; z1  1 denotes the first wafer
is in robot and it is to be load to Phˆ ,1 ; z1  2 denotes the first
wafer is in Phˆ ,1 ; z1  2(mhˆ  1) denotes the first wafer is in
LLB; where ĥ denotes the corresponding wafer type.
Therefore, s  (0,0,,0) represents the start state of wafers
waiting in LLA to be processed.
s  (2(mhˆ  1),2(mhˆ  1),,2(mhˆ  1)) is the final state of
wafers being finished in LLB. Due to the different number of
Fig. 2. Simple flow of 2-type wafers. wafer process steps, the value of each wafer also varies,
ranging from 0 to 2(mhˆ  1) . Particularly, in order to describe
In the scheduling problem of multi-type wafers, different
wafer flow patterns and different process times can exist. As the resource constraints, we introduce the resource state as
shown in Fig. 2, it is a 2-cluster tools with 2-type wafers, in spm  ( x1 ,, y1 ,) , where xi denotes the number of wafers
which the red line and blue line represent different wafer flows. in Pi , the value is limited by the number of PMs, yi denotes
For similar cases with two or more wafer flow patterns, we the number of wafers in Ri , the value is limited by the number
give the following definitions:
of arms of Ri .
N denotes the number of wafer types to be processed;
nh denotes the number of wafers of the h-th type of wafer When the robot action occurs, the process of the wafer
to be processed, h  1,2,, N ; steps forward, the state s and the resource state spm changes
nh denotes the number of processes of the h-th type of accordingly. For example, when we unload the first wafer
wafer needs to be complete, h  1,2,, N ; from LLA, the state of wafer 1 step forward, the value
Ph,l ( PM i , j / BM i ,k / LL) denotes the l -th process of the increases by 1, and the number of wafers in the corresponding
robot also does the same increase. Then, after we load the first
h-th type of wafer, and is processed in modules wafer to Phˆ ,1 , the value of the state of wafer 1 is added by 1 and
( PM i , j / BM i ,k / LL) , l  0,1,2,, mh , mh  1 , i  1,2, ,
the number of wafers in the corresponding robot minuses 1,

96
the number of wafers in Phˆ ,1 increases by 1. On the other hand, B. Algorithm for Multi-type Wafers
the state of resources restricts some actions, which is To find the optimal path in the reachability graph, we
infeasible. On the other hand, the state of resources restricts propose an improved dynamic programming algorithm based
some infeasible actions. If the corresponding robot is occupied on Pareto optimization. Firstly, related definition of Pareto
by wafers and has no free arms, it is not allowed to unload the optimization is introduced. Then, the renumber rule is
wafer. If the corresponding process is full with wafers without proposed to merge the redundant state. Afterwards, the
free PMs, it is not allowed to load the wafer. normalization rules and decision rules are deployed to further
In this way, the feasible transitions from the initial state to promote the convenience of searching the optimal solution.
the final state can form a state transition graph, which is 1) Pareto optimization
named as the reachability graph [13-14]. Fig. 2 shows a simple Similar with [13] and [14], to evaluate the paths based on
flow of 2-cluster tools with 2-type wafers, and the wafer flow ready time, the definition of dominate and Pareto optimal is as
can be described as follows:
P1,0 ( LLA)  R1  P1,1 ( PM1,1 )  R1  P1,2 ( BM1,1 )  and
R2  P1,3 ( PM 2,1 )  R2  P1,4 ( BM1,2 )  R1  P1,5 ( LLB) Definition 1. Path pij dominates pik , if vW ( pij )  vW ( pik ) and
P2,0 ( LLA)  R1  P2,1 ( PM1,1 )  R1  P2,2 ( BM1,1 )  , vR ( pij )  vR ( pik ) , pij , pik , where at least one inequality is
R2  P2,3 ( PM 2,2 )  R2  P2,4 ( BM1,2 )  R1  P2,5 ( LLB) required to be strict.
respectively. We assume that pt1,1  pt1,3  pt2,1  pt2,3  50 ,
Definition 2. A path is called Pareto optimal if there exists no
t1  t2  t3  1 , n1  n2  2 , and s0  (0,,0)  4
, hence the other path which dominates it, i.e. pik , k  j , pij is Pareto-
reachability graph can be constructed as Fig. 3.
optimal, if pik does not dominate pij .
The optimal solution is not affected by deleting the non-
Pareto optimal path. Only the Pareto optimal path is retained at
each state can effectively improve the search efficiency while
keeping the optimality. When the reachability graph is
extended to the final state send , we can get the optimal
makespan by Theorem1.
Fig. 3. Reachability graph for example of Fig. 2.
Theorem 1. For a path pend j
, if it is Pareto optimal, vR1 ( pend
j
)
pi is defined as the j -th path from s0 to si . vW ( p) is
j
is the minimum makespan.
defined to describe the time that the wafer is ready for next Proof: When the last action R1 loads the wafer to LLB,
action, and vR ( p) is the time that robot is ready for next wafers are all complete. The final state only has one effective
action. Action can be executable when the corresponding value, i.e. vR1 . Therefore, it is a single objective problem in
wafer and robot are ready. The executing time of the current the final state. The other paths pend k
would be dominated by
action is defined as at ( p) . After the action occurs, the state j
pend , k  j . Thus, the minimum makespan is vR1 ( pend
j
).
transfers, and the wafer ready time vW ( p) and robot ready
time vR ( p) should be updated. The update of at ( p) , vW ( p) , 2) Redundant state merging
When there are more than one wafer with the same wafer
vR ( p) is formulated as follows:
type, the process position of each wafer is updated and
at ( p)  max(vWˆ ( ppre ), vRˆ ( ppre )) , recorded separately, leading to a lot of redundant states and


unloading vWˆ ( p)  at ( p)  t1
vRˆ ( p)  at ( p)  t1  t2
,
wastes of searching time. For example, state s1 and s 2 in
Figure 3 describe the robot unload the first wafer and the

vv ((pp)) atat((pp)) ptt  t  t ,

second wafer from LLA. The two wafers are the same type, so
Wˆ Wˆ , Pˆ
loading 3
that the order of unloading wafers makes no difference in
Rˆ 2 3
makespan. State s1 and s 2 can be regarded as the same state
where ppre denotes the previous path, Ŵ , P̂ and R̂ denote the with different indices of wafers. Similarly, s3 and s 4 can be
corresponding wafer, process and robot respectively. In this regarded as the same state. Deleting these redundant states
example, ready time of path s0  s1  s5 is updated as Table greatly improves the searching efficiency. Then we introduce
I. the renumber rule to renumber wafers with the same type, and
arrange them in descending order according to the process
TABLE I. READY TIME UPDATE OF PATH s0  s1  s5 position. In this way, similar states can be classified into one
state. For example, state s2  (0,1,0,0) can be renumbered to
state vW1 ( p) vW2 ( p) vW3 ( p) vW4 ( p) vR1 ( p) vR2 ( p) at ( p)
s2  (1,0,0,0) , which is same as s1  (1,0,0,0) , and can be
s0 0 0 0 0 0 0 0 merged into one state. As described above, the reachability
s1 1 0 0 0 2 0 0 graph Fig. 3 will be simplified to Fig. 4, the number of states
s5 53 0 0 0 4 0 2 is reduced by more than half.

97
 C. Computational Efficiency
In the proposed algorithm, search efficiency mainly
depends on the number of states, which is determined by the
number of processes and the number of wafers. The upper
bound for the number of states in a reachability graph derived
Fig. 4. Simplified reachability graph of Fig. 3. and the details are given in Theorem 2.
N
3) Normalization rules and decision rules Theorem 2. O( (2(mi  1)  1)ni bounds the number of
In order to determine whether the path satisfies the i 1

residency constraints, otWˆ ( p)  at( p)  ptWˆ , Pˆ  t3  rtWˆ ,Pˆ is states, where ni is the number of wafers of the i -th wafer type,
introduced to describe the longest residency time of wafer. mi is the number of processes of the i -th wafer type.
When the infeasible Wˆ , at ( p)  otWˆ ( p) appears, the path is
Proof: The state is defined as s  ( z1 , z2 ,) , values of zi
infeasible and should be deleted.
ranges from 0 to 2(mhˆ  1) , which in total is 2(mhˆ  1)  1 .
Even if the wafers and robots are ready, next action time of Then the value of all states is the product of the state values of
the robots cannot be earlier than at ( p) . A normalization rule N
can be introduced during updating the ready time without each wafer, i.e. O( (2(mi  1)  1)ni .
changing the optimal makespan. The modified update ready i 1

time is written as vR ( p)  max(vR ( p), at ( p)) and When the number of wafers increases, the number of states
vW ( p)  max(vW ( p), at ( p)) . becomes large, so that the optimal makespan cannot be
obtained in acceptable time. However, with the deleting of
4) Algorithm framework redundant states, the actual number of states is actually far less
By solving the noncyclic scheduling problem, the recipe is N
Ph,0  R1  Ph,1  Ph,mh  R1  Ph,mh 1 . We define the than the upper bound given by O( (2(mi  1)  1)ni .
i 1
initial state and delete the non-Pareto paths while extending
the reachability graph. When we arrive at the final state, the Furthermore, the ratio between the numbers of states before
optimal solution can be obtain. The whole algorithm is and after the deletion is shown as Lemma 1.
summarized as follows. Lemma 1. The ratio between the numbers of states before
Algorithm 1: Dynamic programming algorithm based on Pareto N
optimization for noncyclic scheduling of multi-cluster tools with multi-type and after the deletion is Ratio   ni ! .
wafers i 1
Input: Initial state, recipe
Output: Makespan and the optimal path Proof: As the number of the wafers of i -th wafer type is ni ,
Add initial state as the starting node in the list and take it as the current
node S c ;
these wafers can form ni ! states before renumbering.
While Sc is not the final node do However, there is one state after renumbering. So, the ratio
between the numbers of states before and after the deletion is
Find all executable actions of Sc ; N
For each executable action of Sc do Ratio   ni ! .
Extend the node to Scnext ; i 1

Renumber the node Scnext ;

IV. EXPERIMENTS
If Scnext exists then
In this section, numerical experiments are conducted to
Select the node Scnext ;
Else
evaluate the overall performance of the proposed method. The
Add the node Scnext to the list, and then select Scnext ; proposed algorithm is implemented using Matlab R2013b with
End Intel i7-4770, 3.40GHz CPU, 8G RAM and Windows 7.
For each path in Sc do We conduct the experiment by the flow showing in Figure
Extend the path to Scnext ; 3. As the method in [14] is for one-type wafer, the experiment
If the path meets the residency constraints then schedule two types of wafers in turn. The result is shown in
Modify the ready time vR ( p ) and vW ( p ) ;
Table II. “Original method” means the method in [14], and
Add the path to Scnext ; “Improved method” means the method in this paper. “A” and
Remove all paths which are non-Pareto optimal; “B” indicates the best completing time of different type wafer
End
End by the original method. “Makespan” shows the best
End completing time in seconds (s), which is the sum of “A” and
Remove S c ; “B” in the original method.
Pick the first node in the list as Sc ;
End

98
 TABLE II. EXPERIMENT RESULT OF FIG. 2 ACKNOWLEDGMENT
n
Original method Improved method This project is jointly supported by National Science and
A B makespan makespan Technology Innovation 2030 Major Project
1/1 116 116 232 173 (2018AAA0101604) of the Ministry of Science and
2/2 173 173 346 287 Technology of China.
3/3 230 230 460 401
4/4 287 287 574 515
REFERENCES
5/5 344 344 688 629 [1] Pan C R, Zhou M C, Qiao Y, et al. Scheduling cluster tools in
semiconductor manufacturing: Recent advances and challenges. IEEE
Transactions on Automation Science and Engineering, 2018, 15(2):
To enrich the experiment, we introduced another set-up 586601.
with more PMs and parallel flow. The wafer flows are set as: [2] Yi J, Ding S, Song D, et al. Steady-state throughput and scheduling
analysis of multicluster tools: A decomposition approach. IEEE
P1,0 ( LLA)  R1  P1,1 ( PM1,1 )  R1  P1,2 ( BM1,1 )  R2 Transactions on Automation Science & Engineering, 2008, 5(2): 321-
 P1,3 ( PM 2,1 )  R2  P1,4 ( PM 2,3 PM 2,4 )  R2  , 336.
P1,5 ( BM1,2 )  R1  P1,6 ( PM1,3 )  R1  P1,7 ( LLB) [3] Chan W K, Yi J, Ding S. Optimal scheduling of multicluster tools with
P2,0 ( LLA)  R1  P2,1 ( PM 1,2 )  R1  P2,2 ( BM 1,1 )  R2 constant robot moving times, part I: Two-cluster analysis. IEEE
Transactions on Automation Science & Engineering, 2011, 8(1): 5-16.
 P2,3 ( PM 2,2 )  R2  P2,4 ( PM 2,3 PM 2,4 )  R2  ,
P2,5 ( BM1,2 )  R1  P2,6 ( PM1,4 )  R1  P2,7 ( LLB) [4] Chan W K, Ding S, Yi J, et al. Optimal scheduling of multicluster tools
with constant robot moving times, part II: Tree-like topology
configurations. IEEE Transactions on Automation Science &
where the processing time is: pt1  (0,60,0,70,40,0,50,0) and Engineering, 2011, 8(1): 17-28. Q. H.
pt2  (0,50,0,60,50,0,60,0) . The result is shown in Table III. [5] Zhu, N. Q. Wu, Y. Qiao, and M. C. Zhou, “Petri net-based optimal one-
wafer scheduling of single-arm multi-cluster tools in semiconductor
manufacturing,” IEEE Transactions on Semiconductor Manufacturing,
TABLE III. EXPERIMENT RESULT OF 2-TYPE WAFERS WITH MORE PMS
vol. 26, no. 4, pp. 578–591, 2013.
Original method Improved method [6] Zhu Q H, Wu N Q, Qiao Y, et al. Scheduling of single-arm multi-
n cluster tools with wafer residency time constraints in semiconductor
A B makespan makespan
manufacturing. IEEE Transactions on Semiconductor Manufacturing,
1/1 242 242 484 246 2015, 28(1): 117-125.
2/2 315 309 624 317 [7] Yang F J, Wu N Q, Qiao Y, et al. Petri net-based optimal one-wafer
3/3 388 376 764 388 cyclic scheduling of hybrid multi-cluster tools in wafer fabrication.
4/4 461 443 984 461 IEEE Transactions on Semiconductor Manufacturing, 2014, 27(2): 192-
203.
The makespan denotes the optimal completing time as [8] Bai L , Wu N , Li Z , et al. Optimal One-Wafer Cyclic Scheduling and
Buffer Space Configuration for Single-Arm Multicluster Tools With
desired in practical applications. As shown in Table II and Linear Topology. IEEE Transactions on Systems Man & Cybernetics
Table III, the makespan of the proposed method, is much Systems, 2016, 46(10):1456-1467.
better than that in the original method, verifying the optimality [9] Wang J , Pan C , Hu H , et al. Scheduling of single-arm cluster tools
of the solution of the proposed algorithm. Especially, when with multi-type wafers and shared PMs// 2017 13th IEEE Conference
the flow of the 2-type wafers has the appropriate matching on Automation Science and Engineering (CASE). IEEE, 2017.
situation, the makespan can be greatly reduced, even with the [10] Wang J, Hu H, Pan C, et al. Scheduling dual-arm cluster tools with
same makespan of 1-type. It is because mixed processing can multiple wafer types and residency time constraints. IEEE/CAA Journal
of Automatica Sinica, 2020, 7(3):776-789.
reduce the waste of equipment resources, and the improved
[11] Ko S G, Yu T S, Lee T E. Scheduling Dual-Armed Cluster Tools for
method can obtain the best makespan of the mixed processing. Concurrent Processing of Multiple Wafer Types With Identical Job
V. CONCLUSION Flows. IEEE Transactions on Automation ence and Engineering, 2018,
PP(99):1-13.
This paper builds a universal model to describe the [12] Wang Z, Zhou B, Lu Z, et al. An equivalent conversion method for
noncyclic scheduling of multi-cluster problem with multi-type dual-armed multi-cluster tool scheduling problems with multi-wafer
wafers, suitable for both single-armed and dual-armed robots. types. International Journal of Manufacturing Technology and
Management, 2019, 33(1/2):14.
An improved dynamic programming algorithm is proposed
based on Pareto optimization, in which a renumber rule and a [13] Wikborg U, Lee T E. Noncyclic scheduling for timed discrete-event
systems with application to single-armed cluster tools using pareto-
normalization rule are deployed to merge redundant states and optimal optimization. IEEE Transactions on Automation Science &
increase the searching efficiency. Theoretically, we analyze Engineering, 2013, 10(3): 699-710.
the computational efficiency, where a simple upper bound on [14] Yan Y, Wang H, Tao Q, et al. Noncyclic Scheduling of Multi-Cluster
the number of the states in the reachability graph is developed Tools with Residency Constraints Based on Pareto Optimization. IEEE
and the improvement on the reduction of states is also Transactions on Semiconductor Manufacturing, 2020, PP(99):1-1.
discussed. Numerical experiments show the effectiveness of
the proposed algorithm in minimizing the optimal scheduling,
i.e. makespan.

99
 2021 the 13th International Conference on Computer and Automation Engineering

The Application of Acceleration-Level Quaternion

Interpolation in the Visual-Servoing Process
Xiaoyu Zhao Yang Liu Zongwu Xie*
State Key Laboratory of Robotics and System State Key Laboratory of Robotics and System State Key Laboratory of Robotics and System
Harbin Institute of Technology Harbin Institute of Technology Harbin Institute of Technology
Harbin, China Harbin, China Harbin, China
[email protected] [email protected] [email protected]

Abstract—Visual servoing process is widely applied in robot In this paper, the quaternion is applied to represent the
control. In the visual servoing process, an interpolation of the robot end attitude. We propose an acceleration-level
robot end pose is necessary. In the interpolation, quaternion continuous pose interpolation method with a smoother
suffers a lot for the discontinuity in trajectory velocity and trajectory and applying it to the visual servoing process.
acceleration. In this paper, we propose an acceleration-level Simulations and experiments are carried out to validate the
continuous interpolation method and apply it to visual servoing method.
process. In the method, we make up use of the conversion
between different rotation descriptors and achieve a smooth The organization of this paper is as follows. Section II
enough trajectory. Simulations and experiments are carried out describes the visual servoing process. Then, acceleration-level
to validate the method and the results show considerable continuous pose interpolation method is derived in Section III.
performance. To implement the visual servoing, robot controller is
introduced in Section IV. The simulations and experiments
Keywords—Visual servoing, Quaternion interpolation, shown in Section V validate the method. The paper is
Acceleration-level continuity, Robot control concluded in the last section.
I. INTRODUCTION II. THE INTRODUCTION OF VISUAL SERVOING PROCESS
Visual servoing is widely used in robots system to increase A common strategy for the visual servoing is PBVS. The
the robot positioning accuracy [1]. In the robot controlling PBVS employs the hand-eye camera as the measurer and plans
process, a commonly used method is the position based visual the robot end trajectory in the controller. The processing
servoing (PBVS) [2, 3]. This method plans the robot end efficiency of the hand-eye camera always leads a much larger
velocity with the measured data from camera in a macroscopic visual sampling period compared with the robot control period.
view. However, it ignores some detailed information, for Then a complete visual servoing process can be divided into
example, the visual sampling period is much larger than the four parts: measurement, prediction, interpolation and
robot control period [4, 5]. As a result, it is necessary to make controlling.
an interpolation during the visual servoing period.
In the PBVS, the robot end velocity is calculated by (1),
Since the three components of the robot end position are and the prediction can be accomplished base on the end
decoupled and satisfying the holonomic constraints, all velocity.
interpolation methods used for one dimension are convenient
to be applied to the position vector directly, such as the Ve =κeae (1)
polynomial or trigonometric interpolation method [6]. For the T
where Ve = (vTe, ωTe) is the spatial velocity; κ is the coefficient
robot end attitude, however, the interpolation process becomes between the spatial velocity and the pose error; eae is the error
complex for the coupling and nonholonomic constraints. between the robot end pose and the aim pose.
Different kinds of attitude descriptors proposed for the
trajectory planning are faced with various problems. The To calculated the pose error eae, a proper descriptors of the
rotation matrix and angle-axis are faced with complex attitude is needed. In this paper, the quaternion is applied for
description of rotation error, and thus restricts the application singularity-free and simple attitude error expression. Suppose
on robot closed-loop control; the Euler-Angle suffers from the the pose vector is denoted by ρ = (pT, qT), where p is the
representation singularity or named gimbal lock [7], and may displacement vector and q = [η, ϵT]T is the quaternion vector,
result in failed trajectory planning. The interpolation of then the pose error can be estimated by:
quaternion has been studied by many researchers [8-10], and
 pa  pe  (2)
methods, such as spherical linear interpolation, are proposed. eae  f  ρa , ρe    
However, these methods did not ensures an acceleration-level 
 e a   a e  a  e
continuity. Method proposed in [11] tried to plan the The prediction of the end pose is then obtained:
quaternion in an acceleration-level continuous way during the
visual servoing process. However, the final trajectory suffers
 p0  veT 
from small waves in the visual-servoing period and affects the ρT   1 ω T  (3)
trajectory smoothness. e
2 e
 q0 

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 100

where ρT is the perdicted pose, the value of T equals to the where ΔAB = xA - xB, ΔCB = xC - xB, h = (t + tacc) / (2tacc).
visual sampling period and ⨂ denotes the quaternion product. If the time t is located in the interval [tacc, T1 - tacc], the
curve equation can be obtained by:
III. THE INTERPOLATION OF VISUAL SERVOING PROCESS
CB (7)
The prediction process accomplishes the trajectory x= t + xB
planning with the sampling frequency equaling to the visual T1
sampling frequency. In order to achieve the robot motion CB (8)
control, a further interpolation is needed within the visual x=
T1
sampling period. If the trajectory is interpolated with x =0 (9)
polynomial of degree five, the velocity obtained by (1) is
If t > T1 - tacc, we can set xA = xt, xB = xC, xC = xD and then
treated as the instantaneous velocity at the sampling moment
the integration process goes into a cycle.
[11] and small waves existing in each sampling period suggest
that the planned trajectory is not as smooth as excepted. An B. The Attitude Interpolation
alternative interpolating method is the linear trajectory with The interpolation of attitude is different from the
polynomial blends of degree four (LTPB4) [10, 12]. The interpolation of the position, since the integration of the
method is acceleration-level continuous and it treats the angular-velocity does not equal to the attitude.
velocity as the average value during the sampling period. In
this way, the interpolation curves are smoother and shows There are several descriptors of end attitude, and the
better performance. quaternion stands out because it is singularity-free and has a
simple form of attitude error.
A. The Position Interpolation and Linear Trajectory with
In this paper, we propose an interpolation method of the
Polynomial Blends of Degree Four quaternion to erase the quaternion ambiguity and obtain an
The LTPB4 can be applied to the position interpolation acceleration-level continuous trajectory. The method can be
since the three components of position are decoupled. Take x accomplished by the following three steps.
direction as an example, the trajectory can be obtained as
Step 1: Convert the quaternion into another attitude
follows.
descriptor, the angle-axis.
The interpolation process is depicted in Fig. 1. The robot Step 2: Interpolate the angle-axis with LTPB4. The LTPB4
end is located at point A at the current time, and B, C and D can be applied since the angle-axis is decoupled. And the
are the following predicted points. The process consists of an real-time desired angle-axis, together with its first and second
acceleration motion with time 2tacc and a uniform motion order derivatives, is obtained.
followed by another cycle of acceleration and uniform motion.
Step 3: Convert angle-axis related variables back to
real-time desired quaternion, angular-velocity and
angular-acceleration for further robot control.
The acceleration-level continuous interpolation for
quaternion is not as convenient as for Euler angle or angle-axis.
Considering the representation singularity of Euler angle, it is
reasonable to accomplish the interpolation of quaternion with
the help of angle-axis. To convert the current and predicted
quaternions to angle-axis, the following equations are
employed.
2 arccos( )
 q0T qT  0 (10)
 
2 arccos( )  2π q0 qT  0
T


n= (11)
Fig. 1. Linear trajectory with polynomial blends of degree four
where ϕ and n are the angle and axis of angle-axis respectively;
q0 and qT are the current and predicted quaternion respectively;
If the time t is located in the interval [-tacc, tacc], the curve || ∙ || is the norm of corresponding variable.
equation can be obtained by:
Then the angle-axis ψ can be obtained by:
 t   (12)
x    CB acc  AB   2  h  h2  2AB  h  xB  AB (4) ψ = n
 T1   The two values in (10) suggest that two possible rotation
 t   1 directions exist with the same initial attitude. qT0qT is employed
x    CB acc  AB   3  2h  h 2  AB  (5)
to erase the ambiguity and ensure a minimum rotation angle.
 T1   tacc
Since the three components of angle-axis are decoupled,
 t  3h (6) the LTPB4 is applied as shown in (4)~(9), and Ψd = [ψTd, ψ̇ Td ,
x   CB acc  AB  1  h  2
 T1  tacc ¨Td ]T is obtained.
ψ

101
 To finish the quaternion trajectory interpolation, Ψd should
be converted back to the trajectory related variables, that is Qd †
( ) (
θ = J † Vt - Jθ + I - J † J ξ ) (21)
= [qTd, ωTd , αTd ]T. The conversion process can be finished as where J is the inverse matrix of J.
follows. Subsisting (19) into (21) gives:

     
q  cos   sin   nT 
T
(13) ( ) (
θ = J † Vd + DVdt + Kedt - Jθ + I - J † J ξ(22) )
A common way of controlling robot is to calculate the
 2 2 
robot inverse dynamics as (23)
 1  cos     sin   
ω   I33 
 
 n  

 n  n   ψ

(14)
( ) ()
τ = M ( θ) θ + C θ,θ θ + Fv θ + Fs sgn θ + g ( θ) (23)
where τ is the joint torque; M is the inertia matrix; Cθ̇ is
   sin     2  2 cos     n    Coriolis and centripetal force term; Fv θ̇ and Fs sgn(θ)
̇ are the
α  ψ
    3sin     cos    2   n   n   
2 viscous friction and Coulomb friction respectively; g is the
  gravity term.
 1  cos     sin    (15)
  I 33   n    n  n   ψ To sum up, the total visual servoing process can be
    concluded into Fig. 2. In the figure, P is a position related
variable and is defined by [pT, vT, aT]T. Besides, in the figure,
  sin  
 ψ  nψ  the judgement module surrounded by the dotted line box is
 2
used to eliminate the ambiguity of quaternion, and it plays the
where n can be obtained by ψ / ||ψ||; ϕ can be obtained by ||ψ||; same role as the judgement in (10).
ϕ̇ can be obtained by (16) and n× is defined as (17).
 = ψTn (16)
 0  nz ny 
 
n   nz 0  nx  (17)
 ny nx 0 

C. Closed-Loop end Trajectory Tracking
Considering the following second-order error model:
Vdt + DVdt + Kedt = 0 (18)
where ∆V̇dt is the error between the desired and real-time end
spatial acceleration, and can be obtained by ∆V̇dt=[(ad - at)T,
(αd - αt)T]T; ∆Vdt is the error between the desired and real-time
end spatial velocity, and can be obtained by ∆Vdt=[(vd - vt)T,
(ωd - ωt)T]T; edt is the end pose error and can be obtained by (1),
edt = f (ρd, ρe); D and K are related coefficients.
Then the real-time end spatial acceleration used for
trajectory tracking can be obtained by:
Vt = Vd + DVdt + Kedt (19)
It can be proved that if the robot end acceleration is
controlled in the way shown in (19), the system works with
satisfactory tracking performance.
IV. THE ROBOT INVERSE KINEMAICS AND CONTROL
A complete visual servoing process includes the end pose
planning process and robot control process. For the latter, it is
necessary to map the robot end pose information to robot joint
space, which is also named by inverse kinematics.
Appling the velocity kinematics gives:
Vt = Jθ (20)
where J is Jacobian matrix and θ is the joint angle.
Take the time derivative and we can get: Fig. 2. The scheme of acceleration-level continuous visual servoing process

102
 V. THE SIMULATOIN AND EXPERIMENT VERIFICATION In Fig. 4, the initial robot end pose is represented by OI and
To verify the proposed algorithm, simulations and the target pose is noted by OT. The figure shows that the robot
experiments are carried out on the robot shown in Fig. 3 [13] end moves from the initial pose to the target pose gradually
and the corresponding D-H parameters and inertia parameters and the progress shows a considerable convergence.
are listed in Table I and Table II.

TABLE I. D-H PARAMETERS OF ROBOT SYSTEM

Joints θi (º) αi (º) ai (m) di (m)

1 0 0 0 0.225
2 0 90 0 0.24
3 -90 -90 0 0.24
4 0 0 0.56 0.2
5 90 0 0.5 0.195
6 0 90 0 0.18
7 0 -90 0 0.476

TABLE II. INERTIA PARAMETERS OF ROBOT SYSTEM

(a) The robot system (b) The D-H coordinate system
Link1 Link2 Link3 Link4 Link5 Link6 Link 7
6.34 6.34 7.93 5.67 2.8 2.7 10.01 Fig. 3. A 7-degree-of-freedom robot system used in visual seroving precess
Mass(kg)
0 0 0.23 0.24 0 -0.03 0
Center of -0.07 0.07 0 0 -0.03 0.02 -0.01
Mass (m)
-0.05 -0.05 -0.03 0 -0.03 0 -0.2
Ixx (kg‧m2) 0.09 0.09 0.08 0.03 0.01 0.01 0.15
Ixy (kg‧m2) 0 0 0 0 0 0 0
Ixz (kg‧m2) 0 0 0.12 0.05 0 0 0
Iyy (kg‧m2) 0.05 0.05 0.57 0.33 0.01 0.01 0.2
Iyz (kg‧m2) 0.02 0.02 0 0 0 0 0.01
Izz (kg‧m2) 0.07 0.07 0.53 0.31 0.01 0.01 0.1

Simulations are carried out firstly to verify the

acceleration-level continuous algorithm. In the simulation,
though the attitude is planned by quaternion, we presented the
attitude in the paper by Euler Angle for it is brief and clear to
understand. Suppose that when measured by the camera, the
position and ZYX-Euler-Angle of the target with respect to the
robot end is [0, -0.015, -0.5]m and [10, 15, 5]º respectively.
And the target is located at [0, 0, 0]m and [0, 0, 0]º. Then
trajectory of the robot end coordinate is shown in Fig. 4 and Fig. 4. Visual seroving end pose motion of the robot in the similation form
the end motion is shown in Fig. 5. OI to OT

(a) The position trajectory of the robot end (b) The attitude trajectory of the robot end
Fig. 5. robot end trajectory during the Visual servoing simulation

103
(a) The measued position of the robot end (b) The measured attitude of the robot end
Fig. 6. Visual servoing result of the experiments

In Fig. 5, the robot end pose, linear and angular velocities, REFERENCES
and linear and angular accelerations are illustrated. In the
[1] S. Hutchinson, G. D. Hager, and P. I. Corke, "A tutorial on visual servo
figure, the pose curves show smooth and asymptotically control," IEEE transactions on robotics and automation, vol. 12, no. 5,
convergent trajectories; the velocity curves show that during pp. 651-670, 1996.
the visual sampling period, the robot end almost keeps a [2] A. A. Hafez, V. Anurag, S. V. Shah, K. M. Krishna, and C. Jawahar,
constant velocity and the transition parts are all smooth; the "Reactionless visual servoing of a dual-arm space robot," in 2014 IEEE
acceleration curves show that the trajectories ensure International Conference on Robotics and Automation (ICRA), 2014:
IEEE, pp. 4475-4480.
acceleration-level continuity.
[3] C. Francois and S. Hutchinson, "Visual servo control Part I: Basic
It can also be inferred that in the interpolation part of the approaches," IEEE Robot. Autom. Mag, vol. 13, pp. 82-90, 2006.
visual sampling process, the LTPB4 can be replaced by other [4] A. C. Allen, C. Langley, R. Mukherji, A. B. Taylor, M. Umasuthan, and
acceleration-level continuous algorithm. Thus the quaternion T. D. Barfoot, "Rendezvous lidar sensor system for terminal rendezvous,
interpolation algorithm can be applied to other areas. capture, and berthing to the International Space Station," in Sensors and
Systems for Space Applications II, 2008, vol. 6958: International Society
To further validate the algorithm, experiments are carried for Optics and Photonics, p. 69580S.
out. The target is located about [-0.02, 0, 0.3]m and [-2, 2, 2.5]º [5] P. Motaghedi, "On-orbit performance of the orbital express capture
away from the robot end at the initial time. The result is shown system," in Sensors and Systems for Space Applications II, 2008, vol.
6958: International Society for Optics and Photonics, p. 69580E.
in Fig. 6. In this figure, the gray curves are plotted from the
[6] L. Biagiotti and C. Melchiorri, Trajectory planning for automatic
raw data and the colored curves are obtained from the filtered machines and robots. Springer Science & Business Media, 2008.
data with mean moving-average filter. [7] B. Siciliano, L. Sciavicco, L. Villani, and G. Oriolo, Robotics:
The curves in the figure suggest that the robot end modelling, planning and control. Springer Science & Business Media,
2010.
converges to the target pose gradually. In the figure, vibration
[8] D. Han, X. Fang, and Q. Wei, "Rotation interpolation based on the
also occurs, and it mainly results from the influence of joint geometric structure of unit quaternions," in 2008 IEEE International
friction and doesn’t affect the algorithm validity. Conference on Industrial Technology, 2008: IEEE, pp. 1-6.
[9] T. Barrera, A. Hast, and E. Bengtsson, "Incremental spherical linear
VI. CONCLUSION interpolation," in The Annual SIGRAD Conference. Special
In this paper, we proposed an acceleration-level continuous Theme-Environmental Visualization, 2004, no. 013: Citeseer, pp. 7-10.
visual servoing method. In the method, the quaternion is [10] Y. Liu, Z. Xie, Y. Gu, C. Fan, X. Zhao, and H. Liu, "Trajectory
planning of robot manipulators based on unit quaternion," in 2017 IEEE
applied to describe the end attitude for singularity-free and International Conference on Advanced Intelligent Mechatronics (AIM),
simple error calculation. To decouple the quaternion, we map 2017: IEEE, pp. 1249-1254.
the quaternion into angle-axis and interpolate the decoupled [11] [X. Zhao, Z. Xie, H. Yang, and J. Liu, "Minimum Base Disturbance
pose in an acceleration-level continuous way. To achieve the Control of Free-Floating Space Robot during Visual Servoing
robot control, the angle-axis related variables are converted Pre-capturing Process," Robotica, vol. 38, no. 4, pp. 652-668, 2020.
back to quaternion, angular velocity and angular acceleration. [12] R. P. Paul, Robot manipulators: mathematics, programming, and
This method achieves the interpolation in visual servoing control: the computer control of robot manipulators. Richard Paul,
process. Simulations and experiments are carried out to prove 1981.
that the method leads to a convergent visual servoing process [13] G. Yang, Y. Liu, and M. Jin, "A new control method of flexible-joint
manipulator with harmonic drive," Proceedings of the Institution of
and the acceleration-level continuous trajectory gives the robot Mechanical Engineers, Part C: Journal of Mechanical Engineering
a smooth motion. Science, vol. 234, no. 9, pp. 1868-1883, 2020.

ACKNOWLEDGMENT
This project was supported by the Foundation for
Innovative Research Groups of the National Natural Science
Foundation of China (Grant No. 91848202).

104
 2021 the 13th International Conference on Computer and Automation Engineering

Event-Based Data Acquisition for Production Process

Analyses: A Systematic Mapping Study
Rocky Telatko Georg Maurer Dirk Reichelt
Informatics/Mathematics Informatics/Mathematics Informatics/Mathematics
University of Applied Sciences University of Applied Sciences University of Applied Sciences
Dresden, Germany Dresden, Germany Dresden, Germany
[email protected] [email protected]

Abstract—The aspiration on modern manufacturing systems manufacturing industry acquire and analyse machine data.
is raising. One reason is the goal of interconnected systems and a There are many data-related technologies to support such
striving for a high degree of automation. Data should be available processes. These include approaches and technologies such as
everywhere to make production as transparent and efficient as IIoT (Industrial Internet of Things), sensor networks, cyber-
possible. A process can only be controlled if its behavior can be physical systems (CPS) and data analyses tools [4]. Common
measured. Intuition gives a direction, data contain knowledge. goals are the improvement of production planning, fault
The term Industry 4.0 covers the contents of the first sentences, diagnosis, energy management, device and condition
but the recognizable benefits have their risks. The increase in monitoring. In general, production efficiency and product
technical complexity in combination with already existing process
quality are to be increased. Concurrently, production costs are
inconsistencies requires mechanisms which identify the causes for
to be reduced [5]. To improve production planning,
deviations between the production program plan and the actual
values. Due to the fact that manufacturing systems are optimized manufacturers use production and condition data from their
continuously, the analyses of the systems with current available machines. These data can be collected automatically by the
data volumes is used comparatively little. Our research project machine itself, by external sensors or systems, or manually by
claims on this gap. The idea is to analyse operating states of human labor. Key Performance Indicators (KPIs) can be
machines with regard to the period a machine is in a certain state. calculated from those data. These KPIs provide information
This can be realised via automated monitoring. As an example, if about the efficiency of production. Included are examples like
the required processing time fluctuates regularly and does not the calculation of machine production rates, downtime, setup
relate to the schedule, there is room for improvement. Same for times or overall equipment effectiveness (OEE) [6]. Besides
other machine states. In first stage, we focus on the machine the consideration of KPIs, it is possible to break down the
runtime by dividing it into production-relevant parts to detect machine runtime into its operative states such as process time,
variations in times. The detection of variation contains potential setup times, waiting/idle time or downtime. Manufacturers aim
for improvement. In this paper a systematic mapping study is for a high degree of machine utilization. To achieve a high
carried out. The focus is to give an overview of current research throughput, the processing time should make up the largest part
on - especially event-based - data acquisition and analyses of of the machine runtime. For example, the setup time should be
initially discrete production machines. In addition, the purposes within a planned time range. Times should not vary generally
of data acquisition, the analyses of machine runtime data and the to allow reliable production planning [7]. Often the required
way of implementation are considered. By reviewing and
time does not match with the calculated schedule.
extracting information from current publications, the research
questions from this paper will be answered and further research
Unforeseeable events occur at any time. If the process time or
gaps identified. any other time of a production machine fluctuates regularly and
is not related to the schedule, there is room for improvement.
Keywords—data acquisition, event-based, information and This refers to all operating states a machine can be in, or in
decision systems, manufacturing improvement, manufacturing other words, all production-related time portions into which the
systems, process analyses, signal processing machine runtime can be divided. If irregularities in planned
times of certain operating states of a production machine can be
I. INTRODUCTION detected quickly, losses in throughput and efficiency can be
Terms such as Industry 4.0 (I4.0), Smart Manufacturing and avoided. For precise monitoring of the operating status of a
Industrial Internet of Things (IIoT) are present in the machine, data must be collected to control the process.
manufacturing industry for a couple of years now. To remain A change of the operating state can be detected by events.
competitive, it is necessary to switch from traditional On the basis of events, the time periods a machine has been in a
manufacturing to smart manufacturing systems [1]. The certain operating state can be calculated [8]. The main
industry is on the move. Trends propagate the change from application area of this paper is event-based data acquisition
mass to customized production. This requires more flexible due to a perceived lack of use in the analyses of production
systems with variable product types, small batch sizes and a systems in terms of process monitoring. This paper gives an
short throughput time [2]. Therefore, companies have to overview of the current literature dealing with this idea. The
evaluate their manufacturing systems and need fast decision following research questions (RQs) were constructed for this
processes [3]. To realise this change, companies in the paper:

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 105

 RQ1: What purpose is the data acquired from discrete feedback from the start or end of processing, logistical
production machines used for and how is the data acquired? operations, setup procedures or expected downtime.
Unexpected events can be failures, lack of equipment or delays
RQ2: If data is acquired specifically on machine runtime -
in production [8].
What is the data used for?
A widely used sensor technology is Radio Frequency
RQ3: Are there methods to capture machine runtime from
Identification (RFID). RFID is a communication technology
discrete production machines using an event-based approach?
that uses radio signals to identify and read tags. Therefore, no
The overview provided by this paper should help to drive mechanical or optical contact is required. This technology can
forward other related projects. The structure is as follows. be used to acquire data in the form of events for machine
Section 2 provides background information on related monitoring or process control. It is mainly used to monitor the
technologies. Section 3 explains in detail the search method production process of individual work pieces [11].
and the search string. This includes the inclusion and exclusion
criteria used to select the papers. Section 4 contains the Depending on the production machine and the degree of
analyses that answers the research questions. Section 5 gives a automation of production, the machine itself can provide or
summary of the results and possible projects that can send data. A common term for a data acquisition system is
subsequently be realised on the basis of the results. "Supervisory Control and Data Acquisition" (SCADA).
SCADA is a control system consisting of software and
II. BACKGROUND hardware elements embedded in the manufacturing
This chapter aims to provide background information on infrastructure. The system controls industrial processes and the
related technologies relevant to this paper. First, the term acquisition and monitoring of machine data. The SCADA
Industry 4.0 and discrete manufacturing in general are system therefore interacts directly with devices such as
discussed. This is followed by information on events and data machine components and sensors. SCADA systems also
acquisition technologies. interact with programmable logic controllers (PLCs) to obtain
and process machine data. PLCs are microcomputers that are
Many papers contain the future challenges of modern suitable for controlling production machines. They are usually
manufacturing. The concepts of Industry 4.0 describe a embedded in the machines [12].
production environment based on interconnected machines and
systems. Not only the production area, but all levels of the III. RESEARCH METHOD
company and subsidiaries as well as suppliers and customers This section introduces the research method. Petersen et al.,
are networked with each other. This forms a network that 2008 presented a paper on conducting mapping studies in
produces and exchanges data that can be transformed into
software engineering [13]. The procedure for this mapping
information [9]. In today's industry, manufacturers have to deal
study is based on this method. The goal is to give an overview
with different production methods depending on their
of a research field. The scope, types and results in this field are
production program. They have to deal with the digital
to be identified. By classifying and counting publication
transformation of their production facilities to gain a
competitive advantage, too. There are two main types of frequencies, it is possible to identify temporal trends and
production, depending on the products and the way they are relevant authors in the research area. In a scientific
manufactured. One is the form of discrete manufacturing, environment where there is a lack of relevant, high-quality
which characterizes a type of production where fixed quantities studies, a mapping study can provide a good summary [13].
are produced, such as cars, smartphones, etc. In contrast, there The procedure follows clearly defined steps, as shown in Fig. 1.
is the type of production in the process industry, which is
concerned with mixing ingredients according to certain
formulas, such as food and drink, cosmetics or chemicals [10].
As each type of production is different and has its own
challenges, this paper will initially address discrete
manufacturing.
Data acquisition at machinery level in a production Fig. 1. The systematic mapping process [13]
environment is the basis for gaining information for
improvement approaches. There are many technologies for this A. Definition of Search String
purpose which were created and developed further with the After defining the scope of research and the research
beginning of the realisation of Industry 4.0 concepts. Sensors in questions, general considerations were made about the search
particular have become very popular. For example, they can be method. A predefined search string can be used to identify
used to detect changes at machinery level. Those data is sent primary studies in scientific databases to obtain a good result
via a network to an upper manufacturing level and processed set of publications. The search string must be derived from the
by software (e.g. a Manufacturing Execution System) which research questions. In addition, keywords were defined in a
facilitates the control and monitoring of complex systems and brainstorming process and an initial screening of potentially
the associated infrastructure [2]. A change on a machine can be relevant work. The final string can be divided into two parts,
detected by an event. An event can be categorized into which are listed in Table I.
expected and unexpected events. Expected events can be

106
 TABLE I. SEARCH STRING Exclusion criteria: If a publication meets an exclusion
Search area Keywords criterion, it will be directly excluded without further
Title, "discrete manufacturing" OR "shop-floor" OR verification:
Abstract, "production system" OR factory OR "manufacturing
Keywords system"  The publication is not written in English.
Full text (“operation time" OR "process time" OR "run time" OR
"running time" OR "machine status" OR "change of  The publication is not available in full text.
state") AND ("Machine data" OR "data acquisition" OR
"sensor data" OR "event data") AND (monitor* OR  Publications that are duplicates of other publications.
log* OR trac* OR track*)
 The publications were produced before 2005.
To achieve a high degree of accuracy, one of the first  The content of the publication was not peer-reviewed.
keywords (Table I, first line) had to appear either in the title, in
the summary or in the related keywords. All other keywords D. Data Extraction and Synthesis
(Table I, second line) had to appear somewhere in the full text. After the refining process, the total number of papers was
Keywords marked with an asterisk can occur in different word reduced from 623 to 28 publications. These 28 publications
forms. were read in full text with the aim of extracting information to
answer the research questions. The publications are then
B. Used Databases for Search
assigned to a classification scheme. The number of publications
Only online databases were used for the research. The in each category can then be extracted. After evaluation of the
following databases were considered: ACM, IEEE, Science results, research gaps for future work are identified. The results
Direct, Scopus. Table II lists the number of search results found are presented in the next chapter.
using the defined search string. For a better result, the search
string may vary slightly depending on the database. Each IV. ANALYSES
database has its own syntax for how a search string should be First, the general frequency by year of publication and
written. publisher is considered. According to the inclusion criteria,
publications will be included in the paper set from 2005
TABLE II. SCIENTIFIC DATABASE AND SEARCH RESULTS
onwards. Fig. 2 shows that between 2007 and 2013 the research
Scientific database Search results area under consideration was apparently not of great interest.
ACM 24 Since 2014, the number of publications has risen slightly. This
IEEE 330 may be related to the fact that around the year 2013 the term
ScienceDirect 245 "Industry 4.0", "Smart Factory" and topics on digitization
Scopus 24 emerged in industry [14]. After 2016 there is a strong increase
Total 623
in relevant contributions. The decline in 2020 is due to the fact
that the study was conducted in mid-2020. If the study were to
There are further important scientific databases that could
be repeated in 2021, it can be expected that the number of
have been taken into account. In order to realise this mapping
relevant papers in 2020 would be at least at the same level as in
study within a given time frame, the four listed research area-
2019.
related databases were selected. Within these databases 623
publications were found. Section 4 contains a detailed analyses.
C. Selection Criteria
Because of the huge amount of potential paper, a reduction
strategy was implemented by defining inclusion and exclusion
criteria. The objective was to reduce the number of publications
to the most relevant ones that could be used to answer the
research questions.
Inclusion criteria: If all features of a publication meet the
inclusion criteria and do not meet any exclusion criteria, it is
assigned to the final relevant paper set. The following inclusion
criteria have been defined:
 The keywords must be included in a logical
combination in the title or abstract.
Fig. 2. Frequency of publications per year
 The paper must refer to discrete production.
 The paper had to be published in 2005 or later to Fig. 3 shows the frequency of publications per publisher.
include papers with theories and concepts that were Less papers were selected from ACM and Scopus than from
developed before the introduction of the term Industry ScienceDirect and IEEE. One reason for this is the significantly
4.0 and Smart Factory, but should still be relevant. lower total number of papers found in ACM and Scopus before
 The content of the publication was peer-reviewed (e.g. the refining process. However, more papers were selected from
articles in journals, conferences and workshops). the ScienceDirect and IEEE databases than from ACM and

107
Scopus. Before and after the refining process, the ratio is includes publications describing the use of data for process
highest at Scopus. optimization. Examples are increasing flexibility, setting
realistic production targets or reducing time-to-market. The
category "Material tracking" includes publications describing
the use of data for material tracking to ensure transparency in
terms of traceability.
Categories 2 & 3 are the most frequently mentioned
purposes for data acquisition according to Table III. More than
28 publications are listed, as some publications mention several
purposes. Due to the size of the paper set it is difficult to derive
a general ratio from this frequency. Furthermore, the research
questions in the paper tend to focus on an event-based approach
to data acquisition which affects the size.
The second part of the first RQ deals with how data is
acquired from discrete production machines. Therefore, the
following three categories have been identified:
Fig. 3. Frequency of publications per database 1. Automatic data acquisition via a machine control unit

RQ1: What purpose is the data acquired from discrete 2. Automatic data acquisition via a retrofitting approach
production machines used for and how is the data acquired? 3. Manual data acquisition
The aim is to collect production and machine data to gain The classification differentiates between machines ready-
information. Data analyses can provide information about for-connection and machines not-ready-for-connection. Ready-
inconsistencies in the process and indicate room for for-connection (category 1) are discrete production machines
improvement. To answer the question in detail, all selected that are equipped with sensors on their own. This enables the
publications are examined. The search was conducted for machine itself to collect and provide data. Machines that are
categories in which data acquisition methods can be not-ready-for-connection (category 2) are usually older
summarized. The categories were not predefined but were machines that do not acquire data themselves or are not able to
created during the analyses process for this paper. The analyses supply them [28], [34]. Sensor technology is used to acquire
is divided into two parts. data from these machines when retrofitting the machines.
First, what purposes is the data acquired for. Many different Category 3 includes publications that include manual data
purposes have been found for data acquisition from discrete acquisition. Manual data acquisition can be paper-based or
production machines. The following four main reasons have performed by an employee via a human-machine interface
been identified: (HMI).

1. Quality control e.g. of manufactured items TABLE IV. HOW DATA IS ACQUIRED

2. Condition monitoring / Predictive maintenance How data is Frequency Publications

acquired
3. Optimization of production planning and processes Machine 21 [4], [5], [8], [9], [11], [12], [15], [16],
control [18], [19], [20], [21], [23], [24], [25],
4. Material tracking [27], [28], [29], [31], [33], [35]
Retrofitting 6 [11], [15], [17], [27], [28], [34]
TABLE III. PURPOSES FOR DATA ACQUISITIONS Manual 3 [11], [15], [21]
Purposes Frequency Publications
Quality Control 6 [4], [12], [15], [16], [17], [18] The distribution in Table IV shows that publications deal
Condition monitoring 17 [4], [9], [11], [12], [15], [16], most often with categories 1 & 2. This is understandable as the
/ Predictive [17], [18], [19], [20], [21], [22], RQs also refer to event-based data acquisition. With manual
maintenance [23], [24], [25], [26], [27] data acquisition this would only be possible to a very limited
Optimization of 16 [4], [8], [11], [12], [18], [20], extent. Many different sensor types are mentioned. For
production planning [23], [24], [25], [27], [28], [29], example acceleration, temperature, optical or position sensors.
and processes [30], [31], [32], [33]
The most commonly mentioned sensor technology is RFID. It
Material tracking 8 [11], [15], [16], [23], [24], [27],
[32], [33]
was also pointed out that data are often collected by a PLC and
transmitted via the OPC UA and MTConnect protocols. Both
technologies are standards for communication protocols for
Table III shows the purposes and the publications in which machines and systems in an industrial environment.
they appear. The categories include further subcategories. The
category "Condition monitoring / Predictive maintenance" RQ2: If data is acquired specifically on machine runtime -
refers to the recording of the general condition of machines and What is the data used for?
the detection and prediction of failures and delays. The Machine data are acquired for a variety of purposes. RQ1
category "Optimization of production planning and processes" mentions several of them. RQ2 specifically refers to machine

108
runtime when acquiring data. The reason for this is to improve System) or SCADA are not available. If higher-level systems
machine performance by reducing inconsistencies in processes are available, data can be extracted from those systems.
via machine runtime. This can be done by analysing machine
states which can be mapped via events to subdivide the Unfortunately, this issue is much less addressed in the
machine runtime into the appropriate parts. Similar to the papers. That is why the final paper set after the refining process
categories of the first part of RQ1, this paper also refers to is quite small compared to the total paper set. The results found
categories 2 & 3 - "Condition monitoring" and "Optimization can be summarized as follows.
of production planning and processes". The machine itself can calculate and transmit the time
Operating states into which the machine runtime can be intervals for its various operating states. This possibility was
divided and which are frequently highlighted in publications already discussed in RQ1. As a further method, backwards
are: processing time, idle time, setup time and downtime. The calculation from a MES or a PPS was specified. An MES
cycle time is also described. Table V shows the frequency of tracks the production processes and stores relevant data of the
the states or times referred. The machine states or time portions individual processing steps for the manufactured work pieces
can be broken down in more detail depending on the interests [30]. A PPS is used to plan and control the production
of the company. The greatest interest is the recording of the environment. Based on the data in the MES / PPS, the periods
processing time. The detection of idle time and setup times are of machine runtime can be calculated backwards. Of course,
also often mentioned. Idle time describes a state in which a this is only possible if the required data are recorded in suitable
production machine is ready but not working. Waiting time can quality. It is also possible that specially developed software is
be used synonymously. Setup time describes the period in used to calculate the time periods. However, this is not be
which a machine gets prepared for processing. Downtime discussed in the papers. A manual method is not recommended
means that a machine is not able to process due to a planned or due to the high input of resources.
unplanned breakdown. The cycle time describes the period of V. CONCLUSION AND FUTURE WORK
one cycle of a machine operation, including several mentioned
times. With the help of those data, KPIs such as "Overall A systematic mapping study was carried out in this paper.
Equipment Effectiveness" (OEE) can be calculated. Three of First, three research questions were created. Then search
the selected papers refer to OEE [4], [5], [29]. OEE is a parameters were defined to find a suitable set of publications to
measure of how effectively a manufacturing system is used answer the RQs. Originally 623 publications were found. These
compared to its full potential. It considers the availability, were reduced to 28 relevant publications in a refinement
quality and performance of a machine. The standardized form process. Subsequently, the information from the publications
enables a comparison between manufacturing units in different was extracted to answer the RQs. It was observed that many
departments or organizations [36]. To calculate OEE it is publications deal with the topics "Condition Monitoring" and
necessary to acquire and record machine data. The KPI "Total "Predictive Maintenance" as well as with optimization issues.
Effective Equipment Performance" (TEEP) was referred twice The data are usually acquired automatically by sensors on the
[4], [29]. The TEEP is closely related to OEE. It measures OEE machine itself or by retrofits. Several operative machine states
by calendar hours instead of planned operating hours [4]. Other were detected into which the machine runtime can be
reasons for collecting machine runtime data in addition to subdivided. Reasons for recording operating states are the
calculating KPIs are monitoring machine conditions or calculation of KPIs, monitoring purposes or the identification
identifying losses in manufacturing. of losses in manufacturing. Unfortunately, there was little
knowledge of how the calculation of the duration of a particular
TABLE V. PARTS OF THROUGHPUT TIME operating state is done. Especially on an event-based approach.
Parts of Frequency Publications A production machine is either able to record its states itself
throughput time or by individual backward calculation depending on the
Processing time 8 [4], [5], [8], [24], [25], [28], [30], available machine data and system functionality. Many papers
[33] were comparatively short or dealt with only a small part of the
Idle time 5 [4], [24], [25], [28], [29] questions asked while the number of publications submitted has
Setup time 5 [5], [8], [19], [24], [25]
increased in recent years. However, there is still a lot of
Downtime 4 [4], [5], [28], [29]
potential for discussion and further research, especially in
Cycle time 3 [5], [26], [29]
identifying inconsistencies in processes to improve systems
performance and create a robust manufacturing system. Our
RQ3: Are there methods to capture machine runtime from paper could motivate further work in this area and a stronger
discrete production machines using an event-based approach? focus on event processing in production environments. This
The last RQ deepens the focus of RQ1 and RQ2. After could be useful for manufacturing environments where higher
answering the question of why manufacturers acquire and level systems such as MES, PPS or SCADA are not available.
record machine data with a specific focus on machine runtime, If higher-level systems are available the data can be used from
RQ3 focuses on methods to achieve this on an event-based those systems. The analyses of variability in machine runtime -
approach. The focus on events is related to the idea of subdivided into production-relevant parts - can provide
subdivide the machine runtime based on those events. An indications of inconsistencies in processes. Fewer process
event-based approach can be useful for manufacturing deviations can improve systems performance and lead to more
environments where higher-level systems such as MES robust production environments. This is the focus of our
(Manufacturing Execution System), PPS (Production Planning research project, starting at the machinery level.

109
 REFERENCES [19] C. Zhang, G. S. Hong, H. Xu et al., “A data-driven prognostics
framework for tool remaining useful life estimation in tool condition
[1] K.-D. Thoben, S. Wiesner, and T. Wuest, ““Industrie 4.0” and Smart monitoring,” in 2017 22nd IEEE International Conference on Emerging
Manufacturing – A Review of Research Issues and Application Technologies and Factory Automation (ETFA), pp. 1–8, 2017.
Examples,” International Journal of Automation Technology, vol. 11, no.
1, pp. 4–16, 2017. [20] E. M. Leao, L. A. Guedes, and F. Vasques, “An Event-Triggered Smart
Sensor Network Architecture,” in 2007 5th IEEE International
[2] Tran, Park, Nguyen et al., “Development of a Smart Cyber-Physical Conference on Industrial Informatics, vol. 1, pp. 523–528, 2007.
Manufacturing System in the Industry 4.0 Context,” Applied Sciences,
vol. 9, no. 16, p. 3325, 2019. [21] B. Babayigit and H. Sattuf, “An IIoT and Web-based Low-Cost SCADA
System for Industrial Automation,” in 2019 11th International
[3] J.-k. Sun, Y.-q. Xu, and F.-p. Yin, “Study on Performance Evaluation Conference on Electrical and Electronics Engineering (ELECO),
and Improvement for the Discrete Manufacturing Enterprises Under the pp. 890–894, 2019.
Integration Framework,” in Proceedings of the 22nd International
[22] R. Kim, J.-H. Han, and S. Chi, “Data Acquisition for Control Level
Conference on Industrial Engineering and Engineering Management
Automation for SMEs: Requirements and Architecture,” in Proceedings
2015, E. Qi, J. Shen, and R. Dou, Eds., pp. 409–416, Atlantis Press,
Paris, 2016. of the 2015 International Conference on Big Data Applications and
Services, pp. 180–183, Association for Computing Machinery, New
[4] Juergen Lenz, Thorsten Wuest, and Engelbert Westkämper, “Holistic York, NY, USA, 2015.
approach to machine tool data analytics,” Journal of Manufacturing
Systems, vol. 48, pp. 180–191, 2018. [23] H. ElMoaqet, I. Ismael, F. Patzolt et al., “Design and Integration of an
IoT Device for Training Purposes of Industry 4.0,” in Proceedings of the
[5] Y. Guo, Y. Sun, and K. Wu, “Research and development of monitoring 2nd International Symposium on Computer Science and Intelligent
system and data monitoring system and data acquisition of CNC Control, Association for Computing Machinery, New York, NY, USA,
machine tool in intelligent manufacturing,” International Journal of 2018.
Advanced Robotic Systems, vol. 17, no. 2, 2020.
[24] Philipp Schäfers, Alexander Mütze, and Peter Nyhuis, “Integrated
[6] H. Rødseth, J. O. Strandhagen, and P. Schjølberg, “Key Performance Concept for Acquisition and Utilization of Production Feedback Data to
Indicators for Integrating Maintenance Management and Manufacturing Support Production Planning and Control in the Age of Digitalization,”
Planning and Control,” in Advances in Production Management Systems: Procedia Manufacturing, vol. 31, pp. 225–231, 2019.
Innovative Production Management Towards Sustainable Growth, S.
Umeda, M. Nakano, H. Mizuyama et al., Eds., vol. 459, pp. 70–77, [25] D. Mourtzis, M. Doukas, A. Vlachou et al., “Machine Availability
Springer International Publishing, Cham, 2015. Monitoring for Adaptive Holistic Scheduling: A Conceptual Framework
for Mass Customization,” Procedia CIRP, vol. 25, pp. 406–413, 2014.
[7] D. T. Matt, V. Modrák, and H. Zsifkovits, Industry 4.0 for SMEs,
Springer International Publishing, Cham, 2020. [26] S. K. a. Subramaniam, S. H. b. Husin, Y. b. Yusop et al., “Real time
shop floor monitoring system for a better production line management,”
[8] Christoph Berger, Alexander Zipfel, Stefan Braunreuther et al., in 2007 Asia-Pacific Conference on Applied Electromagnetics, pp. 1–4,
“Approach for an event-driven production control for cyber-physical 2007.
production systems,” Procedia CIRP, vol. 79, pp. 349–354, 2019.
[27] M. Azangoo, J. O. Blech, and U. D. Atmojo, “Towards Formal
[9] R. S. Peres, A. D. Rocha, A. Coelho et al., “A highly flexible, distributed Monitoring of Workpieces in Agile Manufacturing,” in 2020 IEEE
data analysis framework for industry 4.0 manufacturing systems,” International Conference on Industrial Technology (ICIT), pp. 334–339,
Studies in Computational Intelligence, vol. 694, pp. 373–381, 2017. 2020.
[10] H. Cai, L. Jiang, and K.-M. Chao, “Current and future of software [28] C. Wei, Y. Lee, K. Cao et al., “Implementation of a data acquisition
services in smart manufacturing,” Service Oriented Computing and system for heterogeneous machines,” in 2017 IEEE/SICE International
Applications, vol. 14, no. 2, pp. 75–77, 2020. Symposium on System Integration (SII), pp. 232–235, 2017.
[11] P. Fang, Y. Jiang, and R. Y. Zhong, “Real-time monitoring of workshop [29] Ben Edrington, Bingyan Zhao, Adam Hansel et al., “Machine
status based on internet of things,” Proceedings of International Monitoring System Based on MTConnect Technology,” Procedia CIRP,
Conference on Computers and Industrial Engineering, CIE, 2018- vol. 22, pp. 92–97, 2014.
December, 2018.
[30] M. I. Ali, P. Patel, and J. G. Breslin, “Middleware for Real-Time Event
[12] A.F. Cutting-Decelle, J.L. Barraud, B. Veenendaal et al., “Production Detection andPredictive Analytics in Smart Manufacturing,” in 2019
information interoperability over the Internet: A standardised data 15th International Conference on Distributed Computing in Sensor
acquisition tool developed for industrial enterprises,” Computers in Systems (DCOSS), pp. 370–376, 2019.
Industry, vol. 63, no. 8, pp. 824–834, 2012.
[31] N. Gjeldum, M. Mladineo, M. Crnjac et al., “Performance analysis of the
[13] Kai Petersen, Robert Feldt, Shahid Mujtaba et al., “Systematic Mapping RFID system for optimal design of the intelligent assembly line in the
Studies in Software Engineering,” vol. 2008. learning factory,” Procedia Manufacturing, vol. 23, 2018.
[14] E. Oztemel and S. Gursev, “Literature review of Industry 4.0 and related [32] G. Jiao, W. Xu, J. Li et al., “Research and Application of Intelligent
technologies,” Journal of Intelligent Manufacturing, vol. 31, no. 1, Manufacturing System for Precision Assembly Enterprise,” in 2019
pp. 127–182, 2020. International Conference on Artificial Intelligence and Advanced
[15] Y. Liu, H. Zhang, and R. Y. Zhong, “A Survey on Internet of Things- Manufacturing (AIAM), pp. 750–755, 2019.
enabled Real-time Machine Management System in New Zealand,” [33] Thomas H.-J. Uhlemann, Christoph Schock, Christian Lehmann et al.,
Procedia CIRP, vol. 72, 2018. “The Digital Twin: Demonstrating the Potential of Real Time Data
[16] C. Wang and P. Jiang, “Cognitive computing based manufacturing data Acquisition in Production Systems,” Procedia Manufacturing, vol. 9,
processing for internet of things in job-shop floor,” in 2015 IEEE pp. 113–120, 2017.
International Conference on Mechatronics and Automation (ICMA), [34] Masaaki Maeda, Yuichi Sakurai, Tsuyoshi Tamaki et al., “Method for
pp. 2521–2526, 2015. Automatically Recognizing Various Operation Statuses of Legacy
[17] I. -. Sun and K. Chen, “Development of signal transmission and Machines,” Procedia CIRP, vol. 63, pp. 418–423, 2017.
reduction modules for status monitoring and prediction of machine tools,” [35] G. Gonçalves, J. Reis, R. Pinto et al., “A step forward on intelligent
in 2017 56th Annual Conference of the Society of Instrument and factories: A Smart Sensor-oriented approach,” in Proceedings of the
Control Engineers of Japan (SICE), pp. 711–716, 2017. 2014 IEEE Emerging Technology and Factory Automation (ETFA),
[18] A. Corallo, A. Crespino, C. Dibiccari et al., “Processing Big Data in pp. 1–8, 2014.
Streaming for Fault Prediction: An Industrial Application,” in 2018 14th [36] M. Saez, F. P. Maturana, K. Barton et al., “Real-Time Manufacturing
International Conference on Signal-Image Technology Internet-Based Machine and System Performance Monitoring Using Internet of Things,”
Systems (SITIS), pp. 730–736, 2018. IEEE Transactions on Automation Science and Engineering, vol. 15, no.
4, pp. 1735–1748, 2018.

110
 2021 the 13th International Conference on Computer and Automation Engineering

A Modified HLFET Algorithm for Parallelization of

Manufacturing Tasks with Causal Dependencies
Dirk Frosch-Wilke Vinitha Thanasekaran
University of Applied Sciences Eppendorf Instrumente GmbH
Kiel, Germany Hamburg, Germany
[email protected] [email protected]
m

Abstract—Some typical scheduling problem in manufacturing assigned to heterogenous resources given by the set k=1,2,...,K.
processes can be described as follows: An infinite number of We assume that tasks are non-preemptable; i.e. these tasks
manufacturing tasks which are using one or more devices with cannot be interrupted once they are assigned to a resource and
different features, should be scheduled with a maximum degree have to be completed before the execution of another task
of parallelism. Thereby the tasks have causal dependencies. In using the same resource. Additionally, there may be
this paper we present a modified and extended HLFET precedence constraints between the tasks due to the
algorithm which will be able to automatically detect parallelism manufacturing logic that has to be respected. The objective of
earlier and which is able to increase manufacturing performance such a scheduling problem is to find parallel tasks, precedence
by reducing the makespan further. We demonstrate our
and resource feasible completion times for all tasks such that
algorithm in the context of a manufacturing device, with a
workstation having different resources and device features. This
the execution time of the program is minimized.
device is designed to monitor the contamination-free, precise and This paper is organized as follows: next Section presents
correct measurement and transfer of liquids and to automatically some related works; Section 3 describes our case study, the
control the combining of liquids. implementation of the (unmodified) HLFET algorithm,
followed by our modification of the HLFET algorithm and we
Keywords—scheduling problem, HLFET algorithm, causal task compare the results of both algorithms for different scenarios.
dependencies
A conclusion closes the paper.
I. INTRODUCTION
II. RELATED WORK
Parallel processing of tasks in a manufacturing system is
of interest in order to exploit the maximum utilization of the A. Directed Acyclic Graph Model (DAG)
available resources and to reduce the makespan of A Directed Acyclic Graph (DAG) is a common abstract
manufacturing processes. The objective of identifying potential model for representing a program to be scheduled [4].
parallelism in production scheduling programs is very effective Identifying possible parallelism in an application requires
in exploiting resources to their maximum capability and also extensive knowledge about the procedural structure in the
decreasing the total execution time of the production program. application. The procedural structure can be broken down into
Hence by developing a compiler that can analyzses the set of (atomic) tasks and different kind of dependencies for each task.
commands, required machine resources for the commands and
In our model a DAG is represented by G = (V,E, w, c)
required device features, it is possible to detect potential
where V is the set of v nodes and E is the set of e directed
parallelism of independent commands before program
edges. A node ni of the graph represents a task Ti and a directed
execution.
edge (ni, nj) from node ni to nj indicates that task Ti must
Although the problem of optimal scheduling is not a new complete execution before task Tj begins. w: V R+ and c:
task [1], to find an optimal schedule is complex (NP-complete) E R+ are weight functions. The weight w(ni) is called the
[2], and may only be reasonably faced, with the exclusion of a computation cost of node ni. In our case w(ni) is equal the
few cases, by means of heuristic approaches. execution time of task Ti. c(ni,nj) denotes the inter-task
communication time between Ti and Tj (the time needed for
In this paper we present an approach which aims to identify data exchange between the resources that execute Ti and Tj)
parallel tasks from the command list of some manufacturing (see figure 2 (b) for a simple example of a DAG).
program and calculating the execution time a priori. A
manufacturing program is a sequence of specific tasks, each of The NP-completeness of DAG scheduling algorithm have
which may require a specific set of resources. These tasks have led to the development of many heuristic algorithms. Most of
to be executed such that precedence and the resource these heuristics are based on the list scheduling technique. In
constraints are maintained. this case each task has an assigned priority, and scheduling is
done according to a list priority policy: select the node with the
Thus, the problem we are dealing with in this paper is a highest priority in the scheduling list and assign it for
resource-constrained process scheduling problem [3]. Tasks of scheduling [5]. Further one can distinguished between static
the program are identified by a set t=1,2,...,T. Tasks will be and dynamic calculation of the priorities of unscheduled tasks.

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 111

In the first case the scheduling list is constant throughout the list scheduling algorithm [5]. Although the HLFET algorithm
scheduling process, whereas in the dynamic case the is quite simple the quality of this algorithm is comparable with
scheduling list will be updated after each node scheduling. In more complex DAG scheduling algorithms [8]. Further the
the following we are interested in static-list-scheduling, completion time of the algorithm is lesser compared to other
because in our case study the scheduling of tasks of the device algorithms of the same category [5].
execution program as well as the execution time should be
calculated before the process starts. The following are the main steps of the algorithm [5]:

There exist different priority attributes for nodes in a DAG 1. Compute the static level of each node in the DAG.
model. We will introduce the static-b-level attribute: 2. Create a ready-list in a descending order of static level.
Initially, the ready-list contains only the entry nodes
(nodes with an in-degree equals zero).
3. Repeat until all nodes are scheduled:
(a) Schedule the first node in the ready-list to a
where succ(ni) is the set of all nodes nj, such that (ni, nj) ϵ V. processor that allows the earliest execution, using the
The static-b-level of a node ni is the length of the longest non-insertion approach.
path from node ni to the exit node including the weight of ni. (b) Update the ready-list by inserting the nodes that are
In the case that the inter-task communication time for all now ready.
edges in G are equal to zero, the static-b-level will be called The HLFET algorithm has a polynomial time complexity of
static-level and can be computed as: O(v2) [5].
III. THE CASE STUDY
A. The Workstation with Resources and Device Features
The device in consideration has a workstation with a set of
In our case we can assume zero inter-task communication resources and built-in device features. The device features
times and therefore we are working with the static-level include tool holders, thermomixers, dispensing tools, grippers
attribute for assigning priorities to tasks. etc. The resources include different labwares (e.g. liquid filters,
PCR 1 -grade tips). The application program that runs the
The static-level attributes of nodes in G are bounded by the device consists of a set of commands/tasks. Each of these
length of the critical path of G. The critical path is the weight commands use one or many device features and labwares for
of the longest path in G. Algorithms based on the critical path execution. Hence there are two kind of operating resources: (1)
heuristic produce in average the best results for the DAG device features and (2) labware (slots).
scheduling problem [6].
The control commands are currently executed one after
In parallel computing, the degree of concurrency is the another in a sequential manner although there are times when
number of tasks that can be executed in parallel [7]. An certain input commands can be executed in parallel. The
important measure of the degree of concurrency is the average commands that can be in parallel are commands that are
degree of concurrency which can be calculated by using a independent of each other with respect to the resources used.
DAG as follows: When identifying parallel and sequential commands, care has
to be taken not to change the order in which the commands
were given. So, for a given set of input commands, the problem
, lies in determining which commands can be executed in
parallel without violating their precedence constraints.
where Total amount of work is the sum of all communication
Figure 1 shows an example of some device`s worktable
and execution cost of the DAG nodes and edges.
with six labware slots where one is dedicated for a
Scheduling algorithms use two different insertion policies Thermomixer (TMX). Labware can be placed on all of these
[9]: (1) the insertion-based policy looks for the insertion of a locations that are named A1, A2, B1, B2, C1, C2. There are
task in the earliest idle time slot, between two scheduled tasks many types of labware available such as PCR Plate with 96
on a resource, and (2) the non-insertion based policy considers well, Tubes and Deepwell plates to name a few. The
the possible insertion of a task only after the last scheduled task dispensing tools T1, T2 and the gripper G1 are already defined
on a resource. in the device. The user has no control to add or remove them.
Hence the resources such as number of tool holders,
B. HLFET Algorithm (Highest Level First with Estimated thermomixer modules, dispensing tools, grippers and waste are
Times) pre-defined for each device. The user has control to only add or
After considering different DAG algorithms we choose the remove labware in the six slots.
HLFET algorithm. The HLFET algorithm is considered to be
one of the simplest algorithms of its kind and it is known as a 1
Polymerase Chain Reaction (PCR) is a well-established and popular
technique to amplify and detect specific RNA and DNA sequences

112
 obtain scheduling programs with a higher performance as the
HLFET algorithm.

(a) (b)
Fig. 2. List of tasks with dependencies and executions times and the final
DAG.

C. Our Approach
Fig. 1. Design of the device worktable
The HLFET algorithm is a class of list-scheduling
algorithm which was developed for homogeneous processors.
The application workflow of the device consists of two
For manufacturing scheduling problems, the processors
main steps: (1) Setting up the device’s worktable with required
correspond with manufacturing resources. Analogically, the
labwares and (2) Defining the set of commands to be executed.
application of the HFET algorithm to our case study assumes
There are different types of commands (=tasks) that the device
that all the device features of the liquid handling system are of
can perform such as Sample Transfer, Reagent Transfer,
the same type. But we do have different types of device
Thermomixer etc.
features as described.
B. Task Dependencies and Implementation of the HLFET
Underlying heuristic of our modification: The HLFET
Algorithm
algorithm does not take into consideration the various device
While performing tasks in parallel, the main obstruction to features available. If we create a scheduling-list, such that the
parallelism is task dependencies. We can distinguish two types idle time of each device feature is reduced, then we can detect
of task dependencies which form the precedence constraints in parallelism much earlier in the program. By giving additional
our case study: priority to nodes based on their device feature usage, it is
(1) Labware location dependencies, and (2) device features possible to detect parallelism more effectively.
dependencies. Instead of one ready-list, we have y number of ready-lists in
When assigning tasks to a device feature, it is important to our modified algorithm, where y is the number of the device
consider the labware location conflicts as well. The tasks features of the production system. The algorithm is similar to
should be scheduled by respecting the labware location the HLFET, except that the nodes are scheduled to their
conflicting commands. Hence in our implementation, the respective device feature as early as possible.
commands are first analysed based on their labware location For reasons of simplicity we will assume that we have two
conflicting commands. If there are no labware location device features (y=2) in the system: n Tool holders (TH) and m
conflicts, the commands are scheduled to the earliest available Thermomixers (TMX). It will be easy to extend our algorithm
device feature otherwise they are scheduled after their labware to more than two device features by increasing the number of
location conflicting commands. In this way, the labware ready lists.
location conflicts are also respected.
So, the Modified HLFET algorithm has two ready-lists for
Figure 2 shows an example for a list of tasks and the final TH (device feature X in listing 1) and TMX (device feature Y)
DAG when using the HLFET algorithm for the device having respectively. The nodes in the initial DAG are analysed to
two device features – a dispensing tool at T1 and a identify which nodes use the TH feature and which nodes use
Thermomixer. T0 is the root node with an execution time the TMX feature. When there are nodes in initial DAG that use
equals zero. The static level of each task is added beside the both types of device feature, the Modified HLFET algorithm
node. The average degree of concurrency is 1.125. attempts to efficiently schedule the nodes to both the device
Thus, the two kinds of dependencies are based on the features such that no device feature is idle for a long time.
limited heterogenous resources of the system. In the following Listing 1 shows the Modified HLFET algorithm in pseudo-
we use this observation to modify the HLFET algorithm to code.

113
 Table II shows as an example the commands of application
program AP4.

TABLE II. EXAMPLE OF AN ANALYZED APPLICATION PROGRAM - AP4

For each application program we derived the final DAGs

for the scheduling of commands. Figure 3 shows the final
DAGs for AP 4 of the (a) HLFET and (b) Modified HLFET.
The edge weights for an incoming edge to a node are the
execution time of the node/command

Listing 1. Modified HLFET algorithm

D. Evaluation
Below we present the evaluation results of the modified
HLFET algorithm. We analyszes four different application
programs AP1,…,AP4 for device workstations with different (a) (b)
numbers of tool holders, thermomixers and labware slots. We Fig. 3. Final DAGs for AP 4 for (a) HLFET, (b) Modified HLFET algorithm
calculated the average degree of concurrency for each
application program scheduling as well as the maximum In the following Table III we summarize our evaluation
execution time. We compare these results of the Modified results.
HLFET algorithm with the results achieved by using the
HLFET algorithm. TABLE III. EVALUATION RESULTS
Table I shows the configuration of the device’s
workstations for each of the analyzed application programs.

TABLE I. HARDWARE CONFIGURATION

114
 In AP 1, AP 2 and AP 4 the results of our modified HLFET algorithms help in detecting parallelizable commands, it is
algorithm are far better than the results of the HLFET advisable to use Modified HLFET algorithm for preparation of
algorithm. The modified algorithm in these cases was able to scheduling-lists manufacturing processes with heterogeneous
increase the parallelization of the tasks and thereby achieved resources.
lower execution times. Only in AP 3 the results are the same.
In the latter case both algorithms have achieved the maximum In future work we will formally prove that the Modified
degree of concurrency possible in the application program. HLFET will achieve at least the same average degree of
concurrency as the HLFET algorithm and we will demonstrate
IV. CONCLUSION AND FUTURE WORK the better performance of the Modified HLFET by more use
cases.
Efficient scheduling of manufacturing tasks is an important
issue. The parallelization of tasks can decrease the execution REFERENCES
time of production processes. Often these tasks have causal [1] J. A. Castro-Godinez, Sequential code parallelization for multi-
dependencies, e.g. use of same production resources. coreembedded systems: A survey of models, algorithms and tools. 2014
In this paper we introduced an algorithm for detecting [2] S. Salleh., A.Y. Zomaya Task Scheduling. In: Scheduling in Parallel
Computing Systems. The Springer International Series in Engineering
possible parallelism from a given set of commands and and Computer Science, vol 510, 1999, pp. 37-55 Springer
available resources. By detecting parallelism, the precedence [3] R. Kolisch and S. Hartmann, “Heuristic algorithms for the
constraints of the commands were maintained. The given set of resourceconstrained project scheduling problem: Classification and
commands were represented as a DAG having data and computational analysis. in Project scheduling. Springer, 1999, pp. 147-
resource dependencies. In order to create a scheduling-list, 178.
such that the nodes in the DAG can be scheduled to the [4] R. Tariq, F. Aadil, M.F. Malik, S. Ejaz, M.U. Khan, M.F. Khan,
available resources, the HLFET algorithm was first used for Directed Acyclic Graph Based Task Scheduling Algorithm for
Heterogeneous Systems. In: Arai K., Kapoor S., Bhatia R. (eds)
prioritizing and scheduling nodes. Intelligent Systems and Applications. IntelliSys 2018. Advances in
HLFET algorithm generally created a scheduling-list for Intelligent Systems and Computing, vol 869. 2019, pp. 936-947.
homogeneous processors. Hence it assumed that all the nodes [5] I. Yu-Kwong Kwok, I. Ahmad. Static scheduling algorithms for
allocating directed task graphs to multiprocessors in ACM Computing
in the DAG use the same kind of resources. But in Surveys, vol. 31, no. 4, 1999, pp.406-471.
manufacturing processes often different kind of resources will
[6] E. Popa, M. Iacono, F. Pop, Adapting MCP and HLFET Algorithms to
be needed by the manufacturing tasks. For this reason, we Multiple Simultaneous Scheduling, in Int J Parallel Prog 46, 2018, pp.
presented a modification of the HLFET algorithm. 607–629.
[7] G. Karypis. Introduction to parallel computing. Addison Weslesy,, 2003
The final DAGs obtained using the scheduling-list from
both HLFET and Modified HLFET algorithm was compared [8] U. Hönig, W. Schiffmann, A comprehensive test bench for the
evaluation of scheduling heuristics, in Proceedings of PCDS’04, 2004.
and analysed by a case study. The results have shown that
[9] A. Olteanu, F. Pop, C. Dobre,V. Cristea,: A dynamic rescheduling
Modified HLFET algorithm produces better results by algorithm for resource management in large scale dependable distributed
detecting parallelism earlier in most evaluated cases thereby systems. In: Computers & Mathematics with Applications 63 (9), 2012,
reducing the total execution time of the commands further. For pp. 1409–1423.
other cases, the results obtained by HLFET and Modified
HLFET algorithm were the same. Although both the

115
 2021 the 13th International Conference on Computer and Automation Engineering

Teleoperation Method of Space Manipulator

Considering both Safety and Efficiency
Xudong Wang Qingxuan Jia Gang Chen Wenqian Xu
School of Automation School of Automation School of Automation School of Automation
Beijing University of Posts and Beijing University of Posts and Beijing University of Posts and Beijing University of Posts and
Telecommunications Telecommunications Telecommunications Telecommunications
Being, China Being, China Being, China Being, China
[email protected] [email protected] [email protected] [email protected]

Abstract—It is difficult to realize the security and operability environment. Research shows that in noncontact tasks, vision
of the space manipulator master-slave teleoperation system contributes about 70% to the total perception information,
simultaneously. A motion mapping algorithm for master while in contact tasks, force-perception becomes more
manipulator and slave manipulator based on piecewise curve and important, accounting for more than 70% of the total
contact warning strategy based on distance are proposed. Firstly, perception [4]. However, because the space manipulator is so
based on the terminal velocity mapping, the distance coefficient is far away from ground controllers, the transmission delay is
introduced into the velocity mapping matrix of the motion about seven seconds in low Earth orbit [5]. Therefore, it is of
mapping algorithm to accomplish the dynamic adjustment of the great significance to study the force-sensing prediction
teleoperation system motion mapping. Then, based on Voxmap-
simulation technology in space teleoperating system. For the
PointShell algorithm, a fast distance calculation algorithm is
designed. The force warning when the end of the manipulator
space manipulator and the environment at the end of the
approaches the environment and the contact force prediction contact rigidity, these conventional force prediction methods
when the end of the manipulator contacts the environment are produce force feedback only after two objects have contacted
completed. The safety and operability of space manipulator each other [6-9]. Operators need to operate the manipulator
master-slave teleoperation system are both realized. slowly, so as not to react too late when the feedback force is
felt, causing irreparable losses. Moreover, due to the great
Keywords—manipulator, teleoperation, distance, mapping, difference in the motion space between the master and slave
force manipulator, it is difficult for the teleoperation system to
balance the efficiency and fineness of operation [10-13].
I. INTRODUCTION Therefore, it is difficult to balance the safety and operability of
With the deepening of human exploration of space, the the space manipulator teleoperation system.
great risk and high cost of astronauts' on-orbit operation make In this paper, using virtual reality technology, based on the
space manipulator technology become the key support of on- VPS algorithm ，a kind of force feedback algorithm with force
orbit service. Due to the diversity and low repeatability of on-
early warning ability is proposed. And a motion matching
orbit operations, the traditional space manipulator
algorithm for the teleoperation system is designed, which takes
preprogrammed control method is extremely inefficient, so it is
into account both security and operability. The speed of the
urgent to improve the intelligence of space manipulator
manipulator has adjusted adaptively according to the distance
operation. However, after more than 30 years of large-scale
between the end of the manipulator and the environment. The
research on intelligent robot technology, it has not been able to
safety and operability of the space manipulator teleoperation
achieve fully autonomous operation. Many experts in the field
system are both realized.
of robotics believe that due to the limitations of current sensors
and artificial intelligence, fully autonomous intelligent robots II. MASTER-SLAVE MANIPULATOR MOTION MAPPING
will be difficult to achieve for a long time in the future [1].
Space manipulator teleoperation technology introduces human A. The Design of Space Manipulator Teleoperation System
intelligence into the closed-loop of the manipulator control The teleoperation system built in this article is shown in Fig.
system, which can greatly improve the intelligence and 1. The system mainly includes operators, human-computer
adaptability of the manipulator to complex tasks. Meanwhile, interaction equipment, virtual reality environment,
NASA also pointed out that in the future, 50% of on-orbit and communication link, slave manipulator, vision and force sensor.
planetary surface work will be realized by teleoperation space In the space manipulator teleoperation system, the slave
robots [2]. manipulator is a space manipulator and the master manipulator
In the space manipulator teleoperation task, ground is Geomagic Touch haptic device.
operators' perception of the operating environment is The controller sends out control information through the
particularly important, and telepresence technology has thus haptic device. After the master controller receives the control
become the core of teleoperation technology [3]. Vision and information, it converts the received terminal information of
touch are the main means for humans to perceive the the master manipulator into the joint control information of the

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 116

slave manipulator and sends the information to the slave environment, a piecewise curve mapping based on terminal
controller through the communication link. The operator can velocity is carried out in the Cartesian coordinate system.
get visual feedback and force feedback in real-time from the
virtual environment while sending out control information.

Fig. 3. Slave manipulator workspace.

Based on the working range of master and slave

manipulator, this paper designs the piecewise velocity mapping
Fig. 1. Space manipulator teleoperation system structure. curve f  x  , as shown in (1). kv is a multiple of the maximum
velocity. d1 is the minimum safe distance threshold. d 2 is the
B. Master-Slave Manipulator Motion Mapping
safe distance threshold. x is the distance between the
The Geomagic Touch is the master manipulator used in this manipulator and the environment. The specific calculation
paper. The desktop manipulator has six degrees of freedom and method of x is introduced in the next section.
two Boolean buttons. Its workspace size is about
0.16  0.07  0.12m . The slave manipulator is a self-developed  kv , x  d2
7-DOF manipulator. The D-H method is adopted to establish 
the system. The physical object and kinematic model of the  k 1 d k d
f  x   v x  2 v 1 , d1  x  d 2 (1)
manipulator are shown in Fig. 2. d
 2  d1 d 2  d1
1, x  d1

The terminal velocity of the master manipulator is as
follows

P
Vme    (2)
 R 
The kinematics model from the velocity stage of the slave
manipulator is xe  J  q , J  represents the Jacobian for the
transfer of joint angles to terminal velocities. Thus, the inverse
kinematic solution of the velocity stage from the manipulator is
shown in (3).
q  J+ xe (3)

Fig. 2. Slave manipulator kinematic model. J+ is the pseudo-inverse of the generalized Jacobian matrix J
of the space manipulator at a fixed base. According to the
The working space of the manipulator is analyzed by velocity mapping curve of the master manipulator and slave
graphic method. As shown in Fig. 3, its workspace size is manipulator, the joint angular velocity of the slave manipulator
about 2.70  2.70  2.67m . can be obtained.
In this paper, the attitude mapping is carried out in an  P  f  x 
equal-scale way. For position mapping, the general incremental q  J+   (4)
motion mapping method is improved based on the closest  R 
distance between the end of the manipulator and the

117
 When the haptic device reaches the limit configuration, the distance calculation. If there is no collision, the two-octree is
bool button at the end of the haptic device can be pressed. At traversed by layers to calculate the nearest distance
this time, the haptic device will not collect the speed MinDistance of the two objects. The pseudo-code for distance
information. calculation is shown in Table II. The algorithm traverses only
one branch of octree, and its algorithm complexity is
III. FORCE GENERATION METHOD FOR AUXILIARY OPERATION
T  n   O  log n  . This complexity is far less than the average
complexity of the collision detection algorithm T  n   O  n2  .
A. Collision Detection and Closest Distance Calculation
In this paper, the point-shell model is used to represent the
end of the manipulator, and the voxel model is used to Therefore, it can be guaranteed that this distance calculation
represent the static object in the scene. The model generation algorithm will not affect the force feedback frame rate.
method is consistent with the traditional Voxmap-PointShell
TABLE II. THE MINIMUM DISTANCE CALCULATION ALGORITHM
algorithm.
Algorithm: Calculate Minimum Distance
First, the AABB envelop box of the model was obtained
based on the OSG graph library, and the envelop box was Input: object1, object2
recursively divided in octree form. Then, the intersection test Output: MinDistance
of leaf nodes and the model was conducted to obtain surface
voxel. Finally, the point-shell model was obtained by 1: function GetMinDistance (object1, object2)
calculating point-shell vertices and normal vectors based on 2: {
surface voxel. The data structure of the point-shell model is
shown in Table I. 3: for i in size (object1.child)

4: for j in size (object2.child)

TABLE I. THE DATA STRUCTURE OF THE POINT-SHELL MODEL
5: Find the nearest object1i and object2j
1: struct PointShell
6: MinDistance = distance (object1i , object2j)
2: {
7: if object1i.child == 0 and
3: double Position[3];
8: object2j.child == 0 then
4: double Size[3];
7: return MinDistance
5: double Direction[3];
8: else recursion goto 3
6: vector<PointShell> Child; //The length is less than eight
9: }
7: }
B. Force Calculation for Auxiliary Operation
The collision detection algorithm based on the point shell After the contact between the slave manipulator and the
and voxel model of the octree only needs to detect whether the environment, this paper uses the idea of punishment to
root node or the AABB envelop box on the upper layer of the calculate the contact force. For all the collision points recorded
octree is in collision when the two models are far apart. The by the collision detection algorithm, the contact force at each
running speed is very fast. After the actual collision of the two point is calculated by the spring model.
models, it needs to go through the whole octree to find all the
collision points, and then the algorithm has the lowest For the collision point i , the equation for calculating the
efficiency. And it should meet the requirement of 1000Hz contact force and moment is
frame rate of force feedback. Obviously, when the two models
are far apart, the collision detection algorithm has considerable  K  Di  ni , di  0
Fi   (6)
time affluence. 0, di  0
Firstly, based on the forward kinematics model of the
Mi  (C  S )  Fi (7)
manipulator and the joint Angle of the manipulator in the
virtual environment issued by the master controller, the pose In the formula, K is the environmental stiffness, and ni is
relation of the end of the slave manipulator relative to the
inertial system was solved. the normal vector of the point shell model. Di  di  tsdfi is
the depth of the collision point embedded in the voxel model,
I
eT  bIT b0T 01T (1 ) 12T (2 ) T (n ) neT
n 1
n (5) and d i the embedding depth of the point and the voxel unit
colliding with it, and tsdfi is the base depth, representing the
Then the collision detection or distance calculation between
voxel and point shell can be carried out according to the pose distance between the central point of the voxel unit and the
relation of the environmental object to the inertial system. model surface. C is the position of the collision point, and S
Firstly, the traditional octree collision detection algorithm is is the simplification point of torque.
used to detect whether two objects collide or not. If there is a The resultant collision force and resultant torque are
collision, all collision points are directly recorded without

118
 F  Fi twice during the whole control process in this paper and takes a
(8) total of 14s . The traditional method resets the haptic device 8
M  M i
times in the whole control process and takes a total of 46s .
Before contact with the environment, this paper presents a
calculation method of pre-contact force based on distance.
When approaching the environment, a pre-contact force
warning is provided.

k  Sd  x  s , S d  x  0

F  x   (9)
0,
 Sd  x  0
k is the pre-contact stiffness. x is the nearest distance between Fig. 5. Terminal velocity of the manipulator and task time.
the end of the slave manipulator and the environment. S d is a
safety threshold that can be set. When the closest distance is In (9), take Sd  0.1m , k  20N/m . When the end of the
less than this threshold, a pre-contact force warning is provided. manipulator approaches the operated object, the pre-contact
s is the direction vector determined at the nearest two points, force is shown in Fig. 6.
pointing from the environment to the end of the manipulator.
IV. RESULT
To verify the method proposed in this paper. Simulate the
process of the end of the manipulator approaching the operated
object. Fig. 4 shows the simulation scenario.
Fig. 6. Pre-contact force at the manipulator approaching the environment.

The initial distance between the end of the manipulator and

the target object is modified in the range of 0.1m to 1.5m.
Multiple experiments were carried out according to the above
experimental parameters. Compare the time consumed by the
piecewise curve mapping method and the traditional equal-
scale mapping method. The experimental results are shown in
Fig. 7. When the end of the manipulator is far from the target,
the method presented in this paper has obvious advantages.

Fig. 4. Visual simulation.

The operated object A is located directly below the end of

Fig. 7. Comparison of time consumption.
the manipulator, one meter from the end of the manipulator.
Suppose that the operator controls the end of the manipulator
vertically down a straight line towards the object being V. CONCLUSION
operated. The operator controls the master haptic device at a The method proposed in this paper increases the scale of
speed of 0.05m/s and an acceleration of 0.2m/s2 . Due to the motion mapping in the regions where the slave manipulator can
limitation of the working space of the master haptic device, move safely, and automatically restores the scale mapping in
when the master haptic device moves to the limit configuration, the regions requiring a fine operation, thus realizing the
the operation time of "lock-reset-unlock" for the haptic device dynamic programming of motion mapping. Furthermore, it
is 3s . provides a pre-contact force warning when approaching the
environment from the end of the manipulator, which further
The traditional incremental motion matching algorithm and improves the safety of the teleoperation system. Therefore, the
the motion matching algorithm proposed in this paper are used method proposed in this paper greatly improves the telescopic
for simulation experiments respectively. In (1), the maximum operation efficiency of the manipulator, and the effect is more
velocity multiple is set as kv  5 , the minimum safe distance significant when moving from the manipulator in a large range.
threshold is set as d1  0.1m , and the maximum safe distance However, since force warning is adopted in this paper for
threshold is set as d2  0.5m . The velocity curve at the end of collision warning, how to set environmental stiffness to allow
operators to distinguish actual collision force and warning
the manipulator is shown in Fig. 5. The haptic device is reset force remains to be studied.

119
 ACKNOWLEDGMENT [7] J. Barbic and L. D. James, “Six-DoF haptic rendering of contact
between geometrically complex reduced deformable models,” IEEE.
The project was supported by the National Natural Science Trans. Haptic, vol. 1, pp. 39–52, February 2008.
Foundation of China under Grant 51975059 and Grant [8] D. Zhang, Y. Li, and J. Barbic, “6-DoF Haptic Rendering of Static
61802363. Coulomb Friction Using Linear Programming,” IEEE. Trans. Haptic,
vol. 11, pp. 325–337, September 2018.
REFERENCES [9] M. Sagardia and T. Hulin, “A fast and robust six-DoF god object
[1] Kazerooni H, “Human-robot interaction via the transfer of power and heuristic for haptic rendering of complex models with friction,” Proc.
information signals,” IEEE. Trans. Syst. Man. Cybern, vol. 20, pp. 450– ACM Symp. Virtual Reality Softw. Technol. VRST, vol. N02, pp. 163–
463, April 1990. 172, November 2016.
[2] H. Tamas, K. Levente, P. R. emil, B. Balaazs, B. Zoltan, and P. Stefan, [10] X. Gong, J. Zhao, J. Yu, and G. Zuo, “Motion mapping of the
“Simulation and control for telerobots in space medicine,” Acta. heterogeneous master-slave system for intuitive telemanipulation,” Int. J.
Astronaut, vol. 81, pp. 390–402, December 2012. Adv. Rob. Syst, vol. 15, January 2018.
[3] Satava R M, “Robotics, telepresence and virtual reality-A critical [11] A. Jaju, A. P. Das and P. K. Pal, “Evaluation of motion mappings from
analysis of the future of surgery,” Minimally Invasive Therapy, vol.6, pp. a haptic device to an industrial robot for effective master-slave
357–363, January 1992. manipulation,” Int. J. Rob. Autom, vol. 28, pp. 65–71, 2013.
[4] Z. Yidong, Y. Jinsong, T. Diyin, and C. Xiaoqi, “An improved force [12] G. Liu, X. Geng, L. Liu and Y. Wang, “On certain integrals of
feedback method for teleoperation control system based on standard PD Lipschitz-Hankel type involving products of Bessel functions,” Chin. J.
bilateral control,” Zhongnan. Daxue. Xuebao, vol. 49, pp. 865–873, Aeronaut, vol. 32, pp. 723–736, March 2019.
April 2018. [13] P. Vyacheslav, T. Fariza, S. Vladislav, K. Mikhail, and U. Nikolai,
[5] H. Farzad, H. Iraj, T. Mahdi, and A. Ghasem, “Adaptive control for state “Mathematical method of mapping configuration space for manipulator
synchronization of nonlinear haptic telerobotic systems with asymmetric master-slave teleoperation,” CEUR. Workshop. Proc, vol. 2500, pp. 1–9,
varying time delays,” J. Intell. Rob. Syst. Theor. Appl, vol. 68, pp. 245– September 2019.
259, December 2012.
[6] C. Siyu, Y. Ligang, H. Yehliang, and C. Teddy, “Development of an
Interactive System for a Companion Robot Based on Telepresence
Technology,” Mech. Mach. Sci, vol. 78, pp. 219–226, January 2020.

120
 2021 the 13th International Conference on Computer and Automation Engineering

Prediction of Tensile Strength and Erosional

Effectiveness of Natural Geotextiles Using Artificial
Neural Network
Dante L. Silva Kevin Lawrence M. de Jesus Edgar M. Adina
School of Civil, Environmental and School of Graduate Studies School of Civil, Environmental and
Geological Engineering Mapua University Geological Engineering
Mapua University Manila, Philippines Mapua University
Manila, Philippines [email protected] Manila, Philippines
[email protected] [email protected]

Daelson V. Mangrobang Mareshah D. Escalante Nicole Anne M. Susi

School of Civil, Environmental and School of Civil, Environmental and School of Civil, Environmental and
Geological Engineering Geological Engineering Geological Engineering
Mapua University Mapua University Mapua University
Manila, Philippines Manila, Philippines Manila, Philippines
[email protected] [email protected] [email protected]

Abstract— Soil erosion is a phenomenon observable in sloped is brought about by wind, water and gravity. The natural
terrains which is brought about by heavy precipitation and factors that trigger soil erosion includes rainfall, soil erodibility,
strong winds. The utilization of natural geotextiles which is slope of land and vegetative cover [2]. Due to these conditions,
effective, cost – efficient, environmentally – sustainable and an effective and economical solution for soil erosion is highly
standard – adhering measure is essential to stabilize the natural necessary. Natural geotextiles are permeable, woven or non –
inclination of the soil and to curb erosion. To totally understand woven fabrics from plant fibers which support vulnerable soil
the primary importance of different natural geotextiles in tensile from eroding until vegetation such as grasses and leguminous
strength and erosional effectiveness, an Artificial Neural plants have fully grown. Natural geotextiles are also referred to
Network (ANN) model was developed. Three different fibers
as green geotextiles due to their beneficial effects to soil,
were used as natural geotextiles due to its high tensile strength
and excellent moisture – absorption properties including coconut,
renewability, biodegradability and short – term application. In
abaca and water hyacinth. These three fibers were also employed most developing countries like the Philippines, cost and local
as the predictors in the ANN model. A total of 31 specific ratios availability play key roles in the selection of the geotextile
of these three fibers were utilized as sample geotextiles for the material. Natural fibers are preferred more often in such
development of tensile strength, runoff reduction effectiveness countries since it is more economical choice as compared to
(RRE), and sediment yield reduction effectiveness (SRYE) synthetic materials for a limited time period [3].
models. A ranking of importance of the different fibers to the
The Philippines is geographically rich in coconut, abaca
respective outputs was also performed by Feature Reduction
and water hyacinth fibers which have simpler manufacturing
Approach to evaluate the influence of each type of fiber with
regards to tensile strength, runoff reduction effectiveness (RRE),
processes and drastically lesser negative effects on the
and sediment yield reduction effectiveness (SYRE) of the natural environment than synthetic geotextiles which are composed of
geotextile blend. The proposed ANN models generated excellent materials such as polypropylene, polyester, polyethylene, nylon,
prediction performance based on its extremely high R values and and polyvinyl chloride. Natural fibers are also more effective in
very low mean squared error (MSE). moisture absorption than synthetic materials and usually
transforms into an organic fertilizer which not only leaves little
Keywords—tensile strength, runoff reduction effectiveness, to no harmful effects to the soil but rather increases its
sediment yield reduction effectiveness, artificial neural network, productive capability [4]. Natural geotextiles may be applied
feature reduction for the stabilization of natural sloped lands such as hillsides,
shorelines, levee dikes, river banks and soft soils as well as
I. INTRODUCTION man – made embankments such as temporary or haul roads,
Soil degradation is categorized by the Global Assessment landfills, basal embankments, and consolidation drains [5]. The
of Soil Degradation (GLASOD) into two main classifications: utilization of organic fibers not only creates opportunities of
soil degradation due to displacement of soil and chemical livelihood to individuals in the field of agriculture but also
processes [1]. Soil erosion disposes the fertile topsoil with promotes sustainable development through a geotechnical
favorable properties such as moisture retention and infiltration engineering solution. In the aspiration of producing successful
which in effect reduces the productivity of the soil. Soil erosion prediction models of erosional effectiveness and durability of

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 121

natural geotextiles, the combination of various locally available Effectiveness (SYRE) were utilized in the development of the
fibers including coconut, abaca and water hyacinth is Artificial Neural Network models. A threshold value of 60%
performed. for RRE and 80% for SYRE were used for the samples [9].
In line with the United Nations – Sustainable Development The runoff coefficient (ROC %) was computed using
Goals (SDG) and the onset of different disciplines to Industry equation 1 which used the resulting ROC of the bare soil plot
4.0, the use of machine learning techniques such as Artificial and the respective ROC of each specimen.
Neural Networks allows an accurate and adaptable predictive 𝑅𝑢𝑛𝑜𝑓𝑓 𝑉𝑜𝑙𝑢𝑚𝑒
analysis in determining the values for tensile strength, runoff 𝑅𝑂𝐶 = 𝑥 100% (1)
𝑅𝑎𝑖𝑛𝑓𝑎𝑙𝑙 𝑉𝑜𝑙𝑢𝑚𝑒
reduction effectiveness (RRE) and sediment yield reduction
effectiveness (SYRE) based on the respective specific ratios of where ROC is the runoff coefficient (%), runoff volume is the
the various fibers [6]. A sensitivity analysis is also performed total volume of runoff collected (mL) and rainfall volume is
to study the influence of each fiber to the tensile strength, the total volume of rainfall collected (mL).
runoff, and sediment yield of the natural geotextile blend [7]. The runoff reduction effectiveness (RRE %) is calculated
This study aims to perform an adaptable computation by using equation 2.
predictive analysis of tensile strength, and erosional 𝐵𝑎𝑟𝑒 𝑅𝑂𝐶 %−𝐺𝑒𝑜𝑡𝑒𝑥𝑡𝑖𝑙𝑒 𝑅𝑂𝐶 %
effectiveness defined as the reduction of run-off and sediment 𝑅𝑅𝐸 = 𝑥 100% (2)
𝐵𝑎𝑟𝑒 𝑅𝑂𝐶 %
yield of natural – blend geotextiles by developing an Artificial where RRE is the runoff reduction effectiveness %, bare ROC
Neural Network – based model. is the runoff coefficient of bare soil sample % and geotextile
II. METHODOLOGY ROC is the runoff coefficient of soil sample with natural
geotextile cover %.
A. Collection of Data for Model Development
The sediment yield reduction effectiveness (SYRE %) is
The study targets to determine the effectiveness of coconut, determined using equation 3.
abaca and water hyacinth fibers as natural geotextiles in terms
𝐵𝑎𝑟𝑒 𝑆𝑌−𝐺𝑒𝑜𝑡𝑒𝑥𝑡𝑖𝑙𝑒 𝑆𝑌
of soil erosion mitigation which is defined as the reduction of 𝑆𝑌𝑅𝐸 = 𝑥 100% (3)
𝐵𝑎𝑟𝑒 𝑆𝑌
runoff and sediment yield while owning a tolerable tensile
strength. Soil erosion is defined in the study as only pure water where SYRE is the sediment yield reduction effectiveness %,
erosion or erosion caused by precipitation, simulated for 1-hour bare SY is the sediment yield of bare soil sample (g), geotextile
runs in an aritificial rainfall apparatus manufactured by Maruto ROC is the sediment of soil sample with natural geotextile
Testing Machine Co., Ltd. Located in the Hydraulics cover (g).
Laboratory of the Flood Control and Sabo Engineering Center A total of 31 specific ratios of the three fibers were utilized
(FCSEC) of the Department of Public Works and Highways as sample geotextiles for the model development. The
(DPWH) located in Pasig City. The tensile strength was geotextile sample product utilized in this study is shown in Fig.
evaluated using a Universal Testing Machine (UTM) 1. The natural geotextile blends shall take on dimensions of
manufactured by Murasakino Corp. Shimadzu Corp. located in 0.5m-width by 1.0m-length with 1 in-square holes wherein its
the UTM Center of Mapua University in Intramuros, Manila. ratioes were taken by twine thickness and shall undergo the
This study evaluated the strength of the geotextiles in tensile strength and rainfall simulation tests. The descriptive
tension in Newtons per twine to which a tolerable value for statistics of the fiber ratios used in the model development
tensile strength per twine with a set of 5mm (±10%) thickness were presented in Table I.
shall be 150 N as specified by the DPWH Standard
Spefications for Coconet Bio-engineering Solutions – Item 518.
Tensile strength tests were conducted with 1-m length twines
with uniform twine thickness and were done to the rupture
point of the twines.
The rainfall simulation provided two outputs including
runoff and sediment yield which are measured at 10-minute
intervals for a full duration of 1 hour per run. The geotextile
samples were observed and applied into soil stored in
perforated, wooden boxes inclined at a single, fixed 1H:1V
inclination or simply a 45-degree slope and exposed to Fig. 1. Geotextile Sample Product
precipitation discharged at 5 ± 0.05 cu.m/hr or its equivalent
intensity of 125 ± 10 mm/hr, patterned after the rainfall TABLE I. FIBER RATIOS USED IN THE MODEL DEVELOPMENT
intensity of typhoon Milenyo which made its landfall in Albay
Specific Ratio (in %)
on December 2006 [8]. The soil samples were classified and Description
Abaca Coconut Water Hyacinth
stored in the Hydraulics Laboratory of the DPWH-FCSEC in Maximum 100.00 100.00 100.00
Pasig City. The runoff and sediment yield reduced by the Minimum 0.00 0.00 0.00
natural geotextiles are to be compared with that of bare soil. Average 33.32 33.32 33.32
The performance of the samples in tensile strength, Runoff Standard Deviation 27.73 27.73 27.73
Reduction Effectiveness (RRE) and Sediment Yield Reduction

122
 Independent ANN models for tensile strength, runoff The tensile strength test results for geotextile specimens
reduction effectiveness and sediment yield reduction with varying specific ratios by thickness (in %) of Abaca,
effectiveness were developed and the influence of each fiber Coconut Fiber and Water Hyacinth. The minimum tensile
type to the outputs were eveluated using sensitivity analysis. strength requirements of DPWH for geotextiles is 150 N per
twine (twine diameter thickness of 5mm (±10%)). The various
B. Artificial Neural Network Prediction Model Development twined combinations were exposed to elongation until rupture
The development of the Artificial Neural Network is by a Universal Testing Machine (UTM) and their tensile
composed of three stages: Training, Validation, and Testing as strength at rupture was recorded. The highes tensile strength
shown in Fig. 2. During the training stage, the weights and recorded was 2,015.63 N.
biases are generated by the MATLAB after the simulation.
This phase utilized 70% of the input data which is a default TABLE II. TENSILE STRENGTH TEST RESULT, RUNOFF REDUCTION
value set in the nntoolbox of the MATLAB. These weights and EFFECTIVENESS AND SEDIMENT YIELD REDUCTION EFFECTIVENESS
SIMULATION RESULTS
biases were the link and association between the input and the
output which were initially unknown. The second phase of the Tensile
Cumulative Cumulative
model development is the validation phase. During validation, Description Strength
RRE (in %) SYRE (in %)
(N)
the initial test of the weights and biases were performed which
Maximum 2015.63 69.45 96.69
used the 15% of the overall data set. The last stage is the
testing phase which utilized the remaining 15% of the data. In Minimum 101.87 1.92 79.86
this phase, the final test for the weights and biases were Average 490.46 32.33 92.13
performed [10].
Standard Deviation 411.94 19.00 5.35

For the rainfall simulation, it considers the cumulative

volume of rainfall discharged and cumulative volume of runoff
collected per soil box at a 1 – hour run. The volume of rainfall
was held constant at a discharge of 5 ± 0.05 cu.m/hr for every
Fig. 2. Neural Network Model Development Phases run with a cumulative volume of rainfall of 5,000,000 mL per
run. The volume of rainfall was collected per soil box at every
The neural network model will utilize the Levenberg – 10-minute interval, for a total of 6 intervals per run. The
Marquardt Training Algorithm due to its suitability and being highest value of runoff was at 23,440 mL which was produced
the fastest type of training algorithm for relatively small data by the fiber combination of 45% abaca, 10% coconut and 45%
sets [11]. The transfer functions simulated in the model water hyacinth while the lowest value for volume of runoff is
development includes the tansig, logsig and purelin transfer at 5,302 mL. The minimum threshold value for runoff
functions. Hyperbolic tangent sigmoid function (tansig) reduction effectiveness (RRE) is 60%.
utilized a normalized values ranging from -1 to +1 while the
log-sigmoid transfer function (logsig) employed normalized The cumulative collected sediment yield were measured in
values from 0 to +1 and purelin is the linear type of transfer grams and the sediment yield reduction effectiveness (SYRE)
function [12]. is in percentagae of geotextile specimens. The SYRE was
compared with the sediment yield of the bare soil plot with the
The performance of the models were evaluated based on reduced sediment yield of the soil plots with geotextile
the following criteria: highest value for the Pearson’s application.
Correlation Coefficient (Rall) and lowest Mean Squared Error
[13]. For the number of data sets required for the model B. Neural Network Simulation Results
training and development, the rule of thumb is suggested to Using the Levenberg – Marquardt algorithm as the training
have at least 10 times the number of input parameters [14], algorithm, the simulation results for prediction the tensile
[15]. strength in geotextile considering different fibers including
C. Feature Reduction Approach abaca, water hyacinth and coconut fiber were presented in
Table III.
The contribution and ranking of importance of each fibers
to the tensile strength test results, runoff reduction TABLE III. INTERNAL CHARACTERISTICS OF THE TENSILE STRENGTH
effectiveness (RRE) and sediment yield reduction effectiveness MODEL
(SYRE) were determined using Feature Reduction Approach.
Internal Characteristic Value
The feature reduction approach was performed by simulating
models with reduced or truncated input parameters [16]. Training Algorithm Levenberg – Marquardt
Transfer Function Hyperbolic Tangent Sigmoid (tansig)
III. RESULTS AND DISCUSSION
Number of Hidden Neurons 4
A. Preliminary Data Test and Simulation Results
Pearson’s Correlation
0.97629
The model outputs include the tensile strength test results, Coefficient (Rall)
runoff reduction effectiveness (RRE) and sediment yield Mean Squared Error 0.00116
reduction effectiveness (SYRE) were presented in Table II.

123
 For the runoff reduction efficiency model, the simulation
results for predicting RRE in geotextile considering different
fibers including abaca, water hyacinth and coconut fiber were
presented in Table IV.

TABLE IV. INTERNAL CHARACTERISTICS OF THE RUNOFF REDUCTION

EFFICIENCY MODEL

Internal Characteristic Value

Fig. 3. Architecture of the Governing Tensile Strength Model
Training Algorithm Levenberg – Marquardt
The simulation results presents that the best model for Transfer Function Hyperbolic Tangent Sigmoid (tansig)
predicting tension in geotextile considering different fibers
Number of Hidden Neurons 8
such as Abaca, Water Hyacinth and Coconut Fiber has a final
topology of 3-4-1 (Input Neuron-Hidden Neuron-Output Pearson’s Correlation
0.96019
Coefficient (Rall)
Neuron). Based on the final stopping criteria, it has the highest
of Rall = 0.97629 and lowest MSE of 0.00116 as compared to Mean Squared Error 0.00013
the other parameters simulated. The architecture of the final
tension model was presented in Fig. 3. The simulation results presents that the best model for
The MSE and R values for the different phases it predicting runroff reduction efficiency in geotextile
undergone (training, validation and testing phase) during the considering different fibers such as Abaca, Water Hyacinth and
development of the model were exhibited in Fig. 4 and Fig. 5. Coconut Fiber has a final topology of 3-8-1 (Input Neuron-
Hidden Neuron-Output Neuron). Based on the final stopping
criteria, it has the highest of Rall = 0.96019 and lowest MSE of
0.00013 as compared to the other parameters simulated. The
architecture of the final tension model was presented in Fig. 6.

Fig. 4. Performance Plots for the Phases of Development of the best Tensile
Strength Model
Fig. 6. Architecture of the Governing Runoff Reduction Efficiency Model

The MSE and R values for the different phases it

undergone (training, validation and testing phase) during the
development of the model were exhibited in Fig. 7 and Fig. 8.

Fig. 7. Performance Plots for the Phases of Development of the best Runoff
Fig. 5. Correlation Plots for the Phases of Development of the best Tensile Reduction Efficiency Model
Strength Model

124
 The MSE and R values for the different phases it
undergone (training, validation and testing phase) during the
development of the model were exhibited in Fig. 10 and Fig.
11.

Fig. 8. Correlation Plots for the Phases of Development of the best Runoff Fig. 10. Performance Plots for the Phases of Development of the best
Reduction Efficiency Model Sediment Yield Reduction Efficiency Model

For the sediment yield reduction efficiency model, the

simulation results for predicting SYRE in geotextile
considering different fibers including abaca, water hyacinth
and coconut fiber were presented in Table V.

TABLE V. INTERNAL CHARACTERISTICS OF THE SEDIMENT YIELD

REDUCTION EFFICIENCY MODEL

Internal Characteristic Value

Training Algorithm Levenberg – Marquardt
Transfer Function Hyperbolic Tangent Sigmoid (tansig)

Number of Hidden Neurons 5

Pearson’s Correlation
0.98779
Coefficient (Rall)
Mean Squared Error 0.00012

The simulation results presents that the best model for

predicting sediment yield reduction efficiency in geotextile
considering different fibers such as Abaca, Water Hyacinth and
Coconut Fiber has a final topology of 3-5-1 (Input Neuron-
Hidden Neuron-Output Neuron). Based on the final stopping
criteria, it has the highest of Rall = 0.98779 and lowest MSE of
0.00012 as compared to the other parameters simulated. The
Fig. 11. Correlation Plots for the Phases of Development of the best Sediment
architecture of the final tension model was presented in Fig. 9. Yield Reduction Efficiency Model

C. Feature Reduction Approach

The ranking of importance of Abaca Fibers, Water
Hyacinth Fibers and Coconut Fibers to tensile strength were
analyzed using Feature Reduction Approach. Table VI shows
the simulation results using the governing training algorithm
and transfer function. The result displays that the highest R all
equal to 0.97403 and lowest MSE equal to 0.00482 was
obtained by reducing water hyacinth fibers in the parameters
Fig. 9. Architecture of the Governing Sediment Yield Reduction Efficiency used in predicting tensile strength. This findings signifies that
Model reducing the water hyacinth fibers has a minimal effect in the

125
stopping criteria utilized in selecting the best model. The
ranking of importance were presented in Fig. 12.

TABLE VI. SIMULATION RESULTS FOR FEATURE RDUCTION APPROACH –

TENSILE STRENGTH MODEL
Number of
Parameters Training Transfer
Hidden Training
Considered Algorithm Function
Neurons
Levenberg-
A, C Tansig 4 0.91907
Marquardt
Levenberg- Fig. 13. Ranking of Importance of the Fiber Parameters in Runoff Reduction
A, W Tansig 4 0.94587 Efficiency Model
Marquardt
Levenberg-
C, W Tansig 4 0.95323
Marquardt The ranking of importance of Abaca Fibers, Water
Parameters
Validation Testing All MSE Hyacinth Fibers and Coconut Fibers to sediment yield
Considered reduction efficiency were analyzed using transfer function.
A, C 0.99973 0.90943 0.97403 0.00482 The result displays that the highest R all equal to 0.96559 and
A, W 0.98941 0.99128 0.96693 0.00754
lowest MSE equal to 0.00036 was obtained by reducing Abaca
Fiber in the parameters used in predicting runoff reduction
C, W 0.98195 0.91287 0.94463 0.01483 efficiency. This findings signifies that reducing the Abaca
Fiber has a minimal effect in the stopping criteria utilized in
selecting the best model. The ranking of importance were
presented in Fig. 14.

TABLE VIII. SIMULATION RESULTS FOR FEATURE RDUCTION APPROACH –

SEDIMENT YIELD REDUCTION EFFICIENCY MODEL
Number of
Parameters Training Transfer
Hidden Training
Considered Algorithm Function
Neurons
Levenberg-
A, C Tansig 5 0.99392
Marquardt
Fig. 12. Ranking of Importance of the Fiber Parameters in Tensile Strength Levenberg-
A, W Tansig 5 0.98147
Model Marquardt
Levenberg-
C, W Tansig 5 0.97338
The ranking of importance of Abaca Fibers, Water Marquardt
Hyacinth Fibers and Coconut Fibers to runoff reduction Parameters
Validation Testing All MSE
Considered
efficiency were analyzed using transfer function. The result
displays that the highest R all equal to 0.93928 and lowest A, C 0.98578 0.93154 0.98328 0.00032
MSE equal to 0.00208 was obtained by reducing Water A, W 0.91639 0.9288 0.95655 0.00080
Hyacinth in the parameters used in predicting runoff reduction
efficiency. This findings signifies that reducing the Water C, W 0.93581 0.94402 0.95915 0.00043
Hyacinth has a minimal effect in the stopping criteria utilized
in selecting the best model. The ranking of importance were
presented in Fig. 13.

TABLE VII. SIMULATION RESULTS FOR FEATURE RDUCTION APPROACH –

RUNOFF REDUCTION EFFICIENCY MODEL
Number of
Parameters Training Transfer
Hidden Training
Considered Algorithm Function
Neurons
Levenberg-
A, C Tansig 8 0.98027
Marquardt
Fig. 14. Ranking of Importance of the Fiber Parameters in Sediment Yield
Levenberg-
A, W Tansig 8 0.90136 Reduction Efficiency Model
Marquardt
Levenberg-
C, W Tansig 8 0.98242 IV. CONCLUSION
Marquardt
Parameters
Validation Testing All MSE Models for predicting tensile strength, runoff reduction
Considered
Effectiveness and sediment yield reduction effectiveness were
A, C 0.98531 0.95179 0.93928 0.00208 developed using a feed forward back propagation algorithm
A, W 0.92995 0.94862 0.90228 0.00492 considering the integration of several fibers including Abaca,
Coconut Coir and Water Hyacinth. The tensile strength model
C, W 0.95928 0.9183 0.96158 0.00155
has a final topology of 3-4-1 (Input Neuron-Hidden Neuron-
Output Neuron). The selection of the best model was based on
the highest Rall and lowest MSE of 0.97629 and 0.00116,

126
respectively. The Runoff Reduction Effectiveness model has a Journal of Thermoplastic Composite Materials, pp. 0892705719844546,
governing design of 3-8-1 (Input Neuron-Hidden Neuron- 2019.
Output Neuron) with highest Rall value of 0.96019 and lowest [5] M. Pritchard, R. W. Sarsby, and S. C. Anand, “Textiles in civil
engineering. Part 2 – natural fibre geotextiles,” Handbook of technical
MSE of 0.00013. The Sediment Yield Reduction Effectiveness textiles, pp. 372–406, 2000.
model has a final design of 3-5-1 (Input Neuron-Hidden [6] D. L. Silva, K. L. M. de Jesus, B. S. Villaverde, E. M. Adina, “Hybrid
Neuron-Output Neuron) with 0.98779 and 0.00012 as the Artificial Neural Network and Genetic Algorithm Model for Multi-
highest Rall and lowest MSE, respectively. Objective Strength Optimization of Concrete with Surkhi and Buntal
Fiber,” In Proceedings of the 2020 12th International Conference on
The ranking of importance and influence of each fiber were Computer and Automation Engineering, pp. 47–51, 2020.
calculated using feature reduction approach. For the tensile [7] M. Hosseninpour, Y. Sharifi and H. Sharifi, “Neural network application
strength model, the order of influence observed was Water for distortional buckling capacity assessment of castellated steel beams,”
Hyacinth<Coconut Coir<Abaca Fiber being the Abaca Fiber as In Structures, Elsevier, vol. 27, pp. 1174–1183, 2020.
the most influential factor in the intensity of the tensile strength [8] H. Francisco and M. delos Angeles, “Soil resource depreciation and
of the geotextile. Considering runoff reduction Effectiveness, deforestation: Philippine case study in resource accounting,” Planning
and Statistics Branch, Policy and Planning Division, Forestry
the ranking of importance was Abaca<Water Department, Rome: Food and Agriculture Organization, 1998.
Hyacinth<Coconut Fiber being the coconut coir fiber as the
[9] J. Álvarez-Mozos, E. Abad, R. Giménez, M. A. Campo, M. Goñi, M.
most important parameter. Finally, for the sediment yield Arive, J. Casalí, J. Díez, and I. Diego, “Evaluation of erosion control
reduction Effectiveness, the order of influence was water geotextiles on steep slopes. Part 1: Effects on runoff and soil loss,”
hyacinth<abaca<coconut fiber being coconut fiber as the most Catena, vol. 118, pp. 168–178, 2014.
influential factor to the sediment yield reduction Effectiveness. [10] I. Kucukoglu, H. Atici-Ulusu, T. Gunduz, and O. Tokcalar, “Application
of the artificial neural network method to detect defective assembling
The results present that coupling artificial neural network to processes by using a wearable technology,” Journal of manufacturing
parametric analysis is an effective tool for developing systems, vol. 49, pp. 163–171, 2018.
predictive equations and assessing the behavior for tensile [11] S. Asadi, E. Hadavandi, F. Mehmanpazir, and M. M. Nakhostin,
strength, runoff reduction effectiveness and sediment yield “Hybridization of evolutionary Levenberg–Marquardt neural networks
and data pre-processing for stock market prediction,” Knowledge-Based
reduction effectiveness of natural geotextiles. Systems, vol. 35, pp. 245–258, 2012.
ACKNOWLEDGMENT [12] A. Tümer and S. Edebali, “Modeling of trivalent chromium sorption
onto commercial resins by artificial neural network,” Applied Artificial
The authors would like to express their gratitude to Mapua Intelligence, vol. 33(4), pp. 349–360, 2019.
University for the support towards the completion of this study. [13] N. Attoue, I. Shahrour, and R. Younes, “Smart building: Use of the
artificial neural network approach for indoor temperature forecasting,”
REFERENCES Energies, vol. 11(2), pp. 395, 2018.
[1] V. B. Asio, R. Jahn, F. O. Perez, I. A. Navarrete and S. M. Abit Jr., “A [14] T. Kavzoglu and P. M. Mather, “The use of backpropagating artificial
review of soil degradation in the Philippines,” Annals of Tropical neural networks in land cover classification,” International journal of
Research, vol. 31(2), pp. 69–94, 2009. remote sensing, vol. 24(23), pp. 4907–4938, 2003.
[2] M. Prambauer, C. Wendeler, J. Weitzenböck, and C. Burgstaller, [15] D. Laroza Silva and K. L. Marcelo De Jesus, “Backpropagation Neural
“Biodegradable geotextiles – An overview of existing and potential Network with Feature Sensitivity Analysis: Pothole Prediction Model
materials,” Geotextiles and Geomembranes, vol. 47(1), pp. 48–59, 2019. for Flexible Pavements using Traffic and Climate Associated Factors,”
In 2020 the 3rd International Conference on Computing and Big Data,
[3] R. V. Mendoza, A. Guinto, J. Castor, F. Anzures, and K. Joaquin, pp. 60–67, 2020.
“Terra-rete-the utility of coir nets with seaweed fertilizer for a
preemptive effort in soil erosion and degradation,” International Journal [16] J. N. Velasco and C. F. Ostia, “Development of a Neural network Based
of Advanced Research in Engineering Innovation, vol. 1(1), pp. 22–30, PV Power Output Prediction Application Using Reduced Features and
2019. Tansig Activation Function,” In 2020 6th International Conference on
Control, Automation and Robotics (ICCAR), IEEE, pp. 732–735, 2020.
[4] A. Lotfi, H. Li, D. V. Dao, and G. Prusty, “Natural fiber – reinforced
composites: A review on material, manufacturing, and machinability,”

127
 2021 the 13th International Conference on Computer and Automation Engineering

Development of a Low Cost System for Lifting

Related Injury Analysis
Mahmudul Hasan Miftahur Rahman
Technical Service School of Aerospace, Transport and Manufacturing
Runner Motors Ltd Cranfield University
Tongi, Bangladesh Cranfield, UK
[email protected] [email protected]

Abstract—In the developing countries, most of the Manual skeletal models cannot give us information about the muscle
Material Handling (MMH) related tasks are labor-intensive. It is forces and activations. The injury evaluation using skeletal
not possible for these countries to assess injuries during lifting models may not be accurate, as muscle-tendon dynamics is
heavy weight, as multi-camera motion capture, force plate and
electromyography (EMG) systems are very expensive. In this not counted there. AnyBody (AnyBody technology, Denmark)
study, we proposed an easy to use, portable and low cost system, is a biomechanical modeling software which helps to calculate
which will help the developing countries to evaluate injuries for muscle force, joint reaction forces and moments. But AnyBody
their workers. The system consists of two hardware and three Modeling System is not an open-source software. Most of
software. The joint angle profiles are collected using smartphone these biomechanical models and software are lab-based and
camera and Kinovea software. The vertical Ground Reaction
Force (GRF) is collected using Wii balance board and Brainblox are not free for all, making it infeasible and inaccessible for
software. Finally, the musculoskeletal analysis is performed using the developing countries to use those in industries. OpenSim
OpenSim Static Optimization tool to find the muscle force. The is a widely popular open-source software for biomechanical
system will give us access to a comprehensive biomechanical modeling, simulation and analysis [11]. OpenSim is used in
analysis, from collecting joint angle profiles to generating muscle evaluating the lifting injuries [12], [13], [14], [15], [16]. But,
force profiles. This proposed framework has the potential to
assess and prevent MMH related injuries in developing countries. the input of both Anybody and OpenSim, require kinematics
and kinetics data. These require multi-camera human motion
Index Terms—Symmetric lifting, Kinovea, OpenSim, Wii bal- capture device and force plate which are very expensive.
ance board, Static optimization, Muscle force, Ground reaction Several studies were done on Kinovea [17], [18], [19], [20],
force, Manual material handling [21], [22] and Wii balance board [23], [24], [25], [26] as an
alternative solution to precision motion capture and force plate,
I. I NTRODUCTION mainly for measuring the center of pressure and gait analysis.
Low back pain is one of the leading causes of disability In a study, Wii balance board was integrated with OpenSim to
and musculoskeletal disorder (MSDs). The epidemiology of analyze static balance [27]. No study was done using Kinovea
low back pain related disorders got attention in developed and Wii balance board to analyze lifting related injuries.
countries. The economic impact of low back pain only in the In this study, we will propose a system by integrating Wii
united states is more than $100 billion [1]. However, it remains balance board and Kinovea software with OpenSim for mus-
unnoticed in developing countries due to socioeconomic con- culoskeletal analysis. This low cost system will help the
ditions, lack of awareness and low quality healthcare system. industries of developing countries to perform a comprehensive
Several researchers approached different ways to analyze occu- biomechanical analysis for the workers and assess lifting
pational MSDs. The use of multi-camera motion capture sys- related injuries. In our knowledge, this is the first study where
tem, force plate and EMG sensors is the traditional approach to Kinovea and Will balance board are integrated with OpenSim
evaluate a potential injury. However, these equipment are very to evaluate MMH related injuries.
expensive, requires technically skilled operators, challenging
to set up and transport, making these instruments difficult II. METHOD
to use to evaluate occupational injuries. Last few decades, The low cost biomechanical analysis system consists of
biomechanical human models have become very popular. The three steps: human motion analysis, GRF analysis and muscu-
most simplified biomechanical model to evaluate lifting related loskeletal analysis. To evaluate a potential MMH related injury,
injuries is two-dimensional (2D) skeleton human models [2], human kinematics, external forces like ground reaction forces
[3], [4], [5], [6]. But, for asymmetric lifting, these models are and a muscle force are necessary. We will get anthropometric
not effective as it misses the differences of kinetics on both data of human body and joint angle profile for a lifting motion
sides of the human body [6]. Three-dimensional (3D) skeleton from human motion analysis part. From GRF analysis step, we
models can capture the kinetic differences of both sides of will get the vertical GRF. Finally, from the musculoskeletal
the human body during lifting [7], [8], [9], [10]. Moreover analysis, we will get the muscle force information.

978-0-7381-3219-8/21/$31.00 ©2021 IEEE 128

A. Human Motion Analysis the maximum isometric force, lm is muscle length, vm is
0
For human motion analysis, we used a high-quality camera velocity, f (Fm , vm , lm ) is the muscle force-length-velocity
phone and Kinovea software. The traditional motion capture surface, rm,j is its moment arm about the j th joint axis, τj
system is the multi-camera system, which is expensive and is the generalized force acting about the j th joint axis.
difficult to setup. Kinovea is an open-source two-dimensional III. DEMONSTRATION OF THE SYSTEM
(2D) motion analysis software. There is a growing demand
for 2D motion analysis software in the biomechanics field, as The proposed low cost system requires only two hardware
it reduces complexity and computational cost. Kinovea allows and three open-source software. The hardware are smartphone
measuring the kinematics of the human body frame by frame. camera and Wii Balance Board. The open-source software are
It is easy to use and does not need any physical sensors. The Kinovea, Brainblox and OpenSim. Integration between those
accuracy of the Kinovea software is widely studied [18], [22] devices is shown in Figure 1 as a workflow diagram.
and used for different clinical uses [17], [18], [19], [20], [25].
Deviations in angle estimation between Kinovea and multi-
camera system are less than 5 degrees [25]. The motion video
taken from a smartphone will be analyzed in Kinovea. This
low cost motion capture system will give us access to generate
joint angle profiles during MMH related tasks. As this motion
capture system is portable, it is easy to implement in any
industrial establishment.
B. Ground reaction force analysis
Ground reaction force (GRF) is an essential parameter for
the biomechanical analysis of the human body. The gold
standard for measuring the GRF is the traditional force plate
or platform. However, these force plates are very costly,
challenging to transport and install. The Wii Balance Board
(Nintendo, Kyoto, Japan) is a part of popular video game
Fig. 1. Development of the biomechanical analysis system
WiiFit. The accuracy of the Wii balance board has been proven
in several studies [23], [25]. The sampling rate of the balance
board ranges from 30-50 Hz [24], [25]. We will use the Wii For demonstration purpose, we used a 5-lb (2.26 kg) dumbbell.
balance board to measure the GRF. A healthy volunteer (29 years old, 75 kg) was chosen for
C. Musculoskeletal analysis the demonstration. The participant signed a written a consent
form. The subject did not have any previous physical disorder
The biomechanical analysis of the lifting motion is per- and had an average physique of south Asian sub-continent. We
formed in OpenSim software. OpenSim is a widely popular requested the subject to stand on the Wii balance board and lift
open-source software for biomechanical modeling, simulation the weight from the floor to 1-m height. The subject repeated
and analysis. We will use the static optimization tool of the the lifting three times. The lifting motions were captured by
OpenSim software to find muscle activations and forces. a smartphone camera. The lifting motion video was analyzed
Static optimization is an extension of inverse dynamics. Joint in Kinovea (version 0.9.3) to calculate the joint-angles. The
torques can be found from joint kinematics and ground reac- frame rate of lifting motion at Kinovea is 25Hz. We also got
tion forces using inverse dynamics (Equation 1). the segmental body lengths of the subject at T-position from
M (q)q̈ + C(q, q̇) + G(q) = τ (1) the Kinovea. Figure 2(a) shows the subject with markers for
the weight lifting operation. During the lifting motion, the
Where, q̈R3 is the angular acceleration, M (q)R3x3 is the GRF was collected from the Wii Balance Board (Figure 3).
symmetric positive definite mass (inertia) matrix, C(q)R3x3 The communication was established between the Wii balance
is the centripetal and coriolis force matrix and G(q)R3 is board and a Windows 10 based laptop through Bluetooth.
the gravitational force vector. In static optimization, the joint BrainBlox [28] software was used to collect data from the
torque is distributed among the muscles to find the muscle Bluetooth port.
activations and forces. The muscle forces are optimized by The extracted kinematics and kinetics data were inserted in
minimizing muscle activations (Equation 2). the OpenSim 4.1 software. A full-body musculoskeletal model
minimize : J(am ) [29] was used for biomechanical analysis. The model has
n
X (2) 80 muscle-tendon unit and 17 ideal torque actuators. The
0
subject to : [am f (Fm , lm , vm )]rm,j = τj collected kinematics and kinetic data from Kinovea and Wii
m=1 balance board were inserted in the OpenSim software to calcu-
where n is the number of muscles in the model, am is the late muscle force. Figure 2 (b & c) shows the musculoskeletal
0
activation level of muscle m at a discrete time step, Fm is model in OpenSim. After that, the kinematics and kinetics data

129
were inserted in the OpenSim software to calculate muscle Figure 5 is around 450N, whereas it was reported 1125N in
force. [4]. That is normal, as the lifting weight for [4] was higher
than our experiment. Except that, the pattern of the GRF in
Figure 5 is similar to [4].
Using the static optimization tool in OpenSim, muscle forces
data were calculated for two lower limb muscles: gluteus
maximus and vastus intermedius (Figure 6). The maximum
force generated from Vastus intermedius (Figure 6(b)) was
1702N. The location of the peak muscle force for vastus
intermedius was at 40% of the lifting cycle, whereas in [16]
it was reported 10% of the lifting cycle. The lifting load
reported in [16] was for maximum weight. The difference
between peak points shows that different persons can adopt
different strategies for lifting lightweight and heavyweight.
The maximum muscle force for the glutei muscle was 1580N
at about 80% of the lifting time (Figure 6(a)). The pattern of
glutei muscle force is similar to the muscle force in [16].
V. D ISCUSSION AND CONCLUSION
This study aims to develop a low cost and portable module
to identify and assess MMH related injuries for developing
Fig. 2. (a) Subject with markers (sagittal plane) (b) musculoskeletal model countries. To do this, we develop a comprehensive biome-
frontal plane (c) musculoskeletal model (sagittal plane) chanical analysis system to evaluate MMH related injuries.
The joint angle profiles are collected using a smartphone
camera and Kinovea. The vertical GRF profile is collected
using Wii balance board and Brainblox software. Finally, the
joint angles and GRF are used as the input in the OpenSim
static optimization tool to find the muscle forces. The hardware
used here are affordable and portable. The software used here
are open-source and easy to use. The overall framework gives
us access from collecting joint angle profiles to muscle force
Fig. 3. Wii balance board profiles during lifting weight.
The joint angle profiles collected using smartphone camera
and Kinovea software are similar to literature [4] except
IV. RESULTS elbow joints. The GRF profile is also agreed well with the
We collected six joint angle profiles: elbow, shoulder, lum- literature [4]. The muscle force profiles from OpenSim static
bar spine, hip, knee, and ankle. The joint angle profiles are optimization tool are also similar to [16].
shown in Figure 4. We collected only the sagittal plane joint Although this low cost biomechanical analysis framework
angle profiles. Same joint angles and GRF were collected by shows some promising results, there are some limitations.
another group using a high precision multi-camera motion First, this system can read only symmetric lifting motion.
capture system and laboratory-grade force platform [4]. There To study asymmetric lifting motion, two cameras are re-
are some deviations for elbow joint angle profiles between quired. Moreover, lifting motions from both cameras need
Figure 4(a) and [4]. The trough of the elbow joint angle profile to be synchronized in Kinovea before extracting joint angle
is around 90% of the lifting cycle in Figure 4(a), whereas it profiles. Secondly, there are some potential inaccuracies when
is reported at 60% of the lifting cycle in [4]. The reason for synchronizing Kinovea and Wii balance board data. Our future
this deviation is the lifting weight. As the lifting weight in [4] plan is to develop a custom program to collect and process
was heavy, the subject tried to keep the load and elbow close joint angle and GRF data simultaneously using same platform.
to the body for balancing at the middle stage of the lifting This platform will reduce the synchronization error. Thirdly,
cycle. As a result, the trough of the elbow joint angle profile Wii balance board can collect only vertical GRF data. It can
was 60% of the lifting cycle. On the other hand, the lifting not study horizontal GRF data. However, the horizontal GRF
load for our subject was lighter than [4]. It was easy for the during lifting is insignificant. Fourth, only one subject was
subject to maintain balance without keeping the elbow close used to validate the framework. Although the accuracy of the
to the body at the early stage. The pattern and phase change Wii balance board, Kinovea and OpenSim has been studied
of the shoulder, lumbar spine, hip, knee and ankle joint angle separately in several studies, further research should examine
profiles in Figure 4 are similar to [4]. The total vertical GRF the accuracy of the overall system.
profile is shown in Figure 5. The maximum vertical GRF in The construction and automobile repair industries in develop-

130
Fig. 4. Joint angle profiles of (a) elbow (b) shoulder (c) lumbar spine (d) hip (e) knee and (f) ankle

Fig. 5. Total ground reaction forces

ing countries are mainly dependent on MMH related tasks.

Multi-camera, force plate and electromyography system are
not replaceable with this low cost platform for clinical use.
But, the accuracy of this low cost portable system is ade-
quate enough to evaluate MMH related injuries for different
Fig. 6. Lower limb muscle force for (a) gluteus maximus, (b) vastus
industrial workers in developing countries. In addition, this intermedius
framework can be used for low funded research institutions,
physiotherapists and sports injury analysis.

131
 R EFERENCES [22] A. Puig-Divı́, J. M. Padullés-Riu, A. Busquets-Faciaben, X. Padullés-
Chando, C. Escalona-Marfil, and D. Marcos-Ruiz, ‘Validity and re-
[1] J. N. Katz, ‘Lumbar disc disorders and low-back pain: socioeconomic liability of the kinovea program in obtaining angular and distance
factors and consequences’, J. Bone Joint Surg. Am., vol. 88 Suppl 2, dimensions’, 2017.
pp. 21–24, Apr. 2006, doi: 10.2106/JBJS.E.01273. [23] R. A. Clark, A. L. Bryant, Y. Pua, P. McCrory, K. Bennell, and M.
[2] J. Song, X. Qu, and C.-H. Chen, ‘Simulation of lifting motions using Hunt, ‘Validity and reliability of the Nintendo Wii Balance Board
a novel multi-objective optimization approach’, Int. J. Ind. Ergon., vol. for assessment of standing balance’, Gait Posture, vol. 31, no. 3, pp.
53, pp. 37–47, 2016. 307–310, 2010.
[3] T. R. Waters and A. Garg, ‘Two-dimensional biomechanical model [24] J. M. Leach, M. Mancini, R. J. Peterka, T. L. Hayes, and F. B. Horak,
for estimating strength of youth and adolescents for manual material ‘Validating and calibrating the Nintendo Wii balance board to derive
handling tasks’, Appl. Ergon., vol. 41, no. 1, pp. 1–7, Jan. 2010, doi: reliable center of pressure measures’, Sensors, vol. 14, no. 10, pp.
10.1016/j.apergo.2009.02.006. 18244–18267, 2014.
[4] Y. Xiang, J. Cruz, R. Zaman, and J. Yang, ‘Multi-objective optimization [25] Littrell, M. E., Chang, Y.-H., and Selgrade, B. P., 2018, Development
for two-dimensional maximum weight lifting prediction considering and assessment of a low-cost clinical gait analysis system. Journal of
dynamic strength’, Eng. Optim., p. (in press), 2020. applied biomechanics, 34, 503-508.
[5] Y. Xiang, R. Zaman, R. Rakshit, and J. Yang, ‘Subject-specific strength [26] Yang, C. X., 2018, Low-cost experimental system for center of mass and
percentile determination for two-dimensional symmetric lifting consid- center of pressure measurement (june 2018). IEEE Access, 6, 45021-
ering dynamic joint strength’, Multibody Syst. Dyn., vol. 46, no. 1, pp. 45033.
63–76, May 2019, doi: 10.1007/s11044-018-09661-1. [27] Chakravarty, K., Chatterjee, D., Das, R. K., Tripathy, S. R., and
[6] R. Zaman, Y. Xiang, J. Cruz, and J. Yang, ‘Two-dimensional versus Sinha, A., 2017, Analysis of muscle activation in lower extremity for
three-dimensional symmetric lifting motion prediction models: a case static balance. 2017 39th Annual International Conference of the IEEE
study’, ASME J. Comput. Inf. Sci. Eng., p. (accepted), 2020. Engineering in Medicine and Biology Society (EMBC), 4118-4122.
[7] Y. Xiang, J. S. Arora, and K. Abdel-Malek, ‘3d human lift- [28] Cooper, J., Siegfried, K., and Ahmed, A., 2014, Brainblox: Brain and
ing motion prediction with different performance measures’, Int. J. biomechanics lab in a box software. Version.
Humanoid Robot., vol. 09, no. 02, p. 1250012, Jun. 2012, doi: [29] Rajagopal, A., Dembia, C. L., DeMers, M. S., Delp, D. D., Hicks, J. L.,
10.1142/S0219843612500120. and Delp, S. L., 2016, Full-body musculoskeletal model for muscle-
[8] Y. Xiang, J. S. Arora, S. Rahmatalla, T. Marler, R. Bhatt, and K. Abdel- driven simulation of human gait. IEEE Transactions on Biomedical
Malek, ‘Human lifting simulation using a multi-objective optimization Engineering, 63, 2068-2079.
approach’, Multibody Syst. Dyn., vol. 23, no. 4, pp. 431–451, Apr. 2010,
doi: 10.1007/s11044-009-9186-y.
[9] R. Zaman, Y. Xiang, J. Cruz, and J. Yang, ‘Three-dimensional asym-
metric maximum weight lifting prediction considering dynamic joint
strength’, Proc. Inst. Mech. Eng. [H], p. (accepted), 2020.
[10] R. Zaman, Y. Xiang, J. Cruz, and J. Yang, ‘Three-dimensional symmetric
maximum weight lifting prediction’, St. Louis, MO, Aug. 2020, p. (in
press).
[11] S. L. Delp et al., ‘OpenSim: open-source software to create and analyze
dynamic simulations of movement’, IEEE Trans. Biomed. Eng., vol. 54,
no. 11, pp. 1940–1950, 2007.
[12] E. Beaucage-Gauvreau et al., ‘Validation of an OpenSim full-body
model with detailed lumbar spine for estimating lower lumbar spine
loads during symmetric and asymmetric lifting tasks’, Comput. Methods
Biomech. Biomed. Engin., vol. 22, no. 5, pp. 451–464, 2019.
[13] Y. Blache, F. Dal Maso, L. Desmoulins, A. Plamondon, and M. Begon,
‘Superficial shoulder muscle co-activations during lifting tasks: Influence
of lifting height, weight and phase’, J. Electromyogr. Kinesiol., vol. 25,
no. 2, pp. 355–362, 2015.
[14] Y. Blache, L. Desmoulins, P. Allard, A. Plamondon, and M. Begon,
‘Effects of height and load weight on shoulder muscle work during
overhead lifting task’, Ergonomics, vol. 58, no. 5, pp. 748–761, 2015.
[15] G. M. R. uz zaman Rana, ‘Maximum weight lifting prediction consid-
ering dynamic joint strength’, Thesis, 2018.
[16] R. Zaman, Y. Xiang, R. Rakshit, and J. Yang, ‘Muscle Force Prediction
in OpenSim Using Skeleton Motion Optimization Results As Input
Data’, in Volume 1: 39th Computers and Information in Engineering
Conference, Anaheim, California, USA, Aug. 2019, p. V001T02A046,
doi: 10.1115/DETC2019-97520.
[17] R. M. Abd El-Raheem, R. M. Kamel, and M. F. Ali, ‘Reliability of using
Kinovea program in measuring dominant wrist joint range of motion’,
Trends Appl. Sci. Res., vol. 10, no. 4, p. 224, 2015.
[18] Fernández-González, A. Koutsou, A. Cuesta-Gómez, M. Carratalá-
Tejada, J. C. Miangolarra-Page, and F. Molina-Rueda, ‘Reliability of
Kinovea® Software and Agreement with a Three-Dimensional Motion
System for Gait Analysis in Healthy Subjects’, Sensors, vol. 20, no. 11,
p. 3154, 2020.
[19] C. Guzmán-Valdivia, A. Blanco-Ortega, M. Oliver-Salazar, and J.
Carrera-Escobedo, ‘Therapeutic motion analysis of lower limbs using
Kinovea’, Int J Soft Comput Eng, vol. 3, no. 2, pp. 2231–307, 2013.
[20] N. A. H. Hisham, A. F. A. Nazri, J. Madete, L. Herawati, and J.
Mahmud, ‘Measuring ankle angle and analysis of walking gait using
kinovea’, 2017, pp. 247–250.
[21] Parida, R., and Ray, P. K., 2015, Biomechanical modelling of manual
material handling tasks: A comprehensive review. Procedia Manufactur-
ing, 3, 4598-4605.

132