2016 Knowledge Creation and Intelligent Computing (KCIC)

The Bionic Hand Movement Using Myo Sensor and

Neural Networks
Mahfud Ridho Pambudi, Riyanto Sigit, Tri Harsono
Computer Engineering
Electronic Engineering Polytechnic Institute of Surabaya
Surabaya, Indonesia
Email: [email protected], [email protected], [email protected]

Abstract— One part of human body in which very useful in movement on the human hand [1] [5]. Data is read by EMG
this life is a hand. This life will be disrupted if one or both of these then sent via Bluetooth to a Personal Computer (PC) to be
hands do not work, due to disability or amputation. One of the processed and then sent to the bionic arm robot which was
contributions that help this condition is the bionic hand. This study equipped with a special microcontroller. All functions in Myo
have developed a bionic hand that consist of six servo motors
to drive the fingers movement. Myo sensor is used in this bionic
sensor is processed by ARM Cortex M4. Meanwhile the
hand, it is located in the forearm. It has been used to record the processor of the arm robot is one of ATMega series. The data
muscle signals. Feature extraction of the muscle signal was received will be processed, then ordered to drive the six servo
formulated by using root mean square (rms). Kind of the bionic that is embedded in the arm robot mechanics. All of the above
hand movement was found by identifying the movement signal processes will produce a new function that is a similar to real
pattern using back propagation through the artificial neural human hands.
network. The output data obtained from back propagation was Study on the Myo sensor to move the hand robot has
processed by microcontroller and transmitted to the servo motors been done in some previous researchers [1], but there are still
for getting a movement of the bionic hand. Based on experimental many shortcomings regarding the method of introduction of
data, five kinds of movement were tested in this system,
average of recognition for each movement: handful accuracy
the signal. The previous research using binary control
movement 65%; pointing-fingers 75%; wave in 100%; wave out algoritm method for classifying signals, meanwhile
75%; and spread 100%. Testing for the whole movements (five this research using artifical neural network is back
kinds) were performed with the average error 17%. propagation as a method for classifying signals.
Based on this condition, many researchers [2], [3] have
Keywords— bionic hand, myo sensor, root mean square, back been trying the other method for identifying the signal
propagation, microcontroller, servo motor. including in this study, namely back propagation through the
artificial neural network to produce the better output.
I. INTRODUCTION
II. PREVIOUS WORKS
Along with the development of robotics technology in
Previous researchers have shown some results and
the field of robot, a lot of researches have done to help people
developments that can be used as references for this study.
in everyday life. One of these studies are still currently being
Related studies that have been carried out as follows:
developed is the bionic robot. Bionic robot is a robot that is
The research conducted by Cahyo Setiawan suggested the
implanted in humans to replace the function and role of
use of spring on finger parts of the robot make it more difficult
missing human body parts such as the hands or feet. The
to do a clenching movement. Because when the spring is pulled,
human body parts are very important for people to perform
then there will be a repulsive force of the spring [1]. Changing
daily activities one of which is a hand. Losing a hand due to
the position of Myo sensor that is placed on the arm will result
an accident or paralysis will cause men to have difficulty in
in changes to the data [6]. The only movements that can be used
carrying out their daily activities. For example, a person who
are the ones with unique data compared to the others.
lost his hand, will be difficult to perform tasks related to the
Movements with similar data cannot be used as a reference to
function of the human hand. Researchers are trying to develop
the input of the hand robot.
a bionic arm robot to solve the aforementioned problems. The
Research conducted by Iqbal Maulana Malik suggested that
bionic hand will be controlled via myoelectric signals
back propagation network can be used to classify 6 movements
generated by human muscle. Basically, the human body has
that will be used to control the slide. Back propagation network
electricity, including one containing the muscles. Although
that is most ideal for the system is the one using hidden nodes
muscle has electrical signals, but the strength of the resulting
twice as much as the input nodes [2]. More natural and flexible
voltage is very small. A small electric can be catched up using
interactions have been achieved since the availability of user
a set of electronic devices (sensors). In this study, the Myo
who can control slide from a distance with a more complex
sensor limited only to the EMG (electromyogram) sensor has
features such as next, previous, zoom in and zoom out
been used to get the EMG signals generated by muscle

978-1-5050-5231-8/16/$31.00 @2016 IEEE 259

commands, dragging the slide while zooming, giving in the time domain because if using preprocessing like Fourier
highlights, and free drawing. Calibration to the user can be done transform in order to produce a better signal feature turn out
by recording every movement pattern. happened is a slow response to the performance of the system
According to research by Suresh M., Krishnamohan P. G. should be running close to the real time.
and Mallikarjun S. Holi, the EMG signal is the sum of the While there is no movement, human muscles still emit
disposal of all motor units in the range of pick-up electrodes. signals recorded by EMG sensor, it is evident from the
Nervous system always controls muscle activity (contraction/ persistence of the value of the amplitude of the electrical
relaxation) [3]. Therefore, the EMG signal is a complex signal, muscle, although very small. In this study, the signals are called
which is controlled by the nervous system and depends on the noise, so it needs a way to distinguish the active signal and
nature of the anatomy and physiology of muscle. noise. Active signal is obtained from an average of idle data
According to the study by Nomiyasari, EMG is a bioelectric which then calculated to find the value of the standard deviation
recording instrumentation used to determine the signal caused to produce a maximum value of idle data. If the data is larger
by skeletal muscle activities. This muscle is one of human than the maximum idle data, then the data is an active signal.
organs that move the skeleton [4]. Skeletal muscle has the Value of standard deviation and maximum idle data can be
nature of conscious, unconscious, and irregular because these calculated using equation 1 and 2.
activities depend on the will of the user. In general, the working
principle of the skeletal muscle is relatively similar to the heart ∑ ( )
SD = (1)
muscle, with the difference is the origin of the stimulus. The
motion of the muscle does not possess automaticity. Stimulus
originates in the brain and transmitted through nerves. To Where:
determine the EMG signal, electrode is placed as a medium of SD = standard deviation
interaction. N = sample data
X = raw idle data
III. METHODOLOGY µ = average of idle data

Design of the whole system for this research is shown in Max idle data =X idle data + SD (2)
Fig. 1. idle data)

Where:
Max = max value of idle data
X = average of idle data
SD = standard deviation

B. Feature Exraction
Feature extraction needs to be done in order to get the
features of input signal each movement [2] [7]. In this research,
researchers implement RMS method to obtain signal features
each movement. RMS (root mean square) is "squared and then
averaged, then squared root) [2]. RMS is carried out to identify
Fig. 1. Diagram of the system design the characteristic features of each movement. In this study the
characteristics used to identify each movement are the RMS
In this study, the first thing to do is to get the signals from value of each EMG channel. The RMS value of each channel
muscle contractions in human hands using Myo sensor. The can be calculated using equation 3.
placement of Myo sensor should not be arbitrary because it has
eight electrodes that will be used to record the activity of several
muscles located on the human forearm. These sensors record ∑
= (3)
data from muscle contraction, and then the data is processed on
a PC to extract the features that will be established as patterns
and be classified by method of back propagation. From this Where:
classification, it will generate the type of movement that is RMS = value of RMS (root mean square)
already recognizable from the input signal, then take a decision Xn = nth data
and sent it to the bionic hand robot to perform the same N = the number of signal
movement with user's hands. If RMS value of each channel has been found, then the next
step is to find the minimum and maximum values of the RMS
A. Signal Processing value. The minimum and maximum values are used to
In this research tested the input signal to the system is still normalize the data to all channels with the range of value
in a short times (50ms) per data retrieval. The signals are still

260
between 0 and 1. The normalized data can be calculated using structure which can be used to classify 5 movements. The
equation 4. architectural design for the back propagation of this system is
shown in Fig. 3.
−
= (4)
−

Where :
Dn = data normalized
X = RMS value
dataMin = smallest data
dataMax = biggest data

Calculation of equations 3 and 4 will produce RMS values

that have been normalized for each channel. These data will
form a pattern that will be used as input for back propagation.
Example of movement pattern is shown in Fig. 2.

Fig. 3. Structure back propagation network

RMS Value

Learning Forward

Layer 1:
Each input node (xi, i = 1,..., n) receives signals (xi) and
Channel deliver the signal to all hidden layer node,

= (5)
Fig. 2. Example of RMS data pattern that has been normalized

Layer 2:
C. Backpropagation
= , (6)
Back propagation artificial neural network is one model of
the popular network on artificial neural networks. This network 1
model many used to be applied in the resolution of a problem = ( ) (7)
1+
related to the identification, prediction, pattern recognition, Layer 3:
classification and so forth [6]. In practice repeatedly, this
algorithm will produce a better performance. This means that = (8)
,
the "weight of interconnection" ANN getting closer to weights
that should be. Another advantage of this back propagation
1
artificial neural networks is its ability to learn (adaptive) and = (9)
1+ ( )
immune to error (Fault Tolerance) with that advantage, the
artificial neural networks can create a system that will withstand
damage (robust) and consistently worked well. Thus, in this Calculations of the average error of learning patterns were
research only using back propagation artificial neural networks performed using the MSE (Mean Square Error) function.
method as a method for pattern recognition and classification.
The method used in this research to recognize signal 1
( )= (10)
patterns is back propagation. Back propagation is one of a 2
artificial neural network with supervised learning. Back
propagation has three layers, namely input layer, hidden layer Learning Backward
and output layer. In this research, back propagation method is
selected because it has forward and backward learning so it is Error output Layer 2
more accurate to recognize patterns of signals and classify them
= =
to be a type of movement. Input from back propagation on this
signal is normalized RMS data of each channel. So the network (11)
architecture consists of eight input nodes, 24 hidden nodes and =
5 output nodes. The resulting output is sent to a microcontroller
on the bionic hand robot. It is expected to create network

261
 for more but still there is an error is nearly equal to the number
= = , of hidden 8. Meanwhile, at the time of the training process on
(12)
the number of hidden layer 24 obtained the number of iteration
= = 1− (13) is quite a lot compared hidden 8 and 16 but the resulting error
value is small.
Updates weight L2-L3 (Hidden to Output) Therefore, based on the testing that was done on a number
of hidden used this system requires a high degree of accuracy
∆ , = . = . . so that the researchers chose to use the number 24 because it
, , (14)
has a high hidden small error.
= .
= + ∆ , (15) TABLE I. ARCHITECTURE DESIGN TESTING

Learning Error Number of

Updates weight L1-L2 (Input to Hidden) Hidden Error
Rate Tolerance Iteration
∆ , = . = . . 8 0.6 0.00001 184885 0.000017
, , (16)
= . 16 0.6 0.00001 218852 0.000016
= + ∆ , (17) 24 0.6 0.00001 262359 0.000011

Learning rate updated with In this system design, layer one of back propagation
_ = ( )= (18) network has eight nodes. Each node contains normalized RMS
values of each EMG channel. Normalized back propagation
Information: input values are in the range of 0 to 1. Value range of 0 and 1 is
Xi : Input signal. chosen to make it easier to perform computation on computers.
: Output layer 1. Layer two on back propagation network has eight nodes,
: Addition obtained from the experiment. Table 1 shows the back
, : Weight between layer 1 and layer 2 propagation input of this system.
: Output layer 2. TABLE II. INPUT BACK PROPAGATION
: Addition
, : Weight between layer 2 and layer 3 No Node Layer 1 Input
: Output layer 3. 1 Node 1 Normalization RMS Channel 1 EMG
( ) : Error output layer 3. 2 Node 2 Normalization RMS Channel 2 EMG
: Target. 3 Node 3 Normalization RMS Channel 3 EMG
: Error output layer 2. 4 Node 4 Normalization RMS Channel 4 EMG
∆ , : Delta weight layer 2 and layer 3. 5 Node 5 Normalization RMS Channel 5 EMG
∆ : Delta weight layer 1 and layer 2 6 Node 6 Normalization RMS Channel 6 EMG
,
: learning rate. 7 Node 7 Normalization RMS Channel 7 EMG
K0 : initial time constant 8 Node 8 Normalization RMS Channel 8 EMG
K : constant time.
Output of back propagation is used to classify the five
In this research, the training data used for learning at the movements including clenching, pointing, spreading, wave in
back propagation method is still being done with a trials phase and wave out. Table 2 shows the initialization of the target value
just one user by taking 10 times data of each movement. of the desired output for each movement.
Because only one user, this research has not been conducted
TABLE III. INITIALIZATION TARGET OUTPUT BACK PROPAGATION
validation of a system that is built for example validated with
the data set. Then the data used to train the system as much as Out Out Out Out Out
No Movement
50 data for each movement were taken each 10 times the data. 1 2 3 4 5
To get the most excellent architectural design, the 1 clenching 1 0 0 0 0
researchers tested the value of the parameter value by changing
the number of hidden layer parameter that must be equated, 2 wave in 0 1 0 0 0
namely learning rate and error tollerance. 3 pointing 0 0 1 0 0
In this research pilloted by changing the parameters of 4 wave out 0 0 0 1 0
hidden layer, with learning rate by 0,6 and error tollerance is
0,00001. At the time of the training process in a number of 5 spreading 0 0 0 0 1
hidden layer 8 obtained the number of iterations is small but
there is an error that is high enough. At the time the training From the results of training with the learning rate of
process on a number of hidden 16 found the number iterations 0.00001, is obtained the weight of input to hidden and the

262
 weight of hidden to output. The weight values of input to hidden b. Clenching Movement
and hidden to output will be used to classify 5 output From Fig. 5, it shows a demonstration of clenching
movement. Table 3 shows the input pattern for each movement. movement. When a user's hand is clenching, then the robot will
mimic its movement. In this case, when the user's hand is
clenching, the robot is also clenching.
TABLE IV. INPUT PATTERN EVERY MOVE

Move the input pattern

Movement
Ch1 Ch2 Ch3 Ch4 Ch5 Ch6 Ch7 Ch8
clenching 0.43 1 0.33 0.50 0.77 0.92 0.73 0.28
wave in 0.22 0.12 0.33 0.48 0.29 0.45 0.51 0.88
pointing 0.25 0.44 0.41 0.29 0.48 0.66 0.64 0.49
wave out 0.56 0.93 0.79 0.63 0.15 0.04 0.08 0.11
spreading 0.10 0.40 0.67 0.53 0.18 0.64 0.21 0.15

Table 3 shows the feature extraction pattern for each

channel to be used as input back propagation. The value of each
EMG channel is used as input of back propagation, and then Fig. 5. Robotic arm for movement clenching
mapped using weight value of input to hidden and hidden to
output that have been obtained at the time of training. Mapping c. Pointing Movement
results are shown in Table 4. From Fig. 6, it shows a demonstration of pointing
movement. When a user's hand is pointing, the robot will
TABLE V. RESULT MAPPING OF INPUT TO OUTPUT duplicate its movement. In this case, when the user's hand is
pointing, the robot is also pointing.
Movement Out1 Out2 Out3 Out4 Out5
clenching 0.96 0.00 0.00 0.00 0.00
wave in 0.00 0.99 0.00 0.00 0.00
pointing 0.00 0.00 0.92 0.00 0.00
wave out 0.00 0.00 0.00 0.90 0.29
spreading 0.00 0.02 0.00 0.00 0.99

IV. RESULT
Testing phase of bionic arm robot movement is the end
result of the aforementioned steps. At this test, the bionic arm
robot is already integrated with a PC that is used to process the
signals that have been recognized by back propagation. At this
stage, the arm robot receives data that has been processed on Fig. 6. Robotic arm for movement appoint
the PC and recognized by back propagation as a desired
movement pattern. d. Wave In Movement
From Fig. 7, it shows a demonstration of wave in motion.
a. Relaxing Movement When a user's hand is doing a wave in, then the robot will also
From Fig. 4, it shows a demonstration of relaxing be doing the same movement. In this case, when the user’s hand
movement. When the user's hand is not moving or in a state of is doing wave in, the robot is also doing wave in.
relaxation, the robot will perform relaxing motion.

Fig. 4. Robotic arm for movement relax Fig. 7. Robotic arm for movement wave in

263
e. Wave Out Movement V. CONCLUSIONS
From Fig. 8, it shows a demonstration of wave out motion. From the results of the RMS of each movement
When a user's hand is doing a wave out, then the robot will also respectively, it is obtained quite different patterns. These
be doing the same movement. In this case, when the user’s hand patterns distinguish between each movement and will be used
is doing wave out, the robot is also doing wave out. as input of back propagation and classification. The test results
showed that the accuracy of the pattern recognition of clenching
is about 65%, the pattern recognition of pointing is about 75%,
the pattern recognition of wave in is about 100%, the pattern
recognition of wave out is about 75%, and the pattern
recognition of spreading is about 100%. Overall percentage of
the obtained error is about 17%.

VI. FUTURE WORK

In this research, there are still some weakness. Therefore,
for future researchers will do several things:
Fig. 8. Robotic arm for movement wave out 1. Will validate the data that used to system using a data set.
2. Will use a time window based on getting nicer features.
f. Spreading Movement
From Fig. 9, it shows a demonstration of spreading
ACNOWLEDGMENT
movement. When a user's hand is spreading, then the robot will
mimic its movement. In this case, when the user's hand spread, This research will not work without the helps from some
then the arm robot will also spread. peoples at Computer Engineering Study Program in Electronic
Engineering Polytechnic Institute of Surabaya where this
research has been held. I would like to give thanks to my
advisor, Riyanto Sigit and Tri Harsono whose been helping me
which so much helpful advices on this research. Also thank you
very much for my friend in Computer Engineering 2012 who
helped me write the paper on this research.

REFERENCES
[1] Setiawan, Cahyo, “Tangan Robot Bionik dengan Kendali Otot Berbasis
EMG”, Computer Engineering, Electronic Engineering Polytechnic
Institute of Surabaya, Surabaya, Indonesia, 2015.
[2] Malik, Iqbal Maulana, “Sistem Pengenalan Pola Gerak Tangan Untuk
Persentase Interaktif Dengan Sensor EMG (Electromyogram)”, Computer
Engineering, Electronic Engineering Polytechnic Institute of Surabaya,
Fig. 9. Robotic arm for movement spreading Surabaya, Indonesia, 2015.
[3] Suresh.M, et al, “Electromyography Analysis for Person Identification”,
Kalpataru Institute of Technology, India, 2011.
From the 20 tests on each movement, the percentage of [4] Nomiyasari, “Perancangan dan Pembuatan Modul ECG dan EMG dalam
satu unit PC”, Electronic Engineering, Electronic Engineering
obtained errors is shown in Table 5. Polytechnic Institute of Surabaya,Surabaya, Indonesia, 2011.
[5] B. Peerdeman, D. Boere, H. Witteveen, R.H. in t Veld, H. Hermens, S.
TABLE VI. RESULT 20 TEST ON EACH MOVEMENT
Stramigioli, et al., "Myoelectric forearm prostheses: State of the art from
a user-centered perspective", JRRD, vol. 48, no. 6, pp. 719-738, 2011.
Amount Error
Movement Succeed Failed [6] A.H. Al-Timemy, G. Bugmann, J. Escudero and N. Outram,
Trial (%) "Classification of finger movements for the dexterous hand prosthesis
Clenching 20 times 13 times 6 times 35% control with surface electromyography", IEEE J. Biomed. Health
Informatics, vol. 17, no. 3, pp. 608-618, 2013.
Pointing 20 times 15 times 5 times 25% [7] C. Miller, Real-Time Feature Extraction and Classification of Prehensile
Spreading 20 times 20 times 0 times 0% EMG Signals, 2008

Wave in 20 times 20 times 0 times 0%

Wave out 20 times 15 times 5 times 25%

264