Control System of a Mobile Robot Manipulator

Tomáš Kot, Václav Krys, Vladimír Mostýn, Petr Novák

Department of Robotics
VŠB – Technical University of Ostrava
Ostrava – Poruba, Czech Republic
e-mail: [email protected]

Abstract—The paper presents a practical implementation of a The following chapters will present an example of a robotic
complex control system of a manipulator arm mounted on a manipulator with a control system developed on the
mobile robot, controlled remotely by a human operator. Department of robotics at VŠB-Technical university of
Described is the mechanical construction and parameters of the Ostrava. The system has been thoroughly tested in the recent
manipulator, the overall structure of the control system and then years and improved based on the practical experiences.
in more detail also several software modules which were designed
to aid the operator – including inverse kinematics and an anti-
collision system. The control system is universal and has already II. CONSTRUCTION OF THE MANIPULATOR
been applied to several types of mobile manipulators. The main requirements on a new testing manipulator
developed on the Department of robotics were:
Keywords—robot; manipulator; control system; operator;
• low costs (max. 300,000 CZK),
I. INTRODUCTION • minimal backlashes in joints,
A significant problem of service robotics is the absence of
• bearing capacity min. 1.5 kg in the whole working area,
standardised manipulators suitable for mobile robots.
Therefore, manipulators are designed for particular robotic • reach at least 800 mm in front of the chassis,
systems and are not easily transferable to different systems.
This is caused especially by incompatibilities between control • ability to pick objects from the ground in front and
systems of the robot and the manipulator – making them work alongside the robot,
together usually results in a complicated user interface with • low power consumption.
high demands on the operator. Incompatibilities in mechanical
and electrical interfaces are typically easier to solve. Initial conceptual designs showed that the ideal manipulator
for the given working area would be an arm with 6 degrees of
Recently there have been introduced some commercially freedom, similar to industrial robots; while the absolute
available manipulators for mobile systems, but their prices are minimum of degrees of freedom is 3.
very high and their application requires some compromises in
parameters of working area, bearing capacity and auxiliary Target parameters of drive units in individual joints were
functions, compared with a manipulator designed exactly for calculated using preliminary analysis. Because of the backlash
the given system [1 - 3]. limitation, only harmonic or cycloidal gearboxes were
acceptable. Harmonic gearboxes were chosen for the final
In spite of the huge progress in autonomous mobile robotics design, for their superior mechanical parameters (dimensions
in the recent years, there are still some fields where mobile and loading capacity). These gearboxes however are more
robots controlled remotely by a human operator are preferred expensive and after the price calculation of all components it
or even required. This includes for example the very important became evident that the manipulator will be limited to 3
applications related to safety of people during terrorist attacks, degrees of freedom and all the 3 joints will have to contain
fires and chemical accidents – more complicated tasks identical drive units because of an offered bulk discount.
requiring manipulation with some objects (defuse explosives,
transfer barrels or containers with dangerous chemical The strongest drive unit is required in the joint number 2,
substances etc.) require the decision-making abilities of trained but it is used also in other joints – which therefore are
human professionals and cannot be accomplished by overdesigned. Mass and dimension reduction caused by better
generalized algorithms. dimensioning of the two less-strained joints would not be
significant anyway, because compact disc DC motors exist
Control system of a teleoperated mobile robot can however only if few sizes.
assist the human operator and help him complete the task
successfully and quickly. The assistance can be either passive Fig. 1 shows the final 3D model of the manipulator and Fig.
(providing proper feedback in a way which is easy to read and 2 shows the actual working area of the robot, determined by an
understand) or active (aiding with manipulation tasks, analysis performed on the 3D model of the whole robot.
preventing collisions etc.).

258
978-1-4799-3528-4/14/$31.00 ©2014 IEEE
 Harmonic Drive AG. As already mentioned, three identical
units were used for all joints. Basic parameters are:
• weight of the unit: 2.85 kg,
• nominal output torque: 51 Nm,
• output angular velocity: 4.4 RPM,
• gearbox ratio: 160.

III. STRUCTURE OF THE CONTROL SYSTEM

The control system must secure the following essential
functions:
Fig. 1. 3D model of the Hercules manipulator arm.
• Communicate with the operator – read and process
commands from the operator (movement orders,
auxiliary functions), display feedback to the operator
(camera images, sensor data, robot status).
• Communicate with hardware devices – control action
elements (motors), read and process data from sensors.
The mobile robot Hercules is controlled remotely from an
operator station which is represented by a resistant suitcase
with an integrated touch-screen notebook, communication
devices and a gamepad. Communication between the operator
station and the robot is wireless on two independent lines:
868 MHz radio modem for driving and steering (one-way
communication) and 2.4 GHz Wi-Fi for all other functions of
the robot (bidirectional communication). The reason for this
separation is that the radio communication on lower frequency
offers better reliability and signal range, especially in indoor
environment, and so the chance that the operator will be at least
able to drive the robot back to him is greatly increased. The
Wi-Fi communication line provides better bandwidth, data
speed and lower latency, which is required for camera images
and precise arm manipulations.
Fig. 3 shows the overall structure of the control system and
how it is divided into two parts – server (robot) and client
(operator station). The robot contains a small embedded PC
with a server application that is responsible for camera images
acquisition and compression, sensor data reading and
processing and manipulator arm control (forward and inverse
kinematics, collision prevention, motor control).

Fig. 2. Working area analysis – side view and front view.

Basic technical parameters of the manipulator are:

• manipulator weight: 22.2 kg,
• gripped weight: 0.6 kg,
• maximal allowed payload (incl. gripper): 2 kg,
• maximal reach distance: 1,390 mm,
• nominal supply voltage: 24 V DC,
• degrees of freedom: 3.
The core component of the manipulator is a servo drive unit
consisting of a flat MAXON motor EC90F (60 W) with
encoder and a harmonic gearbox CSG-20-160-2UH by Fig. 3. Simplified structure of the control system.

2014 15th International Carpathian Control Conference (ICCC) 259

 There is also a custom-made circuit board with a
microprocessor which is responsible only for radio
communication and driving/steering control. The notebook in
the operator station runs a graphical client application that
displays data to the operator and accepts commands from him.
Communication between the two applications is done via the
UDP protocol on Wi-Fi.

IV. CONTROL MODES

The operator controls all core functions of the robot using a
single input device – gamepad with two analogue joysticks.
Gamepads are typically used for computer games and are very
ergonomic and easy to use, also in robotics. However, two
joysticks are not sufficient to control both driving and
manipulator so the operator must switch between these two
basic modes. This is not inconvenient, because moving with
the arm while also driving with the robot is difficult and
potentially dangerous, anyway. The current active mode is
clearly indicated on the screen and the joysticks have different
functions associated with them in both modes.
There are many possible ways how to control a manipulator
with 3 or more degrees of freedom using two analogue Fig. 5. Kinematic structure of the arm for IK solution.
joysticks with two axes each. The Hercules control system
provides two arm sub-modes – forward and inverse kinematics. The input position vector of the gripper [X3, Y3, Z3] at the
next calculation frame is computed from the actual position
A. Forward Kinematics vector and the velocity vector requested by the operator. This
new position vector is then transformed into vector of joint
With forward kinematics (FK), individual joysticks axes rotations (q1, q2, q3) by the following sequence of functions:
directly control the angular velocity of individual arm joints
and the control system just applies appropriate scaling. This
mode is convenient for larger rough movements of the arm and q1 = –atan2(Y3, –X3), (1)
also to look around the robot if a camera is mounted on a link
of the arm, which is the case of Hercules. p= X 3 2 + Y32 + ( Z 3  L1 ) ,
2
(2)
B. Inverse Kinematics
In the inverse kinematics (IK) sub-mode the operator § Z L · §  L 2 + L2 2 + p 2 ·
controls how the end-point of the arm (the gripper) should q2 = atan ¨ 3 1
¸  acos ¨ 3 ¸, (3)
¨ X 2 +Y 2 ¸ © 2 L2 p ¹
move in space either in Cartesian or cylindrical coordinates. © 3 3 ¹
The control system applies an IK algorithm on these input
velocities and calculates corresponding joint velocities. This §  p 2 + L2 2 + L32 ·
mode is convenient especially for manipulation with objects. q3 = S  acos ¨ ¸, (4)
© 2 L2 L3 ¹
The chosen IK algorithm is very simple but also very quick
and reliable and uses a closed-form expression – set of
trigonometric functions. This solution could be prone to where L1, L2, L3 are lengths of the arm links and p is an
singularity problems, but the Hercules arm kinematic structure temporary variable. After (2) it is necessary to check the value
does not have any dangerous singular configurations in the of p, which must not be greater than L2 + L3, otherwise the
working area into which the arm is restricted by the control desired target point is out of reach of the arm.
system.
V. FEEDBACK DISPLAY
The operator often controls a mobile robot which is away
from him and may even be completely out of his sight – in a
building, behind a corner etc. So the operator must rely only on
the information given him by the control system and the way
how the feedback is displayed to him is very crucial. The client
application of Hercules control system uses an advanced
graphical user interface rendered using Direct3D (Fig. 6).
Fig. 4. Two basic control modes of the manipulator arm (forward and inverse
kinematics).

2014 15th International Carpathian Control Conference (ICCC) 260

 This way the control system highlights near obstacles for
the operator and informs him about distances using illustrative
colour coding (gradient red-yellow-green) and by numerical
values drawn on detected edges. Moreover, it displays dashed
lines on the floor as a clear visual representation of driving
direction and width of the robot in conjunction with laser
scanner data, to signalize where is the robot going to drive and
whether the path is clear for it.

VI. ANTI-COLLISION SYSTEM

Another way how the control system can actively assist the
operator is by validating his commands before actually
executing them. Even with the advanced feedback display, the
operator still cannot be aware of everything and cannot pay
Fig. 6. Graphical user interface of the client application.
attention to every detail regarding the robot; he must focus on
the goal of his mission. The basic validation includes detection
The screen contains three viewports – the primary and prevention of collisions between individual moving parts
viewport which occupies most of the screen and two smaller of the robot (the manipulator arm) and other components of the
viewports on the right side (these two viewports can be robot (the chassis, antennas, sensors etc.). If the robot is
expanded on demand). The content of all viewports can be equipped with laser scanners or other similar sensors, also
chosen by the operator, it can be images from any robot camera potential collisions with the surrounding environment can be
(stereovision cameras on the arm, rear camera on the chassis, detected.
thermal camera) or an interactive 3D visualisation of the robot
status. A. Oriented Bounding Boxes
The right column also displays the most important data and For collision detection purposes, the Hercules control
control elements including mode selection and sensor values. system internally uses a simplified representation of the robot
All other information and controls are available on collapsible in the form of a set of Oriented Bounding Boxes (OBB) [5]
panels on the right side and some vital feedback is also enclosing all parts of the robot that possibly could be involved
represented via icons appearing directly in the main viewport. in a collision.

The 3D robot model represents the actual state of the robot, Intersection tests can be executed on a pair of OBBs. The
especially the real position of the arm. This is very helpful, theoretical number c of tests needed to check all possible
because in the case of robot Hercules with main cameras intersections between n boxes is given by binomial coefficient,
mounted directly on the arm and thus moving with the arm, the but it is obvious that all the tests are not necessary, because
operator can see only a part of the last arm link with the gripper some pairs can never collide and some pairs even must not be
and it appears to be stationary on the camera images. The only tested at all, because they are always intersecting. The number
way how to determine the current arm configuration is by of collision boxes for Hercules is n = 20, which gives the
looking at the 3D visualisation. maximum number of tests c = 190, but the actual number of
checked pairs is only c’ = 26.
The control system is created with modular approach and
one of the optional modules was created for 2D laser scanners.
Distance data from a scanner mounted on the chassis of robot
Hercules are used for automatic obstacle avoidance during
driving (active assistance), are visualised in 3D in the robot
model viewport and are also rendered in the main camera
viewport using augmented reality (Fig. 7).

Fig. 8. Simplified OBB representation of the robot.

Fig. 7. Using augmented reality to highlight obstacles.

2014 15th International Carpathian Control Conference (ICCC) 261

B. Separating Axis Theorem
Intersections between pairs of OBBs are checked using the
Separating Axis Theorem. The theorem says that for any two
convex bodies there exists a line onto which their projections
will not overlap if and only if the objects are not intersecting
[4, 6]. This line (together with all lines parallel to it) is called
the separating axis (SA).
To rule out intersection, at least one SA needs to be found.
But to prove intersection, it is necessary to verify that none of
all potential separating axes exists. For oriented bounding
boxes, there are only 15 potential separating axes [7 - 9]:
• 3 basic axes of the first OBB,
• 3 basic axes of the second OBB, Fig. 9. Separating axis s for oriented bounding boxes A and B.
• 9 cross-products of the previous axes (one axis from
each object). C. Implementation
Fig. 6 shows a 2D view (for simplicity) of two oriented Finding an existing intersection between two OBBs would
bounding boxes with centre points A and B, each with mean that the real arm already probably is in collision – which
corresponding half-extends (“radii”) ai and bi and basic axes is too late. Algorithm in the control system uses extrapolation
represented by unit vectors ai and bi, where i = 1, 2, 3. The of actual angular velocities of arm joints to predict position of
distance vector between centres of both boxes is: the joints after a chosen constant time and thus checks for
imminent collisions before they actually occur.
d=B–A (5) For every OBB pair, up to 15 separating axes needs to be
checked, but a big advantage of the separating axis theorem is
and its projection onto the separating axis defined by unit that all 15 axes need to be verified only very rarely. If the
vector s: boxes are not penetrating and a valid separating axis is found
for example in the second step, the remaining 13 steps can be
completely skipped.
r = |s  d| (6)
A practical test on the robot Hercules was executed with
The total radius of projection of a box onto the SA is a sum statistical data being written to a log file. During the test, the
of projections of all three radii: arm was navigated to various positions in the whole operating
range, simulating intensive use of the robot. Imminent collision
was detected in calculation 425 frames, which was 3 % of the
3 3
running time. The most important indicator is the number of
ra = ¦ ai ai ˜ s rb = ¦ bi bi ˜ s (7) separating axes checks – average number of separating axes
i =1 i =1
checked every frame for every OBB pair is 1.81 (out of the
maximal 15 possible separating axes for OBB–OBB test).
The projections are not overlapping (s really is a separating
axis) if r > ra + rb. Effectiveness of the algorithm is evident from Table 1,
which aggregates all individual OBB–OBB intersection tests.
For the first three tests where the basic axes of box A are
taken as potential separating axes, the calculations are
simplified to: TABLE I. STATISTICAL DATA OF COMPLEXITY OF INTERSECTION TESTS
Separating Axes Occurences
1 176281
s = aj (8)
2 79898
3 103174
raj = aj (9) 4 92
5 1412
where j = 1, 2, 3 is number of the test. Similar 6 760
simplification applies to rb during the next 3 tests. For the last 9 7 387
tests, potential separating axes are cross-products of basic axes: 8 28
9 37
10 247
si,j = ai × bj (10) 11 0
12 0
13 0
where i = 1, 2, 3 and j = 1, 2, 3. Equation (7) applies 14 0
without the simplification mentioned in (8) and (9). 15 425

2014 15th International Carpathian Control Conference (ICCC) 262

 The table shows total number of times when the separating
axis theorem finished after n axes (n = 1, 2, ... 15). The
maximum number of axes ever tested in case of no collisions
was 10. The 425 occurrences of all 15 SA tests correspond to
the 425 frames with collisions, because collision can be proven
only after going through all 15 SAs. Another interesting fact is
that 99.1 % of all OBB–OBB tests finished after checking only
up to 3 separating axes and 99.7 % during the first six checks.
This is very good for performance, because the first 6 possible
separating axes are faster to calculate, see equations (8) and
(9). Fig. 10. Mobile robot Hercules and mobile robot Hardy

VII. CONCLUSION Practical tests also gave us a lot of feedback about the ideal
The control system introduced above was first implemented placement of individual controls on the screen and about their
on mobile robot Hercules in 2009 and has been improved since assignment to individual control elements of gamepad.
then based on practical experiences from thorough practical As far as the mechanical design of the Hercules arm is
testing in various simulated missions. In 2011 a slightly concerned, the biggest drawback turned out to be the low
modified version was also implemented on Hardy, a heavy- number of degrees of freedom. It is possible to properly
duty tracked mobile robot with a 5-degrees-of-freedom manipulate only with some types of objects in limited
manipulator arm, 2 additional degrees of freedom in the gripper locations. Each additional degree of freedom greatly improves
and many auxiliary functions including an integrated water jet abilities of the arm, but also significantly complicates the
for fire extinguishing (Fig. 10). Alternations of the control mechanical construction and almost doubles the costs.
system are also in use on other, simpler, mobile robots
developed by the Department of Robotics.
ACKNOWLEDGMENT
The first discovered problem with teleoperated mobile This article has been elaborated in the framework of the
robots was that it is very hard to quickly reckon the real width project Opportunity for young researchers, reg. no.
of the robot when driving in confined areas, without having to CZ.1.07/2.3.00/30.0016, supported by Operational Programme
stop and look around using a rotable camera head (if such a Education for Competitiveness and co-financed by the
device is present). Much better situation is if the operator can European Social Fund and the state budget of the Czech
see the whole front side of the robot on the camera picture – so Republic.
the camera should be at the rear top section of the robot. It is
however often not practical or not possible to mount the
camera this way. The current version of the control system uses REFERENCES
augmented reality to visualise the oncoming driving path (the [1] Schunk. “SCHUNK Mobile Greifsysteme” [online]. Available from
width of the robot and the direction, including rounding during WWW: <http://mobile.schunk-microsite.com/en/produkte/produkte.
html>.
turns) straight into the camera image. This increases confidence
[2] Oceaneering. “TERABOT®” [online]. Available from WWW:
of the operator and allows him to focus on something more <http://www.oceaneering.com/advanced-technologies/space-
important. systems/robotics-and-automation/terabot/>.
Another problem is the already mentioned impossibility to [3] Kinova Robotics. “Products Research. Robotic arm for research,
rehabilitation,” [online]. Available from WWW:
see the manipulator arm on the camera and thus to estimate <http://kinovarobotics.com/products/jaco-research-edition/>.
how much it is possible to still move it. If the camera cannot be [4] C. Ericson, “Real-time collision detection,” Morgan Kaufmann
mounted on the chassis to show the arm in all situations (which Publishers, 2005. 632 p. ISBN 978-1558607323.
is the case of Hercules), a real-time rendered 3D model turned [5] Wikipedia. “Bounding volume” [online]. Available from WWW:
out to be a great aid. <http://en.wikipedia.org/wiki/Bounding_volume>.
[6] Wikipedia. “Separating axis theorem” [online]. Available from WWW:
Even with the 3D model clearly visible on the screen, the <http://en.wikipedia.org/wiki/Separating_axis_theorem>.
operator often concentrates strictly on the object of
[7] M. Gomez, “Simple intersection tests for games” [online]. Available
manipulation and occasionally accidentally tries to move the from WWW: <http://www.gamasutra.com/view/feature/3383/simple_
arm into a collision with the robot chassis, typically if intersection_tests_for_games.php>.
manipulating at the ground near the robot. This is why the anti- [8] D. H. Eberly, “3D game engine design: a practical approach to real-time
collision system was implemented. computer graphics,” Morgan Kaufmann Publishers, 2000. 561 p. ISBN
978-1558605930.
[9] Konecny, P. “Bounding volumes in computer graphics,” master thesis,
Brno: Masaryk University, 1998. 40 p.
[10] L.-P. Ellekilde, H. I.Christensen, “Control of Mobile Manipulator using
the Dynamical Systems Approach,” In IEEE International Conference
on Robotics and Automation. 2009. ISBN 1370-1376.

2014 15th International Carpathian Control Conference (ICCC) 263