Brain-Based Learning for Robots
Uploaded by
FrontiersBrain-Based Learning for Robots
Uploaded by
FrontiersBrain-Supported Learning Algorithms for Robots
Chairperson
Florian Rhrbein ([email protected])
Department of Informatics VI, Technische Universitt Mnchen
Boltzmannstr. 3, 85748 Garching, Germany
Discussants
Cecilia Laschi ([email protected])
The BioRobotics Institute, Scuola Superiore Sant Anna
Viale Rinaldo Piaggio, 34, 56026 Pontedera (PI), Italy
Florian Walter ([email protected])
Department of Informatics VI, Technische Universitt Mnchen
Boltzmannstr. 3, 85748 Garching, Germany
Speakers
Sander Bohte ([email protected])
CWI - Centrum Wiskunde & Informatica
Science Park 123, 1098 XG Amsterdam, Netherlands
Egidio Falotico ([email protected])
The BioRobotics Institute, Scuola Superiore Sant Anna
Viale Rinaldo Piaggio, 34, 56026 Pontedera (PI), Italy
Silvia Tolu ([email protected])
Technical University of Denmark, Department of Electrical Engineering
Elektrovej Building 326, 2800 Kgs. Lyngby, Denmark
Stefan Ulbrich ([email protected])
FZI - Forschungszentrum Informatik
Haid-und-Neu-Str. 10-14, 76131 Karlsruhe, Germany
Roboticists have early recognized the high potential of
neuro-biological control structures for robotic applications.
However, limited processing power and the lack of
appropriate models and tools shifted the focus of research
far away from biological neural networks. Today, combined
efforts in the fields of neurosciences, computer science and
many other areas in interdisciplinary research projects like
the Human Brain Project enable the simulation of spiking
biological neural networks with millions of neurons. The
computational power of these networks makes them a very
promising tool for the development for brain-controlled
neurorobots. Major challenges towards this goal include a
meaningful mapping between tasks and neural structures as
well as making the simulated brain exhibit the desired
behavior. The large size and the complexity of biological
neural networks make the development of learning
algorithms a huge challenge. A main prerequisite for the
implementation of learning algorithms for brain-controlled
robots is the availability of appropriate tools like, e.g., the
SpiNNaker board, which is able to simulate the neural
network in real-time. On the software side, these tools
should ease the development and support the researcher
during the evaluation by offering a toolchain for the
implementation and simulation of new algorithms (i.e.
simulators able to connect and synchronize simulated brainsupported algorithms and simulated robotic platforms).
This symposium presents actual brain-supported learning
techniques for robots as well as support tools for the
implementation of these algorithms. In particular, the papers
in this symposium provide evidence of the advantages of the
proposed brain-supported learning solutions and the
effectiveness of tools for the evaluation and implementation
of these algorithms.
11
Continuous-time neural reinforcement learning
working memory tasks
of
This talk will present latest results of bio-inspired learning
mechanisms integrated within anticipative control
architectures implemented on humanoid robots.
Sander Bohte
As living organisms, one of our primary characteristics is
the ability to rapidly process and react to unknown and
unexpected events. To this end, we are able to recognize an
event or a sequence of events and learn to respond properly.
Despite advances in machine learning, current cognitive
robotic systems are not able to rapidly and efficiently
respond in the real world: the challenge is to learn to
recognize both what is important, and also when to act.
Reinforcement Learning (RL) is typically used to solve
complex tasks: to learn the how. To respond quickly - to
learn when - the environment has to be sampled often
enough. For "enough", a programmer has to decide on the
step-size as a time-representation, choosing between a finegrained representation of time (many state-transitions;
difficult to learn with RL) or to a coarse temporal resolution
(easier to learn with RL but lacking precise timing). Here,
we derive a continuous-time version of on-policy SARSAlearning in a working-memory neural network model,
AuGMEnT. Using a neural working memory network
resolves the what problem, our when solution is built on
the notion that in the real world, instantaneous actions of a
certain duration are actually impossible. We demonstrate
how we can decouple action duration from the internal timesteps in the neural RL model using an action selection
system. The resultant CT-AuGMEnT successfully learns to
react to the events of a continuous-time task, without any
pre-imposed specifications about the duration of the events
or the delays between them.
Cerebellar internal models for a modular robot
Silvia Tolu
The problem to solve in controlling a dynamical system is to
find out the input to the system that will achieve the desired
behavior as output even under disturbances or changing
environments. The cerebellum acts in this sense because it
adapts its output in every condition by acquiring intrinsic
models through experience by a perceptual feedback that
allows the motor learning to proceed. Each internal model
(IM) is then instantiated based on what has been learned
about a specific motor control for a specific machine. Apart
from adaptation, another issue is the Central Nervous
System (CNS) capability of recalling the appropriate IM and
using it to make predictions during a movement. Therefore,
after training and adaptation the IM becomes encoded into a
long-term memory. Neurorobots have proved useful for
investigating motor control, and for designing robot
controllers as well. Furthermore, they can generate
hypotheses and test theories of brain functions. In this work,
we have designed a control system that can operate in an
unknown or changing environment, when the dynamical
robot model is unknown (e.g. a modular robot) inspired by
how the brain works. Furthermore a cerebellar model has
been developed with the aim of implementing model
extraction schemes for acquisition of knowledge (forward
and inverse IMs). A modular robot (Fable robot) benefits
from the organization and adaptivity of IMs that are
embedded into its control system architecture. Finally, we
have tested the adaptation of the IMs under a given task and
the robustness of the whole control system.
Bio-inspired learning mechanisms and anticipation in
humanoid robotics
Egidio Falotico
Sensorimotor Learning for Neural Robot Control based
on the Kinematic Bzier Maps and Spiking Neural
Networks
Nowadays, increasingly complex robots are being designed.
As the complexity of robots increases, traditional methods
for robotic control may become complex to handle. For this
reason, the use of neuro-controllers, controllers based on
biological learning mechanisms, have risen at a rapid pace.
This kind of controllers are especially useful in the field of
humanoid robotics, where it is common for the robot to
perform difficult tasks (i.e. visual tracking, gaze guided
locomotion) in a complex unstructured environment. In
order to perform these tasks, motor control cannot be based
on sensory feedback, which would be too slow. Indeed, in
humans, perceptual activity is not confined to the
interpretation of sensory information, but it anticipates the
consequences of action. Also in robotics, the anticipatory
control, generated thanks to internal models built by
experience, properly combined with reactive behaviours,
can greatly improve the effectiveness of perception-action
loops and the overall behaviour in real-world environments.
Stefan Ulbrich
The Kinematic Bzier Maps are a highly specialized model
representation of robot kinematics and dynamics with
related, optimal learning algorithms. By means of complex
basis transformations embedding prior knowledge, these
complex functions are transformed into a high-dimensional
space where they can be represented in a linear form and,
thus, efficiently be learned. In this work, we present our
ongoing research on how this model representation and
learning algorithms can be translated into a novel form
based on spiking neural networks exploiting the high degree
of parallelism in order to benefit from the increased
performance in robotic applications when applied on
neuromorphic hardware.
12