Systèmes robotisés intelligents
Smart Robotic Systems
Sensors and Variables Estimation
Gilles TAGNE
[email protected]
1 1
Sensors and variables estimation
Summary:
1. Sensors for mobile robots
2. Variables estimation
3. Multi-sensor fusion
2 2
Sensors and variables estimation
Purpose:
Usual problem in robotics:
• Where am I? // In which configuration am I?
• Where should I go? // What configuration
should I reach?
• How to get there?
➔ We need sensors to get data and we need
some algorithms to use them!
3 3
Sensors for mobile robots
We have two class of sensors:
• Proprioceptive sensors: provide information on the
internal state of the robot
➔ Example: get the relative position of the robot.
• Exteroceptive sensors: provide information on the state
of the environment
➔ Example: get the absolute position of the robot.
Sensors can be:
• Passive / Active
• Analog / Digital
• Logic / Smart
4 4
Sensors for mobile robots
Classification example:
5 5
Sensors for mobile robots
Sensors for position measurement:
▪ Odometry sensors:
Use encoders to measure the rotation of the wheels and/or
steering angles.
Pros: proprioceptive sensor → always able to give an
estimation of the position.
Cons: accumulation of error.
▪ Orientation sensors:
Use gyroscopes or accelerometers to measure speed of rotation
or accelerations.
Pros: proprioceptive sensor.
Cons: drift over time (error integration)
6 6
Sensors for mobile robots
Odometry sensors
Measurement of wheel speed rotation
➔ Estimation of the robot's displacement
Most common daily use: dead-reckoning (estimation of the
position of your car in a tunnel into which you can’t receive GPS)
Two types of odometry sensors:
• Relative optical encoder
• Absolute optical encoder
7 7
Sensors for mobile robots
Optical encoders
How does it work?: a ray of light aim at a
photosensor and is periodically interrupted by a
perforated disc that turn with the shaft of the motor.
➔ Measuring the position consists in
counting the number of impulses.
(example: step motor)
8 8
Sensors for mobile robots
Relative optical encoders
Those encoders usually measure the speed of rotation from which we can
easily retrieve the position.
Same principle than the previous example but with two engraved tracks
shifted by 1/4th of the period → allow to get the direction of rotation and to
improve the resolution therefore the precision. (2 tracks = resolution x4)
9 9
Sensors for mobile robots
Relative optical encoders
10 10
Sensors for mobile robots
Relative optical encoders
A compromise has to be made between:
- low resolution → high noise especially at low speed
- high resolution → need high frequency counting electronics
Pros:
- Low cost (but price increase with the resolution)
- Few cables (usually 5)
- Small size
Cons:
- No absolute position → need a recalibration procedure!
11 11
Sensors for mobile robots
Absolute optical encoders
Same use of light through a wheel, but with an engraved binary code.
Each position of the axis is a number on Nbits ➔ absolute position!
12 12
Sensors for mobile robots
Absolute optical encoders
Be careful about which kind of encoder wheel you have!
13 13
Sensors for mobile robots
Absolute optical encoders
Be careful about which kind of encoder wheel you have!
« Natural » binary VS Gray code
14 14
Sensors for mobile robots
Absolute optical encoders
Each position is on Nbits ➔ Sensor resolution = 2N
Each new track on the wheel ➔ Resolution x2 but price ≈x4 !
Pros:
- Absolute position
Cons:
- More expensive (price increasing with resolution)
- Slightly bigger component
- More cables (one for each track + power)
15 15
Sensors for mobile robots
Resume: Optical encoders for odometry
Pros:
- Low cost
- Easy to integrate
Cons:
- We never know exactly the diameter of the wheel (tire
pressure, ground condition…)
- Wheel drift on accelerations/brakings
➔ Cumulative error on the position estimation!
➔ Need the support of complementary sensors to
correct the localization estimation of a mobile robot.
16 16
Sensors for mobile robots
Resume: Optical encoders for odometry
17 17
Sensors for mobile robots
Resume: Optical encoders for odometry
18 18
Sensors for mobile robots
Inertial Measurement Unit (IMU / Compass)
➔ Widely used to measure accelerations and orientation
(phones, GPS, VR headsets…)
19 19
Sensors for mobile robots
Inertial Measurement Unit (IMU / Compass)
Typically, one sensor per type needed on each axis:
● 6DOF = 3-axis accelerometer + 3-axis gyroscope
● 9DOF = 6DOF + 3-axis magnetometer
● 10DOF = 9DOF + barometric pressure sensor (altitude)
➔Additionnal feature: thermometer for self-calibration.
➔Need to be placed as close as possible to the gravity
center of the robot or its position relatively to this point
must be known precisely for the conception (and use) of
the robot model.
20 20
Sensors for mobile robots
Inertial Measurement Unit (IMU / Compass)
Gyroscopes:
➔Gives angular velocity in degrees/s or rad/s.
➔Has constant bias which is affected by temperature.
➔Bias changes over time (bias stability).
21 21
Sensors for mobile robots
Inertial Measurement Unit (IMU / Compass)
Accelerometer:
➔Gives acceleration in m/s2 or g.
➔At rest an accelerometer measures the gravity vector pointing up.
➔Accurate long term (no drift) but not short term (noise).
22 22
Sensors for mobile robots
Inertial Measurement Unit (IMU / Compass)
Gyroscope:
➔Measures magnetic field strength on each axis in Gauss (unit
of magnetic flux) or µT (unit of magnetic field strength).
➔Points generally towards magnetic north.
➔Can be distorted by nearby metals or electronics.
23 23
Sensors for mobile robots
Inertial Measurement Unit (IMU / Compass)
24 24
Sensors for mobile robots
Telemetry:
Principle: To measure the time of flight to get the distance.
Transmitter and receiver are close so we measure the echo.
Can be done using:
- Sound waves (ultrasound)
- Electromagnetic waves (radio)
- Light (mostly infrared)
25 25
Sensors for mobile robots
Sonar / Ultrasonic sensors:
Proximity and distance measurement
Sound waves that reflect on the obstacles Advantages: Precision,
low cost
Drawbacks: Maximum
frequency of
measurements, cone
angle of opening
➔ Check min and max
range!
26 26
Sensors for mobile robots
Sonar / Ultrasonic sensors:
Directivity patterns for two different sonar sensors.
27 27
Sensors for mobile robots
Radar / Doppler / proximity sensor:
Distance and Speed measurement
Similar to ultrasonic telemeter but with electromagnetic waves (radio)
Advantages: Low cost, Highly
industrialized
Drawbacks: Waves carried by air
therefore unusable at high speed,
aerodynamic effects that deteriorate
the signal, absorption of the signal by
certain materials
Main applications: Parking
assistance
28 10
28
Sensors for mobile robots
Infrared telemeter
Proximity and distance optical measurement
Advantages: Measurement acquisition frequency,
precision, short range
Drawbacks: Low distances <5m (Typically <2m), what
about transparent materials?
29 29
Sensors for mobile robots
LIDAR
• Rotating laser with high acquisition
frequency
• 180/360° Precise distance
measurements
• Mainly used for mapping
Example (characteristics):
30 30
Sensors for mobile robots
LIDAR
Can provide 2D or 3D data in an array of
data or in a point cloud
➔ Depends on the need and the
computational power available (need
to apply a transform to each point to
get their absolute position)
➔ Laser can be problematic with
some surfaces (mirrors, windows…)
➔ Depending on the need there is a
choice to be made between the
frequency of acquisition and the
precision (resolution)
31 31
Sensors for mobile robots
LIDAR
32 32
Sensors for mobile robots
Cameras (Vision)
2D Cameras important features:
- Resolution
- RBG // BW or Grayscale // IR (need
infrared light emitters for night vision or a very
sensitive sensor for thermal vision)
- Frame rate (FPS)
- Apperture & Sensitivity
- Fixed / Adjustable focal length
- Rolling or global shutter
- Connectivity (USB? CSI? Other?)
- CMOS / CCD
- Horizontal & Vertical FOV
33 33
Sensors for mobile robots
Cameras (Vision)
2D Cameras important features:
• “Eye to hand” → “Caméra deportée”
• “Eye in hand” → “Caméra embarquée”
≠
34 34
Sensors for mobile robots
Cameras (Vision)
2D Cameras variation: Omnidirectional camera
- Use of optical lenses or shaped mirrors to get a 360° picture.
➔ Nice idea but need to use time-consuming computer vision
algorithm to compensate the optical deformation before any
processing of the image…
35 35
Sensors for mobile robots
Cameras (Vision)
2D Cameras variation: Stereovision
➔ Two 2D cameras used to get precisous depth information
= cheap 3D camera!
➔ Based on how our eyes work with our brain
→ The distance between the cameras must be
known precisely!
36 36
Sensors for mobile robots
Cameras (Vision)
2D Cameras variation: Stereovision
Easy and cheap but results
are not very precise…
37 37
Sensors for mobile robots
Cameras (Vision)
3D Cameras:
Time of Flight (TOF) cameras = TOF sensor matrix
= Infrared telemeter marix
➔ Low resolution, often short range but good precision
➔ The higher the resolution or range, the higher the price!
➔ More and more integration into new smartphones
so it might become cheaper and better in a near future…
38 38
Sensors for mobile robots
Cameras (Vision)
3D Cameras:
Structured IR light = projection of an infrared pattern
➔ The distortion of the pattern show the depth variations
➔ Good precision and resolution
➔ Quite unexpensive
➔ The data fusion with a RGB camera gives a RGB-D camera
39 39
Sensors for mobile robots
Cameras (Vision)
3D Cameras:
Structured IR light = Example of the D415 and D435
Intel RealSense cameras
40 40
Sensors for mobile robots
Cameras (Vision)
41 41
Sensors for mobile robots
Cameras (Vision systems)
Advantages:
• Usually cheap and easy to use
• Allows customed 2D/3D data depending on the need
• Combined with computer vision algorithms it can give a lot of
information about the environment around the robot and allow it to
achieve many tasks (tracking a target, recognize object or faces,
detect patterns…)
Drawbacks:
• The form factor of some camera may not be easy to integrate in
constrained environment
• Useless without computer vision algorithms that need some
computation power to run in real time (if possible)
• Those algorithms can provide false positive detections
42 42
Sensors for mobile robots
GNSS* sensors:
GPS (Global Positioning System)
- Measuring the absolute position of a point in a
fixed landmark (the center of the earth)
- 3 satellites signals needed (4 is more robust)
- Only for outdoor navigation
→ Useful for position recalibration
- Differential GPS (DGPS) → 1-3 cm precision
Advantages: No drift
Drawbacks: Low frequency <5Hz (typically 1Hz),
unavailability of satellite signals (indoor, under water…)
*Global Navigation Satellite System-
43 43
Sensors for mobile robots
Other sensors:
A lot of sensors exist for all your needs !
44 44
Robot components: sensors
What levels of processing for those sensors ?
➔ How much processing power? Which board process the
data?
Low
Electronics: the sensor which detects bumping into obstacles
=> binary data or analog value through ADC.
Signal Processing: microphone sensor for recognizing a voice
=> filtering, clipping…
Computation: a camera to find your best friend at the party
High => algorithm, machine learning…
Low Medium High
45 45
Sensors for mobile robots
Classification example:
• Analog sensors
• Digital sensors
• Logic sensors
Exercise: Build an array
sensors of the following
robots :
• Pepper
• Turtlebot 3 Burger
• Turtlebot 2i
• ISEN Mobile robot
46 46
Variables estimation
What need to be estimated?
• Noise from the data of a sensor (odometry, IMU) or
from any other component (resistance, motor…).
Why?
• Data is too noisy → Impossibility to measure precisely
• Very expensive sensor so we want the best precision
and we need to remove precisely the noise
How to estimate?
• By developing a mathematical estimator
• By filtering the data (Kalman filter)
• Combining or merging many information
47 47
Variables estimation
Mathematical estimator:
We make several measure with the sensor and notice some
fluctuations on the data → there is noise!
Most of the time we use a linear model:
𝑦 =𝑎𝜃+ 𝜔
with: 𝑦 = measured data
𝑎 = known parameter (gain...)
𝜃 = real sensor value
𝜔 = random noise
We will use statistics to characterize the random noise
➔ Hypothesis: 𝜔 is a non zero mean Gaussian white noise
This is a simple example but building an estimator of a system is
a really complex task that needs a lot of data and math skills!
48 48
Variables estimation
Kalman Filter:
Widely used for filtering random noise.
Use the mathematical model of the system and more statistics!
The equations of the Kalman Filter algorithm may seem complex but the most
important is to know how it works and to know how to use it!
Based on 3 simple steps:
1. Prediction: Based on the mathematical model we
estimate the value that has yet to be measured.
2. Measure: We get the raw data read by the sensor.
3. Correction: We calculate the difference between
the estimation and the measurement, and we
update the mathematical model.
The recursive use of this algorithm over time ensure the error to
be null over time as the error is removed at each step.
49 49
Variables estimation
Kalman Filter:
Other existing similar algorithms
depending on your need and your
available computation power:
- Extended Kalman Filter
- Unscented KF
- Particle Filter
…
➔ More to be seen in the chapter
on Localization!
50 50
Multi-sensor data fusion
Purposes of data fusion:
• More robust estimation of the robot state
• Analysis of the robot near environment
• Planning for future actions
Data fusion examples:
• Odometry + GPS = Robust vehicle localization
• Odometry + IMU = Robust simple position for small mobile robots
• Camera + Odometry + IMU + US + Lidar = Development of the
robot dynamic local map (SLAM)
51 51
Multi-sensor data fusion
Data fusion Methods:
• Probability theory
• Theory of possibility
• Theory of fuzzy sets (fuzzy logic)
• Theory of belief functions
• Decision trees
• Rule bases
• Nearest neighbors
• Neural networks
• Bayesian networks
• Markov chains
• EFK
• UKF
• Particle Filtering
• … Kalman Filter!
52 52
Multi-sensor data fusion
Application of the Kalman Filter to data fusion:
The odometry data (red)
accumulate error over
time.
The fusion of the
odometry and IMU data
provide a much better
result but still has a huge
error in the end.
➔ Data fusion is not
magical!
➔ It helps to get more
reliable data from
sensors but absolute
positionning is always
required for precision!
53 53
Multi-sensor data fusion
Application of the Kalman Filter to data fusion:
The same way we considered our sensors data to be noisy,
we can merge two data from different sources by
considering that each one has some statistic error.
➔ A lot of applications: position and LIDAR data fusion
54 54
Multi-sensor data fusion
Application of the Kalman Filter to data fusion:
The same way we considered our sensors data to be noisy,
we can merge two data from different sources by
considering that each one has some statistic error.
➔ A lot of applications: 2D image and Depth data fusion
55 55
Multi-sensor data fusion
Application of the Kalman Filter to data fusion:
➔ A lot of applications: 3D image + position = 3D map
56 56
Next steps…
• Locomotion
• Localization
• Trajectory planning
• Robot Control
• Power supply
• Practical work
57 57
