Collaborative robotics
Dr. Ing. Lorenzo Scalera
Polytechnic Department of Engineering and Architecture
University of Udine
Outline
• Introduction to robotics
• Introduction to collaborative robotics
• Regulations and standards for safety
• Robots in the market
• Research in collaborative robotics at
University of Udine
06/09/2023 Collaborative Robotics 2
Introduction to Robotics
• What is Robotics?
The study of machines that replaces man in the execution of a task, both in terms
of physical activity and decision-making
Automaton:
mechanical creature with the task of replacing
the human being in tasks of subordinate work
The term robot comes from the Czech word
robota = executive work,
coined in 1921 by Čapek in his science fiction play
Rossum's Universal Robots
The birth of industrial robotics (Unimate)
The term robot comes from the Czech word "robota," which means "heavy labor" or "forced labor“.
▪ Unimate was the first industrial robot
ever;
▪ It was invented by George Devol and
Joseph Engelberger (founders of the
Unimotion company) with a patent
registered in 1954;
▪ Created for General Motors, the
robot served a production line for
handling automotive die-cast parts;
▪ The idea was to substitute humans in
hazardous operations (toxic fumes
and possible physical injury);
Introduction to Robotics
• What is a robotic system?
• A robotic system is a mechanical / electronic / IT device capable of
performing a specific task independently
• Elements of a robotic system
• Workspace
• Mechanical structure
• Sensors
• Elaboration unit
• Actuators
Example of a robotic system:
vacuum cleaner robot
• Is it a robotic system?
Example of a robotic system:
vacuum cleaner robot
• It has a very specific goal: remove dust from the
floor of the room
• It does it alone (autonomy)
• It works in a work environment (the room)
• It has a mechanical structure
• It has sensors (position, obstacles, dirt, etc.)
• It has a processing unit that controls the
operations related to the management of the
trajectories and the activation of the aspirator
• It has actuators (motors for locomotion and
aspiration)
Introduction to Robotics
• Robotics can be defined as a science that studies the intelligent connection
between perception and action
• mechanical system (locomotion and manipulation)
• sensory system (proprioceptive and exteroceptive sensors)
• Proprioceptive = acquire information on the internal state of the mechanical system
• Exteroceptive = to acquire information on the external state of the environment
• control system
• ability to intelligently connect action and perception
Introduction to Robotics
• proprioceptive sensors measure the internal state of the robot (position and
velocity of joints, but also torque at joints or acceleration of links)
• kinematic calibration, identification of dynamic parameters, control
• exteroceptive sensors measure/characterize robot interaction with the
environment, enhancing its autonomy (forces/torques, proximity, vision, but also sensors
for sound, smoke, humidity, …)
• control of interaction with the environment, obstacle avoidance, localization of mobile
robots, navigation in unknown environments
Introduction to Robotics
Proprioceptive sensors Exteroceptive sensors
• Position • Force Sensors
• linear displacements: • strain gauges and joint
potentiometers, linear torque sensor
variable differential
transformers (LVDT) • Proximity/distance sensors
• angular displacements: • infrared (IF),
potentiometers, resolvers, • ultrasound (US),
encoders (digital) • laser with structured light
• Velocity and acceleration • Vision
• Tachometer
• Accelerometers
What are the components of an
industrial robot?
• Mechanical structure • Control system (control logic,
(manipulator) with drives)
actuators and sensors
What are the components
of the mechanical system?LINK
END – EFFECTOR
JOINT
The manipulator is a series
of rigid bodies (links)
connected by joints
BASE
What are the components of
the mechanical system?
• Support structure and wrist, which gives dexterity
• Main function: orient the end-effector in the space
• I can reach the same point with more than one orientation
• Actuators impart the movement of the manipulator through
the actuation of the joints
• Electric, hydraulic and pneumatic actuators
• Sensors measure the state of the manipulator
(proprioceptive) and possibly the state of the surrounding
environment (exteroceptive)
The control system
• The control system of a robot is a very
complex and sophisticated product, which
requires high software development times
• Its main tasks are as follows:
• Interface with the operator
• Trajectory planning
• Real-time control of joint motion
• Data storage
• Interface management with other machines
• Diagnose and manage malfunctions
Structure and classification of manipulators
Open kinematic chain Closed kinematic chain
There is only one sequence of arms The arm sequence of the manipulator
that connects the two ends of the constitutes a closed chain (a ring)
chain
Example of parallel
Example of serial robot robot (Delta Robot)
Structure and classification of manipulators
Articulation between two consecutive arms:
• REVOLUTE JOINTS (relative rotation between two
joints)
• Allows the link i to move relative to the i-1 link by
rotating around an axis passing through the joint i
• PRISMATIC JOINTS (relative motion of translation
between two joints)
• It allows the link i to move with respect to the i-1 link
translating along an axis passing through the joint i
Structure and classification of manipulators
SCARA manipulator (Selective Compliant
Assembly Robot Arm)
• 3 or 4 degrees of freedom
• Two revolute joints, one prismatic and eventually one more
revolute joint
• The motion axes are all parallel to each other
• High stiffness to vertical loads and compliance to horizontal
loads
• Assembly for vertical insertions
• Handling of small objects
• The positioning accuracy of the end-effector is reduced as the
distance of the wrist from the axis of the first joint increases
Examples of SCARA manipulators for pick-
and-place operations
SCARA manipulators for pick and place
Structure and classification of manipulators
Articulated manipulator
• Three revolute joints
• The axis of rotation of the base joint is
orthogonal to the other two parallel ones
• The second joint is called shoulder while the
third elbow
• The positioning of the wrist changes
throughout the workspace
• Workspace is a portion of a sphere
• Large volume of workspace compared to the
volume of the manipulator
Examples of articulated manipulators
Structure and classification of manipulators
Spherical wrist
• To guarantee arbitrary orientations in a 3D space,
the wrist must have at least three degrees of
freedom achieved by rotary joints
• The realization that gives the wrist greater dexterity
is that in which the three axes of rotation intersect
at one point (spherical wrist)
• Uncoupling between position and orientation of the
end-effector
• The supporting structure positions in the space the
intersection point of the three axes of the wrist
• The wrist itself determines the orientation of the
end-effector
Structure and classification of manipulators
• Spherical wrist • Non-spherical wrist
Structure and classification of manipulators
• Parallel robots
• The end-effector is connected directly to its base usually by a number
of arms independent of each other and that work in parallel, or work
together and coordinated, but not necessarily aligned in parallel
Stewart platform,
used for example Parallel robot,
for flight Mitsubishi
simulators Electric
Structure and classification of manipulators
• Stewart's platform as a flight
simulator
• 6 spatial degrees of freedom
• 6 prismatic joints (hydraulic jacks or linear
electric actuators) fixed in pairs from three
positions on the platform base plate, to three
mounting points on a top plate
• All 12 connections are composed of universal
joints
• Three linear movements x, y, z (lateral,
longitudinal and vertical) and three rotations in
space (pitch, roll and yaw)
Structure and classification of manipulators
• Parallel robot for haptic interfaces
• Haptic interface: device that allows to
operate a robot, real or virtual, and to
receive tactile sensations in response
(feedback)
• The haptic word derives from the Greek
apto, "touch"
• Applications in areas of advanced robotics,
such as surgical and space robotics, and
those related to virtual reality and tele-
manipulation
• Haptic interface (robot master) and remote
or virtual device (slave)
Structure and classification of manipulators
Parallel robots
• Fast pick-and-place applications
• Light structure (arms often made of carbon)
• Reduced inertia
• Greater dynamic performance
• High speeds and high accelerations
• Greater stiffness
• Generally low load capacity (a few kilos)
• Parallel robots are generally more limited in the
work area; for example, generally they cannot get
around obstacles
Delta robot - packaging
Cable-driven parallel robots
• Flexible link mechanisms (cables)
• Parallel robots
• Light, easily reconfigurable, transportable
• Used for transporting loads in large workspaces
• The end-effector may be completely constrained or suspended
Cable-driven parallel robots
The Skycam The Cable Robot Simulator
Cable-driven parallel robots
• The Arecibo Radio • Cable-robot for rehabilitation
Telescope (Puerto Rico)
Mobile robots
• Automatic machine able to
move in the surrounding
environment
• Presence of a moving and
autonomous base
• Service applications, where
extensive and autonomous
mobility is required:
• Indoor applications
• Outdoor applications
Mobile robots
Indoor applications:
• Cleaning of large rooms
• Service robotics for shops,
museum, industries, etc.
• Internal surveillance of buildings
• Goods storage in automated
warehouses
• AGV (Automatic Guided Vehicle)
for palletizing and transporting
finished products
Mobile robots
Outdoor applications:
• Military applications
(mine clearance)
• Space and
underwater
exploration
• Civil protection
• Automated
agriculture
Mobile robots
One or more rigid bodies equipped with a
locomotion system:
Wheeled Mobile Robots (WMR):
Consisting of a rigid body (base or chassis)
and a system of wheels that move it with
respect to the ground
Mobile robots on legs:
Consisting of several rigid bodies, connected
to each other by prismatic or revolute joints
Biomimetic robotics: their project is inspired
by living organisms
What are the typical applications of an
industrial robot?
• MANUFACTURE • MEASURE
• TRANSPORT • Spot welding • Dimensional test
• Arc welding • Profile detection
• Handling
• Sealing, gluing • Identification of
• Pick-and-place • Assembly (assembly) manufacturing defects
• Loading / Unloading • Milling, drilling
machines
• Pressing
• Palletizing packing • Deburring, grinding
• Selection of parts • Screwing, wiring
• Painting
Production line BMW 3 series -
welding robots
CMA Robotics - Robotic painting line
From industrial to collaborative robots
06/09/2023 Collaborative Robotics 41
What does collaborative robotics mean?
Being collaborative means:
«To work with someone to create or produce something»
Collaborative robots (cobots):
• bring together the potential of human and robot
capabilities to complete tasks in a semi-automated
manner that were impossible to achieve fully
automatically
• are a new generation of industrial robots designed to
work together and in safety with human operators,
without the need for barriers or protective cages to
divide them
• collaborate with humans in the Smart Factory
42
The concept of
collaborative
robotics is strictly
related to the
framework of
Industry 4.0
43
Collaborative robotics in the framework of Industry 4.0
44
Collaborative robots
• Slower and less powerful than traditional robots
• Work at the same level of strength and speed as humans
• Safe, light, transportable and flexible
• Easy to program thanks to intuitive interfaces and the provision
of tablets (teach pendant)
• Simplify the production layout (no barriers, small footprint)
• Simplify the installation
• Improve working conditions of human operators
• Reduce prolonged repetition of the same gestures
• Staff qualification and gratification
45
Security features of collaborative robots
• Collaborative robots are provided with safety systems
for speed limitation, force and power control
• They have rounded joint covers, made of plastic or
covered with special materials, to ensure the
protection of operators in the event of contacts
• They are as ergonomic as possible, on a human scale
• There are no motors, cables, mechanical or electronic
components exposed
• Joints of collaborative robots are usually variable-
stiffness joints, to decrease contact forces in cases of
contacts or shocks
46
Enabling technologies for collaboration
• Sensitive Lightweight Robots • Safe Gripping
• reduced mass = reduced kinetic energy • Never cause injuries
• advanced sensors for torque sensing • Detect human contact
• Safety-rated functions & reliability • Never loose the manipulated object
• Safety sensors & actuators • Safe Mobile Manipulation
• Functionally-safe control schemes • Operate in shared environments
• Redundant system architecture • Full body interaction
• Perceive the environment
47
Enabling technologies for collaboration
• A safe sensitive manipulator and a safe gripper are not enough!
• We also have to consider workpiece, task, environment, context
• Essential aspects of applications in collaborative robotics are:
• Safe (sensitive) robot
• Safe (sensitive) gripper
• Safe grasp
• The manipulated objects
• The object’s fixture
• The path of the robot
48
Common applications for collaborative robots
• Structured tasks, if high precision is not required
• Unstructured tasks where it is difficult or too expensive
to use traditional industrial robots
• Examples are:
• Pick-and-place operations
• Packaging
• Manufacturing industry
• Assembly tasks
• Test, quality control
49
Traditional industrial robots and collaborative robots
Traditional industrial robots Collaborative robots
"Blind" and unaware of the surrounding environment They see, perceive the environment and people
Dangerous Safe
Characterized by high precision and repeatability Limited precision and repeatability
Programmed for a specific task Flexible, in a way similar to a human operator
They require components and integration Perfectly integrated
They require expert programmers Easy to used and to be programmed
Generally expensive Generally low-cost
50
Types of interaction with industrial robots
Human-Robot Co-X
From the perspective of the human: To date, coexistence
and cooperation,
• Separation between robot and human defined by the
fundamental
parameters of space
• Co-existence between robot and human
and time, have
accounted for about
• Co-operation between robot and human 90% of the situations of
interaction between
• Collaboration between robot and human man and robot!
51
Coexistence between robot and human
• Different spaces, same time
• Situation in which human operators and machines operate
simultaneously in adjacent work areas
• Typical example: loading station with a rotating table in a
robotic area
• Operating simultaneously and safely in adjacent work areas is
possible, for example, thanks to the use of safety barriers
52
Cooperation between robot and human
• Same space, different times
• Humans and robots operate in the same working area, but at
different times
• Typical example: robotic assembly station
• The robot and the operator alternate in the assembly area
• The distance between operators and robots becomes the key
parameter for safety
• A safety laser scanner is usually adopted to detect the
operator's position and to ensure that the robot's speed is
reduced or eventually stopped in case of a potential collision
53
Collaboration between robot and human
• Same space, same time
• The type of full interaction between robot and human
• Sharing the same workspace at the same time
• Example: mobile platform where a robot collects pieces from a
conveyor belt and supplies them to a workstation chaired by an
operator
• The security parameters used for the cases of coexistence and
cooperation are no longer sufficient
• The position and speed of the robot must be monitored, limited
and, if necessary, interrupted according to the degree of danger
• The robot thus acts as a flexible production assistant which
lightens the operator's work during ergonomically unfavorable
manual work sequences
54
Regulations & Standards for Safety
06/09/2023 Collaborative Robotics 55
Regulations & Standards for Safety
The 3 fundamental laws of Robotics
(Isaac Asimov, 1940s, science fiction writer):
• A robot cannot hurt a human being or allow, by
remaining inactive, a human being in danger
• A robot must obey orders given by human beings,
unless those orders conflict with the first law
• A robot must protect its existence unless such
protection conflicts with the first or second law
Regulations & Standards for Safety
Industrial robot systems operate according to:
• ISO 10218: Robots and robotic devices: Safety requirements
for industrial robots
• ISO/TS 15066: Robots and robotic devices: Collaborative
Robots
• ISO 13849 or IEC 62061: Safety of machinery: Safety-rated parts
of control systems
• IEC 61508: Functional Safety
• ISO 12100: Risk Analysis
• 2006/42/EC: Machinery Directive
57
Regulations & Standards for Safety
• The concept of humans and robots working hand in hand is
not new
• ISO 10218 defines human-robot collaboration as:
• “State in which a purposely designed robot works in direct
cooperation with a human within a defined workspace”
• purposely designed means that highly reliable components
and redundancy are exploited
• defined workspace means collaborative workspace, which is
identified as:
• “Space inside the protected area where robot and man can
perform tasks simultaneously during production
operations” (ISO 10218 / ANSI RIA 15.06)
58
Regulations & Standards for Safety
• ISO 10218 allows collaboration in four basic principles
of protection as part of the human robot collaboration:
1. Safety-rated monitored stop: The robot stops as
soon as the operator enters the common
workspace and goes back to work as soon as
he/she leaves it
2. Hand guiding: The movements of the robot are
actively controlled by the operator thanks to
suitable devices
3. Speed and separation monitoring: Contact
between the operator and the moving robot is
prevented
4. Power and force limiting: The forces released by
the contact between the operator and the robot
are rendered harmless with technical measures
59
Regulations & Standards for Safety
Safety-rated monitored stop
• Safety-rated control system ensures that the robot
remains in a clearly defined pose and re-starts in a
defined and monitored way
• If a human enters the safety zone, the active
collision avoidance is activated, declined in various
ways:
• audible warning (if the operator has just entered
the safety zone)
• controlled stop (the robot must stop - it does not
turn off, but the brakes are activated)
• recession from the operator or path modification
• The active collision avoidance is mainly used when
a robot works mostly alone, but occasionally a
human operator can enter its workspace
60
Regulations & Standards for Safety
Hand Guiding (ISO 10218-1/5.10.4 & ISO/TS 15066)
• The safety of the human–robot collaboration is assured by the
robot being guided manually and controlled at an appropriately
reduced speed
• The operator can teach the robot positions by moving the
robot without the need of an extra interface, e.g., a robot
teach pendant
• User can release the robot through holding the enabling device
• Ensured maximum speed (according to the risk analysis)
• Safety-rated monitored stop when released from hand
guidance
61
Regulations & Standards for Safety
Speed and Separation Monitoring (ISO 10218-1/5.10.4 & ISO/TS 15066)
• Both speed and motion path of the robot are supervised and adjusted
based on the position and speed of the operator in the safeguarded
space
• The risk is mitigated by ensuring a sufficiently large separation between
robot and human
• Static regions and dynamic distances are identified
• Distances are defined according to the risk analysis that takes into
account: robot, tool, workpiece and task
• A safety-rated monitored stop is implemented in case of violation
(reaction-time & stopping-time need to be considered)
• This scenario requires safety-rated hard-and software for robot,
distance-and speed monitoring
62
Regulations & Standards for Safety
Speed and Separation Monitoring (ISO 10218-1/5.10.4 & ISO/TS 15066)
• The work environment is equipped with lasers or vision systems in order
to track the position of the operators
• Safety zones are graduated so that the robot can react differently based
on the position of the operator
• the robot can operate at full speed when the human is in the green
zone, at reduced speed in the yellow zone, and stops when the human
moves into the red zone, which defines the minimum separation
distance
• It waits until it receives a command to be able to resume its task
63
Regulations & Standards for Safety
Power and Force Limiting (ISO 10218-1/5.10.5 & ISO/TS 15066)
• The risk is mitigated by limiting the mechanical impact
• The risk analysis considers robot, tool, workpiece and task
• A protective stop is implemented in case the safety-rated force
monitoring reports limit-violations
• This scenario is widely adopted in modern collaborative robot
applications
• It is realized though speed reduction, reduced masses, limited
torques, specific geometry and padding, inherently safe
actuators, inherently safe control schemes, workplace
ergonomics, and clearly defined task of the robot
• Both foreseeable transient and quasi-static contact situations
are considered
64
Collision classification between human
and robot
Collision Classification and Pain Thresholds (ISO/TS15066)
Transient (short-term) contact
• Quick contact, in which the robot control system does not have time to act
• Energy transfer linked to the relative speed between robot and human, from the
moving masses and from the contact area
• Example: unconstrained collision of the moving robotic arm with the human
operator's arm
Quasi-static (prolonged) contact
• Extended contact over time, the control system has time to reduce speed and
force
• Danger due to the application of force and pressure
• Example: crushing of the operator's hand trapped between the robot arm and the
work bench
65
Sensors for collaborative robots
To ensure safety without using physical
barriers, the following are used:
• Safety curtains and laser systems
• Non-contact safety switches
• Safety laser scanner
• Force and torque sensors (integrated in
the robot or external)
66
Sensors for collaborative robots
• Vision systems
• Recognition of voice commands
• Contact sensors
67
Annual installation of industrial robots
68
Operational stock of industrial robots
06/09/2023 Collaborative Robotics 69
Annual installation of industrial robots
06/09/2023 Collaborative Robotics 70
Industrial robots by application
06/09/2023 Collaborative Robotics 71
Collaborative and traditional industrial robots
06/09/2023 Collaborative Robotics 72
Annual installation of industrial robots
06/09/2023 Collaborative Robotics 73
North America collaborative robot market size, by
payload capacity
74
Leading manufacturers of
collaborative robots
75
Leading manufacturers of
collaborative robots • Danish company founded in 2005
Universal Robots UR3, UR5, UR10 • 60% of the market
• More than 10,000 robots installed
worldwide
• Simple and cheap
• Force / torque sensor on the wrist
• Light
• Wide range of accessories
• New model: UR16e: load 16 kg, reach
0.90 m, weight 33 kg
76
Leading manufacturers of
collaborative robots
Fanuc
77
Leading manufacturers of
collaborative robots
TechMan – OMRON
2018: strategic alliance between
Techman Robot and OMRON, leader in
industrial automation, to promote the
distribution of collaborative robots of
the TM series to the global market
78
Leading manufacturers of
collaborative robots
Baxter:
Rethink Robotics (2012-2018)
• 360 ° laser scanner for identification of
Baxter, Sawyer people and obstacles
• Flexible movement guided by the vision
and object recognition system
• 7 degrees of freedom per arm
• Interchangeable end-effector to face
different tasks and operations
• Flexible joints with force sensors and
force control to increase safety
79
Leading manufacturers of
collaborative robots
Rethink Robotics Baxter
80
Leading manufacturers of
collaborative robots
AUBO Robotics F&P Robotics
81
Leading manufacturers of
collaborative robots
KUKA LBR iiwa
• «intelligent industrial work assistant»
• LBR: 7 axis e 7/14 kg payload;
programmable in Java
• Torque sensors at joints
• Possible to be mounted on a mobile robot
82
Leading manufacturers of
collaborative robots
ABB Yumi
It means "you and me"
• 2 coordinated arms
• Limited payload (500 g x2)
• Weight: 38 Kg
• Reach: 559 mm
• Accuracy: 0.02mm
• Cost: $ 40,000 approx
• Used for assembling or moving
instruments
83
Leading manufacturers of
collaborative robots
Comau Aura
Franka Emika
• Advanced Use Robotic Arm
• Panda robot • Contact sensitive coating
• Articulated robot with 7 DOF • Integrated vision system
• 3 Kg payload, reach 855 mm • Industrial derivation
• Force / torque sensor on the wrist
• Load: 170 kg
• Reach: 2.8 m
84
Mobile collaborative robots
MIR Mobile Industrial Robots
KUKA
Clearpath
Robotics
OMRON
85
Humanoid collaborative robots
Robonaut 2 (NASA)
NextAge by Kawada
Industries
86
Humanoid collaborative robots
Justin by DLR (German Aerospace Center) Toro by DLR
87
Research in
collaborative robotics
at University of Udine
06/09/2023 Collaborative Robotics 88
An approach for collision avoidance in
collaborative robotics
• Human and robot are considered to be
enclosed in bounding volumes (capsules)
• The robot safety zones are considered
dynamic: their size changes with respect to the
actual speed of the robot
• A supervisory controller checks the distance
between human and robot in real time
• If a potential collision is detected, the robot is
steered to stop engaging a stop trajectory
Proposed approach
• The collision avoidance algorithm computes
in real time the minimum stop time needed
for the robot to stop
• The stop time of the robot has to satisfy the
kinematic and dynamic limits of the
manipulator
• If a potential collision is detected, the robot
is stopped with a stop trajectory
• The robot restarts as soon as the safety
condition is respected
Numerical simulations
• Three different approaches have been compared:
• Approach (1): online optimization of the stop time
• Approach (2): selection of the stop time among predefined values
• Approach (3): implementation of the ISO/TS 15066, resulting in static
safety zones
Experimental validation
• Franka Emika Panda robot with 7 degrees of
freedom
• Stereo camera RGB-D Intel Realsense D435
• Workstation
• Software implemented in ROS Melodic using
Ubuntu 18.04 and Python
• Marker and calibration tip printed in 3D
Online computation of the distance
between human and robot
• Position of the human derived by
the camera and a skeleton tracking
software
• Position of the robot derived from
the forward kinematics
• Computation of the distance
between each segment belonging
to the human and to the robot
Experimental tests
• 27 subjects participated to the
experimental tests
• The human has to walk along a
predefined path close to the
robot Test n.1, percorso in rosso, 2 Test n.2, percorso in blu, 3
metodi, capsule dinamiche metodi, statico e dinamico
• The robot has to repeat a
predefined task, stopping in
case a potential collision with
the human is detected
Experimental test (Test 1)
Experimental test (Test 2)
Static capsules Dynamic safety zones – Dynamic safety zones –
iterative search online optimization
… not only industrial applications!
• Development and test of image processing and path
planning algorithms for artistic applications
• Watercolor, tempera, palette-knife techniques
• Recently, we adopt an eye-tracker device to control the
painting robot with eye gaze only
Artistic robotic painting
06/09/2023 Collaborative Robotics 98
Robotic painting with eyes
06/09/2023 Collaborative Robotics 99
Thank you for your attention!
Dr. Lorenzo Scalera
Polytechnic Department of
Engineering and Architecture
University of Udine
Via delle Scienze, 206
33100 Udine
[email protected]
06/09/2023 Collaborative Robotics 100
