Journal of Scientific & Industrial Research

Vol. 83, August 2024, pp. 819-829

DOI: 10.56042/jsir.v83i8.7879

Humanoid Left Arm Collaboration Empowered by IoT and Synchronization of

Human Joints
Ayush Kumar Agrawal & Jayendra Kumar*
Department of Electronics and Communication Engineering, National Institute of Technology Jamshedpur 831 014, Jharkhand, India

Received 08 January 2024; revised 13 May 2024; accepted 14 June 2024

A robotic left arm has been developed in this research to assist individuals in safely navigating hazardous environments,
utilizing Internet of Things (IoT) technology to synchronize its movements with those of a human operator in real-time. The
primary objective of this work was to enhance the precision and responsiveness of the arm. Sensors and microcontrollers
(ESP32, Arduino UNO) were integrated into the robotic arm to collect and transmit movement data. Advanced machine
learning algorithms, including Inverse Kinematics, Reinforcement Learning, and Deep Learning, were employed to facilitate
the arm's learning process and improve its actions. The system was evaluated in simulated hazardous scenarios to assess its
performance. The results indicate that the robotic arm can adjust and move accurately, aligning with the human operator's arm
movements. The robotic arm safely manages hazardous materials, demonstrating high precision in executing complex tasks.
These findings underscore the effectiveness and safety enhancement achieved by integrating IoT and intelligent algorithms into
robotic systems for high-risk occupations. This technology significantly enhances emergency response capabilities and critical
operations, delivering dependable and precise robotic assistance. Overall, this research underscores the potential of IoT-enabled
robotic arms to revolutionize human-robot interactions in demanding environments. By showcasing how real-time
synchronization and advanced algorithms enhance the functionality and safety of robotic systems, this research presents a
promising solution for strengthening the operational safety and efficiency in hazardous situations. The development and
utilization of this technology represent substantial progress in fields necessitating precise and dependable robotic assistance,
ultimately promoting improved collaboration between humans and robots in high-risk contexts.

Keywords: High-risk environment, Humanoid robot, Human-robot interaction, Real-time synchronization, Robotic arm

Introduction done perfectly in areas where human touch is still

Robots have come a long way, making it easier to necessary, marking a significant change in how we
use resources smartly, speed up production, cut costs, use robots in different fields.
and do tasks that humans can't do as well. They're Our research explicitly addresses disaster response,
essential in keeping emergency workers safe during industrial accidents, and hazardous material handling
disasters because they can adapt and work precisely in scenarios, where humanoid robots can significantly
dangerous situations. This progress is changing how enhance safety and efficiency. By providing real-time
we manage resources and produce goods while also synchronization and control, our study fills a critical
highlighting how robots help protect people in harsh gap in the current robotic technology landscape,
conditions. Humans have limits in extreme or risky particularly in high-risk environments.
environments, but robots controlled by the Internet of
Significant strides have been made in developing
Things (IoT) can adapt better, work more precisely,
robotic arms and humanoid robots in recent years, yet
and keep people safer. The study of ergonomics,
persistent gaps need addressing. For instance, Pravin
which looks at how people interact with their
et al.1 emphasized the importance of gyroscopic
environment, helps design these humanoid robots by
acceleration sensors for the 360° movement of robotic
considering human body measurements, movement,
arms. These sensors enhance balance and stability, yet
and mental and physical needs. Collaborative robots,
their study lacks a comprehensive approach to
or cobots, are a big step forward. They work
optimizing the algorithms processing this data. This
alongside humans, combining human skills with
gap suggests a need for further research into advanced
robotic efficiency. This teamwork ensures tasks are
—————
software solutions that can fully use these sensors,
*Author for Correspondence enabling more precise and adaptable robotic motion
E-mail: [email protected] control in various operational contexts.
820 J SCI IND RES VOL 83 AUGUST 2024

Similarly, Khaleghi et al.2 investigated using enhancing adaptability and learning capabilities,
actuators, sensors, and cameras for controlling robotic ensuring long-term durability, and optimizing
hand gestures through a neural network trained on communication protocols, we can significantly
26,000 images. While their work demonstrates advance the development of more intelligent, reliable,
significant progress in gesture recognition, it does not and versatile robotic systems for practical
sufficiently address real-time processing and latency applications.
challenges, which are critical for practical The field of Robotics with IoT is called Internet of
applications. Exploring solutions to these issues could Robotic Things (IoRT) which links either of them.
enhance the robot's responsiveness and reliability in IoRT is robotics driven by IoT, an amalgamation of
dynamic environments. Dutta et al.3 developed a both disciplines. Robots can operate independently
robotic arm to assist individuals with paralyzed lower and make wise decisions, including functions such as
body parts by enabling multiple degrees of freedom in motion and interaction with the physical
movement. Their innovative design includes environment.8–10 In disaster management scenarios,
synchronized finger movements and five-finger IoRT can enable robots to perform search and rescue
rotation. However, the study does not investigate such operations, identify hazards, and assist in medical
systems' long-term durability and maintenance. Future emergencies without risking human lives. For a robot
research should focus on the materials and to become intelligent and collaborative, IoRT is the
mechanisms that ensure the sustained functionality of way forward. The application of IoRT includes
these assistive devices under continuous operation. automation in supply chain solutions, warehousing,
Maurice et al.4 highlighted the role of robotic systems manufacturing, e-commerce, healthcare, personal
in preventing musculoskeletal disorders by robots, disaster management and military
performing physically demanding tasks. Their applications. Now, this research focuses on the
assessment covered balance, force, and vision but did building, assembling and operating the humanoid
not consider adaptive algorithms for dynamically robot's left arm, wherein various parts such as sensors,
adjusting these parameters based on real-time accelerometers, gyroscopes, boards, motors and
feedback and task requirements. Integrating adaptive others are deployed for the construction. Initially, the
learning capabilities could significantly improve the fundamentals and workings of the Arduino board will
effectiveness of robots in unpredictable human be discussed, followed by a literature review
environments. examining the research paper referred to in the
While many researchers have explored enhanced context of constructing the humanoid robot.
communication methods between robotic systems, a Furthermore, humanoid robots, including their
comprehensive approach that considers adaptation to working, sensing, perception, locomotion, and
different contexts and varying task complexities is human-robot, will be addressed. During the
often missing.5,6 Future studies should aim to develop construction of the humanoid robotic left Arm, every
scalable and adaptable communication protocols that piece of hardware has its importance. For instance,
can handle diverse robotic operations seamlessly. the Arduino Board will act as a link between the robot
Sugihara et al.7 advanced the field with algorithms and the hardware. Sensors will control the angular
enabling humanoid robots to move with human-like velocity, collect data from the physical environment,
agility and fluidity. Their foundational work on Zero and convert it into an electrical signal. The
Moment Point (ZMP) and Center of Gravity (COG) gyroscopecan handle movement up to 360° in the
algorithms was pivotal. However, they did not explore robot. This is how the functioning of each hardware is
the integration of these algorithms with modern different yet adapted to the objectives. Moreover, the
artificial intelligence and machine learning methodology will delve into various aspects such as
techniques. Such integration could enhance humanoid the usage of parts, their application and deployment,
robots' cognitive and adaptive capabilities, enabling characteristics and the process followed to achieve the
them to learn new tasks and adapt to evolving desired result of the movement of the robotic arm.
environments more effectively. In conclusion, while This research addresses the gap in integrating IoT
these studies contribute valuable insights into technology into humanoid robots for high-risk
robotics, they also highlight areas that require further environments, which remains underexplored. Current
investigation. By refining motion control algorithms, robotic systems lack real-time synchronization and
 AGRAWAL & KUMAR: IOT BASED JOINT SYNCHRONIZED COLLABORATIVE HUMANOID ARM 821

precise control to assist humans in hazardous with sensors, actuators, and AI, they can precisely
situations effectively. By developing an advanced perceive their surroundings, make decisions, and
humanoid robotic left arm, this study utilizes IoT for execute tasks. Humanoids are increasingly employed in
enhanced adaptability, precision, and safety. This healthcare, aiding in patient care and rehabilitation, and
research advances human-robot collaboration with a they hold promise in addressing labor shortages in
humanoid robotic left arm by: industries.11 Researchers focus on enhancing their
 Showcasing cutting-edge integration of IoT and cognitive abilities as they evolve, enabling them to
sensors. comprehend and respond to complex human emotions
 Demonstrating effective risk mitigation in high-risk and social cues. Ethical considerations surrounding
scenarios. their deployment, such as privacy and the potential
impact on employment, also accompany their
 Enabling real-time collaboration between a
development. Despite these challenges, humanoid
humanoid robot’s left hand and a human’s left hand
robots continue to push the boundaries of what
through IoT integration.
machines can achieve, offering innovative solutions to
Focusing on real-time collaboration and control, real-world problems and opening new avenues for
our study aims to improve human-robot interactions human-robot collaboration.12,13 The forthcoming
and risk mitigation strategies in challenging discussion will delve further into various techniques,
environments. For example, during a chemical spill, a shedding more light on the intricacies of humanoid
humanoid robot equipped with our technology can robotics.
precisely handle hazardous materials, reducing the
risk to human workers. Such practical applications  Characteristics of Humanoid Robot: Humanoid
highlight the transformative potential of our research robots boast striking resemblances to humans,
in real-world scenarios. featuring anthropomorphic designs that emulate
This paper discusses the state-of-the-art techniques human anatomy. Their naturalistic motions, a key
used to date and the benefits of using IoT, which can characteristic, enable fluid and lifelike movements,
be the breakthrough to advance human-robot enhancing their adaptability in various environments.
interaction and represent a groundbreaking use of IoT The versatility of humanoid robots extends beyond
in collaborative humanoid robotics, signalling the mere physical resemblance, allowing them to perform
next generation of interactive robotic systems. a wide range of tasks with agility and precision. An
Further humanoid robotics and IoT are also inherent trait is their interactive nature, facilitating
presented to channel the research properly, followed user engagement through speech, gestures, and
by the methodology, formulas used, and real-time lab expressions. Furthermore, these robots exhibit
images of the work done in the result section. collaborative capabilities, seamlessly working
alongside humans in tasks that demand teamwork.14–
17
Humanoid Robot These defining characteristics collectively position
Humanoid robots, with their anthropomorphic humanoid robots as multifaceted entities poised to
design and advanced capabilities, represent a cutting- revolutionize industries and human-robot interactions.
edge frontier in robotics. Inspired by human form and Which are shown in Fig. 2.
function, these robots aim to replicate human-like
movements and interactions, making them highly
versatile in various applications. Fig. 1 illustrates the
dynamic evolution of robotics from the 1950s to the
2020s, capturing milestones such as the introduction
of industrial robotic arms, programmable robots, and
the integration of Artificial Intelligence (AI). This
visual chronicle showcases robotics' continuous
advancement and diversification, shaping
technological landscapes across various industries.
Humanoid robots showcase remarkable agility,
mobility, and adaptability, from assisting in high-risk
scenarios to augmenting industrial processes. Equipped Fig. 1 — Evolution of robotics development over decades
822 J SCI IND RES VOL 83 AUGUST 2024

 Sensing and Perception: Sensing and perception  Locomotion in Humanoid Robot: Locomotion in
are vital facets of humanoid robots, encompassing robotics is a dynamic interplay between legged
various modalities to enhance their interaction with movement, balancing, and stability control, crucial
the environment as mentioned in Fig. 3. Vision for navigating diverse terrains and environments.
capabilities empower robots with sight, enabling Balancing mechanisms ensure the robot maintains
object recognition and navigation. Auditory sensors stability during motion, preventing falls and enabling
equip them with sound perception, facilitating agile maneuvers. Stability control algorithms
responsiveness to auditory cues.18 Tactile and force contribute to real-time adjustments, optimizing
sensors endow robots with a sense of touch, crucial weight distribution and adapting to changing
for delicate tasks and safe interactions with objects surfaces.21 This comprehensive approach to
and humans. Environmental awareness, a critical locomotion empowers robots to move effectively,
dimension, involves the integration of sensors to negotiate obstacles, and operate in challenging
comprehend surroundings, ensuring adaptability to conditions, extending their applicability across
dynamic conditions. This comprehensive sensory various fields, from search and rescue missions to
suite enables humanoid robots to interpret and industrial tasks.22
respond to their environment, fostering versatility  Human-Robot Interaction: Human-robot interaction
and effectiveness in diverse tasks and scenarios.19,20 is enriched through multi-faceted communication
channels, incorporating speech and natural language,
emotional and social intelligence, gesture and facial
expressions, and haptic interaction. Speech and
natural language capabilities enable robots to
comprehend and respond to verbal commands,
facilitating seamless communication with humans.
Emotional and social intelligence equip robots with
an understanding of human emotions and social cues,
fostering empathetic interactions.23–25
The integration of gestures and facial expressions
allows robots to convey information non-verbally,
enhancing the subtlety and nuance of communication.
Haptic interaction, involving touch and physical
feedback, deepens the interactive experience, enabling
robots to engage with users on a tactile level.26,27
Together, these elements form a comprehensive
Fig. 2 — Characteristics of humanoid robot framework for human-robot interaction, enriching
communication and collaboration and paving the way
for robots to become integral partners in various
domains, from healthcare to customer service, as
highlighted in Fig. 4.

Methodology for the Proposed Working Model of

Robotic Hand
The methodology for the operation of the humanoid
robotic left arm involves a systematic approach to data
acquisition, model implementation, and the utilization of
machine learning algorithms. Data acquisition
encompasses the integration of sensors such as MPU-
6050 and MPU9250, capturing essential physical
information from the environment. The MPU-9250 was
chosen due to its advanced 9-axis sensor fusion
Fig. 3 — Sensing and perception capabilities, combining a 3-axis gyroscope, a 3-axis
 AGRAWAL & KUMAR: IOT BASED JOINT SYNCHRONIZED COLLABORATIVE HUMANOID ARM 823

accelerometer, and a 3-axis magnetometer. This allows Learning for adaptive decision-making in dynamic
for highly accurate motion tracking and orientation scenarios, and Deep Learning to process sensory inputs
sensing, which are crucial for the precise control of the and recognize patterns. The methodology also involves
robotic arm. In contrast, the MPU-6050, which using standard methods to ensure the accuracy and
combines only a 3-axis gyroscope and a 3-axis reliability of the experimental setup. These methods
accelerometer, lacks the additional magnetometer, include rigorous calibration of sensors to minimize
making it less suitable for applications requiring precise measurement errors and validate kinematic models to
orientation in three-dimensional space. ensure that the robotic arm's movements accurately
The employed model, particularly relevant to the correspond to the desired inputs. Data transmission is
robot's working, integrates Inverse Kinematics for facilitated through IoT connectivity, enabling seamless
precise joint parameter calculation, Reinforcement communication between the robotic arm and human
joints. This holistic methodology ensures a
comprehensive workflow, allowing the robot to operate
collaboratively, make informed decisions, and respond
dynamically to its surroundings, showcasing a blend of
advanced robotics and machine learning.
The units of the X and Y axes in the given plot
would be in arbitrary units since the specific lengths
of the robot arm segments (l1 and l2) are defined. The
movement pattern and trajectory shape are
meaningful regarding the robot arm’s configuration.
Still, the exact scale is not defined unless the lengths
Fig. 4 — Human-robot interaction
of the robot arm segments are specified in real-world
units (e.g., meters), as shown in Fig. 5. The units are
relative and depend on the scaling of the robot arm
lengths, which are not explicitly provided here.
Data Acquisition
The 3D printing of this robotic arm was initiated
using the PLA+ material for this project. MPU6050
and MPU9250 sensors are deployed to achieve the
desired degree of rotation and movement in the
robotic arm.15 The primary function of the MPU-6050
sensor is to measure the acceleration and angular
velocity in 6-directions of freedom. Meanwhile, the
MPU-9250 measures acceleration, angular velocity,
and magnetometers in 9 degrees of freedom.
Sensor-wise specification is discussed in Table 1
Fig. 5 — 2-DOF robot arm forward kinematics trajectory below:
Table 1 — Specification of MPU-6050 and MPU-9250
Specification MPU-6050 MPU-9250
Axis 3 Axes viz., X, Y & Z 3 Axes viz., X, Y & Z
Accel full scale range (in g) User Configurable; ± 2g, ± 4g, ± 6g, ± 8g ,& ± 16g User Configurable ± 2g, ± 4g, ± 6g, ± 8g
& ± 16g
Gyroscopic axis 3Axes viz., X, Y & Z 3 Axes viz., X, Y & Z
Gyro full scale range (in °/sec) User Configurable; ± 250°, ± 500°, ± 1000°, User Configurable; ± 250°, ± 500°, ± 1000°,
and ± 2000° and ± 2000°
Communication Inter-Integrated Circuit communication is 3 Axes viz., X, Y & Z in Magnetometer, with a
configurable to the I2C slave address. measurement range of ±4800µT
Operating voltage 2.5V– 3.5 V 2.5V–3.5V
Temperature sensor Accuracy upto ± 1°C Accuracy upto ± 1°C
824 J SCI IND RES VOL 83 AUGUST 2024

Mathematical Analysis 𝑿𝒉 Calculation

This implementation is crucial in enhancing spatial 𝑋 is the horizontal component of the magnetic field
awareness by utilizing the MPU-6050 and MPU-9250 in the x-y plane after correcting for pitch and roll.
sensor data. By extracting Pitch, Roll, and Yaw
angles, the code enables a nuanced understanding of 𝑋ℎ 𝑚𝑎𝑔𝑋 𝑐𝑜𝑠 𝑝𝑖𝑡𝑐ℎ 𝑚𝑎𝑔𝑌 𝑠𝑖𝑛 𝑟𝑜𝑙𝑙
orientation in three-dimensional space. Its versatility 𝑠𝑖𝑛 𝑝𝑖𝑡𝑐ℎ 𝑚𝑎𝑔𝑍 𝑐𝑜𝑠 𝑟𝑜𝑙𝑙
makes it invaluable for applications ranging from 𝑠𝑖𝑛 𝑝𝑖𝑡𝑐ℎ
immersive virtual reality experiences to precise Yaw Calculation
navigation systems, where accurate motion tracking The yaw is the angle around the z-axis. It is the
and orientation detection are paramount. Integrating angle between the projection of the magnetic vector
accelerometer and magnetometer data ensures a onto the x-y plane and the x-axis.
robust and reliable solution, making this code a
cornerstone for projects demanding high-precision 𝑦𝑎𝑤 𝑎𝑟𝑐𝑡𝑎𝑛2 𝑌 , 𝑋
spatial information. Below is the code used to Further, the calculated pitch, roll, and yaw angles
implement Pitch, Roll, and Yaw angles using are converted from radians to degrees.
Accelerometer and Magnetometer sensor data from
MPU-6050 and MPU-9250 sensors. 𝑟𝑜𝑙𝑙 𝑟𝑜𝑙𝑙 57.3
𝑝𝑖𝑡𝑐ℎ 𝑝𝑖𝑡𝑐ℎ 57.3
𝑝𝑖𝑡𝑐ℎ 𝑎𝑟𝑐𝑡𝑎𝑛2 𝑎𝑐𝑐𝑒𝑙𝑌, 𝑎𝑐𝑐𝑒𝑙𝑋 𝑎𝑐𝑐𝑒𝑙𝑍 𝑦𝑎𝑤 𝑦𝑎𝑤 57.3
𝑟𝑜𝑙𝑙 𝑎𝑟𝑐𝑡𝑎𝑛2 𝑎𝑐𝑐𝑒𝑙𝑋, 𝑎𝑐𝑐𝑒𝑙𝑌 𝑎𝑐𝑐𝑒𝑙𝑍 In summary, these formulas are used for sensor
𝑌ℎ 𝑚𝑎𝑔𝑌 𝑐𝑜𝑠 𝑟𝑜𝑙𝑙 𝑚𝑎𝑔𝑍 𝑠𝑖𝑛 𝑟𝑜𝑙𝑙 fusion, specifically for calculating pitch, roll, and yaw
𝑋ℎ 𝑚𝑎𝑔𝑋 𝑐𝑜𝑠 𝑝𝑖𝑡𝑐ℎ 𝑚𝑎𝑔𝑌 𝑠𝑖𝑛 𝑟𝑜𝑙𝑙 angles from accelerometer and magnetometer
𝑠𝑖𝑛 𝑝𝑖𝑡𝑐ℎ 𝑚𝑎𝑔𝑍 𝑐𝑜𝑠 𝑟𝑜𝑙𝑙 measurements, taking into account the orientation of
𝑠𝑖𝑛 𝑝𝑖𝑡𝑐ℎ the sensor. The resulting angles are often used in
𝑦𝑎𝑤 𝑎𝑟𝑐𝑡𝑎𝑛2 𝑌ℎ , 𝑋ℎ applications such as orientation tracking in robotics or
𝑟𝑜𝑙𝑙 𝑟𝑜𝑙𝑙 57.3 mobile devices.
𝑝𝑖𝑡𝑐ℎ 𝑝𝑖𝑡𝑐ℎ 57.3
Employed Model
𝑦𝑎𝑤 𝑦𝑎𝑤 57.3
The implementation process was initiated with the
Functions:- Further breaking down each part of design phase, followed by a thorough evaluation of
the formula and explaining its purpose is discussed the software and hardware requirements in
below: accordance with the design specifications. Once all
components were in place, separate integration
Pitch Calculation processes for the software and hardware systems were
The pitch is the angle around the x-axis and is executed. A rigorous verification process was
calculated using the arctangent function. It is the conducted after successful implementation to ensure
angle between the y-axis and the projection of the the systems operated as intended. Upon completing
vector onto the x-z plane. the evaluation and readiness of personal equipment,
the final step involved the comprehensive integration
𝑝𝑖𝑡𝑐ℎ 𝑎𝑟𝑐𝑡𝑎𝑛2 𝑎𝑐𝑐𝑒𝑙𝑌, 𝑎𝑐𝑐𝑒𝑙𝑋 𝑎𝑐𝑐𝑒𝑙𝑍
of software and hardware systems into a unified
Roll Calculation operational framework.
The roll is the angle around the y-axis, calculated To facilitate a clear and structured implementation,
similarly to pitch. It is the angle between the x-axis a block diagram outlining the entire working
and the projection of the vector onto the y-z plane. procedure was developed, as depicted in Fig. 6. In this
diagram, the flow of data from the gyroscope (input)
𝑟𝑜𝑙𝑙 𝑎𝑟𝑐𝑡𝑎𝑛2 𝑎𝑐𝑐𝑒𝑙𝑋, 𝑎𝑐𝑐𝑒𝑙𝑌 𝑎𝑐𝑐𝑒𝑙𝑍 through the Arduino Uno and ESP32 (process) to the
servo motor (output) is illustrated. The transmitter end
𝒀𝒉 Calculation
𝑌 is the horizontal component of the magnetic comprises a gyroscope interfacing with an Arduino
field in the y-z plane after correcting for roll. Uno, where the motion data is processed. The
processed data is then transmitted to an ESP32,
𝑌ℎ 𝑚𝑎𝑔𝑌 𝑐𝑜𝑠 𝑟𝑜𝑙𝑙 𝑚𝑎𝑔𝑍 𝑠𝑖𝑛 𝑟𝑜𝑙𝑙 serving as the central processing unit. On the receiver
 AGRAWAL & KUMAR: IOT BASED JOINT SYNCHRONIZED COLLABORATIVE HUMANOID ARM 825

Fig. 6 — Block diagram of system workflow

end, another Arduino Uno receives the data from the

ESP32 and controls the servo motor, ensuring
accurate and coordinated movements.
In Fig. 7, the detailed schematic of the connections
required for optimal performance of the robotic hand
is shown. This figure illustrates the connections
between the ESP32, TCA9548A multiplexer, and
multiple MPU-9250 sensors, where Arduino Uno is
not used as the data from various sensors are collected
by TCA9548A and sent to ESP32 for further process
of transmission directly, using IoT. The ESP32
processes signals from the MPU-9250 sensors via the Fig. 7 — Connection schematic for optimal robot hand performance
TCA9548A, which acts as a communication interface.
The servo motor, equipped with built-in Wi-Fi
communication, receives these processed signals and
commands the DC motor to execute precise
movements. This setup ensures that the robotic hand
can accurately mimic the motions of the human hand
by translating sensor data into actuator commands.
The MPU-9250 sensor functions as the central
component for motion control in the robotic hand,
with the robot's movements being managed through
this sensor. The MPU-9250, acting as the controller,
interfaces with the ESP32 servo driver and
TCA9548A multiplexer. Communication between the
robot's motion control and the hand's movements is
Fig. 8 — Machine learning algorithms
facilitated by the TCA9548A. The ESP32,
functioning as a servo driver, processes the signals capabilities of the humanoid robotic left arm. One key
received from the MPU-9250 and controls the servo algorithm employed is Inverse Kinematics, which
motors accordingly. This methodological approach calculates the joint parameters required to achieve a
highlights the integration of advanced sensor desired end effect or position, as discussed in Fig. 8.
technology, robust communication interfaces, and This algorithm enables precise control of the robot’s
precise actuator control to achieve high-performance, movements, particularly in collaborative tasks with
coordinated movements in robotic applications. human joints, ensuring fluid and accurate motions.20,27
The utilization of Reinforcement Learning further
Considered Machine Learning Algorithms augments the robot’s adaptability in high-risk
In this paper, the integration of machine learning environments. Through trial and error, the robot learns
algorithms plays a pivotal role in enhancing the optimal actions based on environmental feedback,
826 J SCI IND RES VOL 83 AUGUST 2024

refining its decision-making process and enhancing to process complex sensory inputs, recognize patterns,
overall performance during complex operations. and make informed decisions, contributing to its
Reinforcement Learning (RL)3,20 is a powerful efficiency in responding to environmental changes.
machine learning paradigm suitable for training The synergistic application of these machine-learning
agents to make decisions in an environment through algorithms reflects a sophisticated approach, enabling
trial and error. In humanoid robotic arms, RL could the humanoid robotic left arm to navigate intricate
play a crucial role in training the robotic arm to tasks collaboratively and autonomously, marking a
perform tasks in high-risk situations in collaboration significant advancement in human-robot interaction
with human movements. The methodology for the and robotic capabilities.
reinforcement learning is discussed below,
Data Transmission
 Training for Coordination: RL can be used to train
For data transmission, the preceding steps involve
the robotic arm to coordinate its movements with
initial data acquisition from IMU sensors and angular
the human joints. The algorithm can learn from the
and linear motion data capture. The captured
movements of a human arm in various situations
measurements are then transmitted to the ESP32 node
and adapt its actions to work collaboratively in
MCU. From there, data is relayed to the Arduino
high-risk environments.
UNO via Universal Asynchronous Receiver and
 Adaptive Responses: In high-risk scenarios, the Transmitter (UART) communication for further
environment may change dynamically. RL allows processing and control of the robotic arm.
the robotic arm to adapt its responses based on UART is a serial communication protocol to
feedback. For example, if the arm senses a transmit and receive data between electronic devices.
deviation or disturbance, the RL algorithm can The robotic arm uses an ESP-32 Node MCU, a
adjust the control signals to maintain stability. development board with an ESP-32 microcontroller,
 Learning from Human Demonstrations: RL can and its primary function is enhancing wireless
incorporate learning from human demonstrations. capabilities using Wi-Fi and Bluetooth. Generally, it
By observing and imitating human movements, the is suitable for IoT projects and Prototyping. ESP32 is
robotic arm can acquire practical skills and compatible with Arduino, which implies that it can be
strategies in high-risk situations, enhancing its programmed using Arduino IDE, having pertinent
ability to assist in emergencies. built-in flash memory to store program code and data.
 Real-time Decision Making: RL algorithms can The data is then sent from the ESP-32 MCU node
facilitate real-time decision-making for the robotic to the Server driver using TCDA 9548A-I2C
arm. In unpredictable or hazardous scenarios, the multiplexer, allowing it to communicate with up to 8
ability to make adaptive decisions based on the devices via wireless communication protocols shown
current environment is crucial. RL enables the in Fig. 9. Four Servo Motors are installed in the
robotic arm to learn and optimize its actions on the humanoid robotic hand to achieve various degrees of
fly. motion; hence, 4 degrees of freedom are used in one
 Risk Mitigation: RL can contribute to risk hand of the robot.
mitigation by allowing the robotic arm to learn safe
and efficient strategies. Through a reward-based
system, the algorithm can guide the arm to perform
actions that minimize the risk of collateral damage
and enhance the safety of emergency service crews.
Reinforcement learning is a potential candidate for
training and optimizing the humanoid robotic right
arm to operate effectively in collaboration with
human joints, especially in high-risk environments,
which is done for designing this model.
Deep Learning constitutes another essential facet,
leveraging neural networks to empower the robot with
advanced perception and decision-making abilities.
Integrating deep learning algorithms allows the robot Fig. 9 — Data transmission from Arduino UNO to ESP-32
 AGRAWAL & KUMAR: IOT BASED JOINT SYNCHRONIZED COLLABORATIVE HUMANOID ARM 827

Results and Discussion augmenting overall operational efficiency and

Our experiments demonstrated how the integration effectiveness.
of hardware components, including motors, servo  A critical aspect of our experimentation was the
drivers, and IMU sensors, led to the achievement of strategic deployment of sensors, notably the MPU-
precise and coordinated movements of a robotic arm. 9250. These sensors exhibited Remarkable
This integration played a crucial role in validating our responsiveness, enabling rapid detection of
hypothesis, emphasizing the effectiveness of environmental changes and deviations. This
combining computational intelligence with physical dynamic adaptation capability was instrumental in
machinery. Notably, movements were smoothly ensuring the agility and adaptability of the
executed by the robotic hand, showcasing this humanoid robot, particularly in navigating complex
integrated approach's success. and unpredictable terrains.
Several significant findings were yielded by our A significant step forward in the practical
investigation into humanoid-robot collaboration, realization of collaborative and assistive robotics is
particularly in high-risk environments: represented in our research. By unraveling the
complexities of humanoid robot collaboration and
 Significantly enhanced movement precision and leveraging cutting-edge technologies, the groundwork
coordination were achieved through the seamless for future advancements in this burgeoning field has
integration of hardware components. This been laid. The robotic arm was seamlessly
integration enabled the robotic system to adapt to communicated with the human arm, integrated with
changing conditions dynamically, showcasing a sensors, through IoT, enabling successful and
level of sophistication previously unattained in coordinated movements in all directions – up, down,
humanoid robotics. right, and left, as demonstrated in Figs. 10 to 12.
 The understanding of effective collaboration
between humanoid robots and humans, especially
in challenging contexts, was advanced by our
research. Insights into potential applications and
associated challenges were gained by dissecting the
nuances of this collaboration, thereby informing
future deployment strategies.
 The emergence of IoT technologies as a
foundational element of our experimentation was
observed. A robust communication framework was
established between human operators and robotic
arms, facilitating synchronized movements and

Fig. 11 — Left hand up motion

Fig. 10 — Left hand down motion Fig. 12 — Left hand side motion
828 J SCI IND RES VOL 83 AUGUST 2024

2 Khaleghi L, Artan U, Etemad A & Marshall J A, Touchless

control of heavy equipment using low-cost hand gesture
recognition, IEEE Internet Things Mag, 5 (2022) 54–57.
3 Dutta N, Saha J, Sarker F & Zaman H U, A novel design of a
multi-DOF mobile robotic helping hand for paralyzed patients, Int
Conf Adv Comput Commun Informatics (IEEE) 2018, 2219–2225.
4 Maurice P, Padois V, Measson Y & Bidaud P, Human-
oriented design of collaborative robots, Int J Ind Ergon, 57
(2017) 88–102.
5 Agrawal A K & Bharti M, Feature, technology, application,
and challenges of internet of things, in Big Data, IoT, and
Machine Learning (CRC Press) 2020, 255–276.
6 Agrawal A K, Pidge P, Bharti M, Dev M P & Kedare P K,
Innovations and future of robotics, in Advances in Manufacturing
and Industrial Engineering: Select Proceedings of ICAPIE
Fig. 13 — Sensor placement in the hand (Springer Singapore) 2021, 975–980.
7 Sugihara T, Nakamura Y & Inoue H, Real-time humanoid
Additionally, the sensor placement in the human motion generation through ZMP manipulation based on inverted
hand, which sends data to the collaborative robot pendulum control, IEEE Int Conf Robotics Automation, 2 (2002)
through IoT, is illustrated in Fig. 13. 1404–1409.
In our research, we stressed how crucial human 8 Bharti M & Agrawal A K, War field spy and fighter robot,
thinking is. Nothing can replace the human brain. But Recent Adv Mech Eng: Select Proc NCAME 2019 (Springer
Singapore) 2020, 565–574.
we showed that using robots controlled by people 9 Singh K D & Singh P D, Machine learning in robotics with
from safe places during dangerous situations makes fog/cloud computing and IoT, EAI Endorsed Trans AI Robotics,
things safer. These robots can handle heavy loads and 2 (1) (2023).
go into areas with hazardous gases, protecting human 10 Zhang T & Nakamura Y, Humanoid robot RGB-D SLAM in
lives better. the dynamic human environment, Int J Humanoid Robotics,
17 (2020) 2050009.
Conclusions 11 Satheeshkumar R, Real time robotic arm control using human
hand gesture measurement, J Adv Res Dyn Control Syst, 12
In conclusion, this research has successfully (2020) 984–996.
enhanced the precision and responsiveness of a 12 Tzafestas S G, Synergy of IoT and AI in modern society: The
robotic arm in hazardous environments by integrating robotics and automation case, Robotics Autom Eng J, 3
IoT technology with sensor data and mathematical (2018) 00118–00132.
calculations. The findings highlight significant 13 Sato T, Nishio H & Hosaka H, New IoT world created by
potential for applications in emergency response and ubiquitous computing and robotics, J Robotics Soc Jpn, 35
(2017) 88–92.
other critical operations requiring precise and reliable 14 Kang X & Teng Y, Intelligent control system design of ring
robotic assistance. However, limitations were light via IoT mobile robot, J Robotics, 2023(1) (2023) 1–13.
encountered, such as high computational demands and 15 Rustam I, Md Tahir N, Mohd Yassin A I, Wahid N & Kassim
challenges in real-time data processing. Future A H, Linear differential driven wheel mobile robot based on
research should optimize these calculations to reduce MPU9250 and optical encoder, TEM J, 11(1) (2022) 30–36.
computational requirements and improve system 16 Mohd Yusoff Z, Nordin S A, Markom A M & Mohammad N
N, Wireless hand motion controlled robotic arm using flex
robustness in diverse and unpredictable conditions. sensors, Indonesian J Electr Eng Comput Sci, 29 (2022) 133.
Further exploration into enhancing the human-robot 17 Hopkins M A, Leonessa A, Lattimer B Y & Hong D W,
interaction interface is recommended to ensure Optimization-based whole-body control of a series elastic
seamless and intuitive operation. Despite these humanoid robot, Int J Humanoid Robotics, 13 (2016) 1550034.
challenges, the study demonstrates a promising 18 Erden M S, Leblebiciog lu K & Halici U, Multi-agent system-
based fuzzy controller design with genetic tuning for a mobile
direction for improving operational safety and manipulator robot in the hand over task, J Intell Robotic Syst,
efficiency in high-risk scenarios through the 39 (2004) 287–306.
development of IoT-enabled robotic systems. 19 Dou R, Yu S, Li W, Chen P, Xia P, Zhai F, Yokoi H & Jiang Y,
Inverse kinematics for a 7-DOF humanoid robotic arm with joint
References limit and end pose coupling, Mech Mach Theory, 169 (2022)
1 Parvin R, Al Adnan R, Uddin M R & Hasan M, Human- 104637.
machine interface (HMI) robotic arm controlled by 20 Cho B-K & Oh J-H, Dynamic balance of a hopping humanoid
gyroscopically acceleration, 4th Int Conf Electr, Comput robot using a linearization method, Int J Humanoid Robotics,
Telecommun Eng (IEEE) 2022, 1–4. 09 (2012) 1250020.
 AGRAWAL & KUMAR: IOT BASED JOINT SYNCHRONIZED COLLABORATIVE HUMANOID ARM 829

21 Xing D & Su J, Motion generation for the upper body of 25 Okamura A M, Haptic dimensions of human-robot
humanoid robot, Int J Humanoid Robotics, 07 (2010) 281–294. interaction, ACM Trans Human-Robot Interact, 7 (2018)
22 Oztop E, Franklin D W, Chaminade T & Cheng G, Human- 1–3.
humanoid interaction: Is a humanoid robot perceived as a 26 Ma L M, Ijtsma M, Feigh K M & Pritchett A R, Metrics for
human?, Int J Humanoid Robotics, 02 (2005) 537–559. human-robot team design: A teamwork perspective on
23 Ozer T, Agir I & Henry C S, Rapid prototyping of ion- evaluation of human-robot teams, ACM Trans Human-Robot
selective electrodes using a low-cost 3D printed Internet-of- Interact, 11 (2022) 1–36.
Things (IoT) controlled robot, Talanta, 247 (2022) 123544. 27 Williams T, Briggs P & Scheutz M, Covert robot-robot
24 Saxena S, Gupta V K, Hrisheekesha P N, Singh R S & Singh communication: Human perceptions and implications
V, Aimed robot: Autonomous IoT controlled medical for human-robot interaction, J Human-Robot Interact, 4
assistance robot, Coronaviruses, 02(9) (2021) 1–5. (2015) 23.