Vision Transport

In this section, miniAuto will use the ESP32-S3 vision module to recognize the
red line and follow it. During the line following, if the green block is recognized,
it will transport the block.

1. Program Flowchart

2. ESP32-S3 Vision Module

ESP32-S3 development board integrates an ESP32 chip and a camera

module. After being inserted into the expansion board, it uses the I2C
communication interface to read color and face data through I2C
communication.
ESP32-S3 is equipped with an OV2640 camera module, which transmits the

1
captured image data to the ESP32 chip via a serial port. The ESP32 chip can
process the image and transmit it to other devices through I2C communication.

3. Program Download

3.1 ESP32-S3
1) According to “5.3 Live Camera Feed/01 Live Camera Feed”, flash the
firmware located in “02 Program File/01 ESP32-S3 Line Following
Program/line_patrol.bin” in the same directory as this section to the ESP32-S3
vision module.

2) After flashing, disconnect the Type-C cable. Turn on the miniAuto to run the
program.

3.2 Arduino UNO

Note:

 The Bluetooth module must be removed before downloading the program, otherwise

the program download will fail due to serial interface conflict.

 Please turn the switch of the battery box to “OFF” when connecting the Type-B

download cable in case the download cable touches the power pin of the expansion board

by mistake, which may cause a short circuit.

1)Locate and open “02 Program File/02 UNO Vision Transport/clear_roadblock”

program file in the same dictionary as this section.

2)Connect the Arduino to the computer with the UNO cable (Type-B).
3)Click the “Select Board”, and the software will automatically detect current
Arduino serial interface. Then click to connect.

2
4)Click to download the program to the Arduino, and then wait for the
download to complete.

4. Program Outcome
After being powered on, miniAuto will recognize the red line and follow it. If
detecting a green block during the line following, it will transport the block.

5. Program Analysis
The name of this program is “clear_roadblock.ino”. The implementation logic of
the program can be referred to the following flowchart:

3
5.1 Import Library File
Import the necessary timer, tracking action, servo control, and ESP-S3 vision
module communication header files for the game.

5.2 Define Pin and Create Objects

The timer, servo and vision module objects are defined. The servo pin is also
defined. The line to be followed is set to red, with the green block to be
transported. Then, coordinate point variables and variables for controlling the
robot’s linear velocity, angular velocity, and motion angle are defined.

4
A variable for line following calibration is defined to record the position of the
block. This facilitates judgment and processing when following line. Define line
following coordinate variable for storing the coordinate values of the line. The
obstacle coordinate parameter stores the block coordinate.

5.3 Initial Settings

1)Initialize related hardware equipment in “setup()” function. The first is serial
interface, which sets the baud rate of communication to 115200.

2)Initialize each module. Timer1 starts counting with a single time unit of 20ms.

5
1. “ticksFuncSet(ticksGetFunc)”: Binds the system's preset function
“ticksGetFunc” which is for obtaining the current system time to the
“platFormTicksFunction” function. The “platFormTicksFunction” function is
called to obtain current system time.
2. “Motors_Initialize()”: Initializes the motors.
3. “hw_cam.Init()”: Initializes the ESP32-S3 vision module.
4. “Controller_Init()”: Initializes the computing unit for the robot’s motion control
parameters.
5. “myServo.attach(servoPin)”: Initialize PWM servo.
6. “set_servo(60)”: Set the servo angle to 60.
7. “timerStart(&timer1, 60000, timerPIDCalculateCallBack, "60ms cycle PID
calculate")”: Timer1 starts working. The callback function
“timerPIDCalculateCallBack” is triggered and called 60ms later. It passes
"60ms cycle PID calculate" as a parameter to the callback function.

5.4 Loop Call Sub-function

After initialization is complete, the program enters the “loop” function. The
"timersTaskRunning" function is called in a loop. It traverses and processes the
timers in the timer list that have reached their trigger time. The interrupt
functions are executed.
Then, the “Velocity_Controller(set_angle, set_speed, set_rot,
SIMULATE_PWM_CONTROL)” function is called to control the robot's
movement speed.

6
5.5 Timer Callback Function “timerPIDCalculateCallBack”
The function is used to determine if the robot has detected the target color
position. This is achieved by reading the vision module's line following function
register data via the “Linepatrol_Data_Receive” function. Then, calculate the
offset of the X-axis.

Determine the recognized color. If only the line is recognized, perform line
following and save the coordinates of the line following to a variable.

If an obstacle is detected during the line-following process, the robot will stop
line following. Start Timer2 to transport the block.

If the robot is currently in the transport state, it may deviate from the line. In this
case, position correction is required to bring the robot back to the line.

7
5.6 Timer Monitoring Function “timersTaskRunning”
The timer list is designed as a linked list by default. You can traverse and
add/remove members of the timer list via two pointers “pTimerList” and “entry”.
When a timer reaches its trigger time, the timer list considers it invalid and
removes it from the list. Therefore, if you want a timer to work repeatedly, call
the “timerStart” function again or multiple times in the callback function.

5.7 Line Following Callback Function

“timerPositionCalibrationCallBack”
The “Linepatrol_Data_Receive” function obtains color information recognized
by the vision module.

8
 If red is recognized, the timer1 is restarted to continue line following.

Otherwise, the line following will be executed. It involves checking the position
of the saved block and setting corresponding linear velocity, angular velocity,
and motion angle. Next, restart the line following until the line is recognized
again.

5.8 Motor Initialization Function

This function is used to initialize the motor pins. Call the “pinMode” function in
a “for” loop to set the motor IO port to output mode. Then, the
“Velocity_Controller” function is called to set the robot to stop.

9
5.9 Robot Velocity Control Function

The function sets the speed of each wheel of the robot. By analyzing the
kinematics of the robot's Mecanum wheels, the speed of each wheel can be
calculated. The calculated speed variables can be used as input parameters
for the “Motors_Set” function to adjust the motor speed and control the overall
movement of the robot. Please refer to relevant tutorials to learn about
kinematic analysis.

5.10 Motor Setting Function

This function adjusts the speed and direction of each wheel of the robot based
on the input parameter values. In the “for” loop, the robot's steering is adjusted
based on the calculated results. The “map” function maps the calculated
values from “0 to 100” to the range of “pwm_min to 255”. Next, the “digitalWrite”
and “analogWrite” functions sets the motor direction and speed.
10
6. FAQ
Q1:Since the code was upload, the color can not be recognized.
A:Please check that you have connected the 4-pin cable to the I2C interface.
Q2:Why is the color recognized by the camera inaccurate or even incorrect?
A:Please try to minimize the complexity of the background. You can use a
single color or a simple environment as the background.

11