Proprioception Is All You Need: Terrain Classification for Boreal Forests

Proprioception Is All You Need: Terrain Classification for Boreal Forests
Damien LaRocque, William Guimont-Martin, David-Alexandre Duclos, Philippe Giguère, Francois Pomerleau
Paper: https://arxiv.org/abs/2403.16877

This repo contains the source code and the datasets used in our paper Proprioception Is All You Need: Terrain Classification for Boreal Forests that was submitted to the 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024).

Source Code

If you want to use the BorealTC dataset, you can use the simple PyTorch dataset provided in borealtc.py. The BorealTC class offers direct access to the data sequences and their labels. The SlidingWindowDataset class can be used to create a dataset with sliding windows of a fixed size, useful for training models. Here's a simple example of how to use the BorealTC and SlidingWindowDataset classes:

from borealtc import BorealTC, SlidingWindowDataset

# Load the full dataset
dataset = BorealTC("data/borealtc")

# Create sliding window dataset
window_dataset = SlidingWindowDataset(dataset, window_size=170, step_size=50)

# Iterate through the sliding window dataset
for i in range(len(window_dataset)):
    sample = window_dataset[i]
    print(
        f"Window {i}: {sample['window'].shape}, Class: {sample['class_name']}, Run ID: {sample['run_id']}"
    )

Most functions are available in the utils directory :

• utils/preprocessing.py contains the functions for the preprocessing pipeline
• utils/models.py contains the code for the trained models
• utils/frequency.py contains the functions for the multi-channel spectrogram

Installation

We recommend installing the dependencies of the repo in a virtual environment. Two dependencies, mamba-ssm and causal-conv1d require a NVIDIA GPU.

python3.10 -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt

We also provide a Dockerfile and a DockerfileGPU to build a Docker image with all the dependencies.

# Build the Docker image
docker build -t borealtc-gpu -f DockerfileGPU .

# Run the Docker image
docker run --gpus all -e CUDA_VISIBLE_DEVICES=$CUDA_VISIBLE_DEVICES --rm --ipc host \
	  --mount type=bind,source=.,target=/code/ \
	  --mount type=bind,source=/dev/shm,target=/dev/shm \
	  borealtc-gpu python3 main.py

Dataset

The data directory contains two different datasets:

• the BorealTC dataset, our publicly available dataset
• the Vulpi dataset, from the 2021 paper of Vulpi et al. This dataset is originally sourced from their repository Ph0bi0/T_DEEP. The original data is available in the form of .mat files. We converted their data in *.csv files to work in Python.

Each experiment is provided in two files imu_XX.csv and pro_XX.csv, for the IMU and the wheel service data, respectively. Both datasets are organized as the following :

data
├── borealtc
│   ├── CLASS1
│   │   ├── imu_00.csv
│   │   ├── imu_01.csv
│   │   ├── ...
│   │   ├── pro_00.csv
│   │   ├── pro_01.csv
│   │   └── ...
│   └── CLASS2
│       ├── imu_00.csv
│       ├── imu_01.csv
│       ├── ...
│       ├── pro_00.csv
│       ├── pro_01.csv
│       └── ...
└── vulpi
    ├── CLASS1
    │   ├── imu_1.csv
    │   ├── imu_2.csv
    │   ├── ...
    │   ├── pro_1.csv
    │   ├── pro_2.csv
    │   └── ...
    └── CLASS2
        ├── imu_1.csv
        ├── imu_2.csv
        ├── ...
        ├── pro_1.csv
        ├── pro_2.csv
        └── ...

Model hyperparameters and checkpoints

Mamba

Dataset	Data Type	d_state	d_conv	expand	d_model	Norm epsilon	Initial LR	LR drop factor	Reduce LR patience	Max epochs	Minibatch size	Valid patience	Gradient threshold	Focal loss alpha	Focal loss gamma	Checkpoint
Vulpi	IMU Wheel service	56 56	3 3	4 4	56 56	5e-3	5e-3	0.33	4	60	64	8	2	0.75	2	mamba_vulpi.ckpt
BorealTC	IMU Wheel service	16 16	4 3	4 6	32 8	6.3e-6	1.5e-3	0.25	4	60	16	8	None	0.75	2.25	mamba_borealtc.ckpt
Combined	IMU Wheel service	16 16	4 3	4 6	32 8	6.3e-6	1.5e-3	0.25	4	60	16	8	None	0.75	2.25	mamba_combined.ckpt

In the above table, d_state, d_conv, expand and d_model are parameters specific to Mamba. During optimization, each data type branch can have separate parameters.

CNN

The CNN model used the same hyperparameters for both datasets and their combination. Here are the hyperparameters used for the CNN model:

Parameter	Value
Using Hamming Window	True
Initial Learning Rate	5e-3
Learning Rate Scheduler	ReduceLROnPlateau
Scheduler LR Factor	0.1
Scheduler Patience	4
Max Epochs	150
Early Stopping Patience	8
Gradient Clipping Value	6

Acknowledgments

We would like to acknowledge Fabio Vulpi, Annalisa Milella, Roberto Marani and Giulio Reina for their paper Recurrent and convolutional neural networks for deep terrain classification by autonomous robots. This repo started as a fork of Ph0bi0/T_DEEP, their repo, which contains the source code and the dataset of their paper.

Citation

If you use the code or data in an academic context, please cite the following work:

@inproceedings{LaRocque2024,
  title     = {{Proprioception Is All You Need: Terrain Classification for Boreal Forests}},
  url       = {http://dx.doi.org/10.1109/IROS58592.2024.10801407},
  doi       = {10.1109/iros58592.2024.10801407},
  booktitle = {2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  publisher = {IEEE},
  author    = {LaRocque,  Damien and Guimont-Martin,  William and Duclos,  David-Alexandre and Giguère,  Philippe and Pomerleau,  Fran\c{c}ois},
  year      = {2024},
  month     = oct,
  pages     = {11686–11693}
}

License

This project is licensed under a MIT license.