Autonomous Fire Sensor and Human Detection System for First Responders

---

Overview

This repository contains the design, implementation, and testing code for our Senior Capstone Project (June 2024–May 2025): an autonomous sensing platform that detects and locates people in smoke-filled environments to assist first responders during fires.

The system integrates Texas Instruments mmWave cascading radar with an Intel RealSense depth camera to combine RF point cloud data with optical depth imaging. A custom-trained machine learning model identifies human figures and reports their location and count in real time, even in low-visibility conditions.

---

Features

• Dual-Sensor Fusion: Combines RF (mmWave radar) and optical (RealSense depth) data for enhanced detection accuracy.
• Autonomous Human Detection: ML models trained to identify human figures through smoke and environmental noise.
• Real-Time Reporting: Outputs human count and locations for rapid decision-making in emergencies.
• Custom Test Environment: Fog simulation chamber to validate performance in realistic fire-like conditions.
• User Portal: Simple interface for first responders to view detection data live.

---

System Architecture

[ TI mmWave Radar ] ---> RF Point Cloud ----┐
                                            │--> Sensor Fusion --> Human Detection (YOLOv8, Roboflow)
[ Intel RealSense ] --> Depth Images -------┘                         │
                                                                    Output to UI

---

Technologies

Hardware: TI mmWave Cascading Imaging Radar, Intel RealSense Depth Camera
Software: Python, OpenCV, ROS2, YOLO, NumPy, Intel RealSense SDK
ML Frameworks: PyTorch / YOLOv8, Roboflow
Tools: Docker, Git, Fog Simulation Test Rig

---

Repository Structure

📂 docs/               # Documentation, diagrams, and system design files
📂 media/              # Images and Videos of Testing
📂 src/                # Core source code for data acquisition, fusion, and detection
📂 outputs/            # Results obtained from source code
README.md              # Project documentation (this file)
LICENSE                # License information (MIT recommended)

---

Getting Started

Prerequisites

• Python 3.9+
• ROS2 Humble (or compatible version)
• Intel RealSense SDK
• mmWave Radar SDK (TI)
• PyTorch (with CUDA if using GPU)
• OpenCV

Installation

# Clone the repository
git clone https://github.com/YOUR-USERNAME/fire-sensor-human-detection.git
cd fire-sensor-human-detection

# Install Python dependencies
pip install -r requirements.txt

---

Usage

# Run sensor fusion and detection
python src/main.py

The system will:

1. Initialize radar and RealSense streams
2. Perform sensor fusion
3. Run ML-based human detection
4. Output results to the terminal and optional UI

---

Testing

We used a fog simulation chamber to recreate low-visibility conditions.
Testing apparatus is available to see in media/images/ for reproducing results.

---

Team

• Natalia Wilson
• Sameeha Boga
• Daniel Fontaine
• Arya Goyal
• Eve Mooney
• Daniela Salazar

Advisor: Professor Jose Angel Martinez Lorenzo

---

License

This project is licensed under the MIT License — see the LICENSE file for details.