1. Project Overview
2. System Requirements
3. Installation
4. Project Setup
5. Running the System
6. Project Structure
7. Troubleshooting
8. Additional Documentation

This project is a comprehensive Data Distribution Service (DDS) implementation using Fast-DDS middleware. It provides a complete solution for secure, real-time data distribution across multiple domains with support for publishers, subscribers, monitoring, and web-based visualization.

The system includes:

• IDL-based type definitions and code generation
• Secure DDS communication with PKI authentication and encryption
• Multi-domain support for topic isolation
• Real-time monitoring and data collection
• Web-based dashboard for visualization
• Scenario-based testing with JSON data injection
• Cross-platform portability

! IMPORTANT: Currently, development and automated setup scripts are primarily focused on Linux systems.

• Linux: Fully supported with AppImage (recommended) or manual setup scripts
• Windows: Native support planned for future releases. Currently, manual installation required or use WSL 2 for Linux compatibility
• macOS: Not currently supported. macOS support is planned for future releases. Manual installation and setup may be possible but is not tested or documented.

• Linux (Ubuntu 20.04 or higher recommended, Debian-based distributions supported) - ✅ Currently fully supported
• Windows (with WSL 2 recommended for Linux compatibility) - ⚠️ Windows native support coming soon
• macOS - ❌ Not currently supported (planned for future releases)

• CPU: Multi-core processor (2+ cores recommended)
• RAM: Minimum 4GB (8GB+ recommended for development)
• Disk Space: Minimum 2GB free space
• Network: Ethernet or WiFi connection (UDP multicast support required)

• Bash shell (version 4.0 or higher)
• Git (for cloning the repository)
• sudo access (for package installation)

• CMake version 3.10 or higher
• C++ compiler with C++17 support:
  • GCC 7.0 or higher (Linux)
  • Clang 5.0 or higher (Linux/macOS)
  • MSVC 2017 or higher (Windows)
• Make or Ninja build system

• Fast-DDS libraries (libfastdds-dev)
• Fast-CDR libraries (libfastcdr-dev)
• OpenSSL development libraries (libssl-dev)
• Standard C++ runtime libraries

• Python 3.6 or higher (Python 3.8+ recommended)
• Java JDK 11 or higher (for fastddsgen)
• Node.js 14.0 or higher (for demo dashboard)
• npm 6.0 or higher (comes with Node.js)

• libtinyxml2-dev
• libfoonathan-memory-dev
• libasio-dev
• libboost-dev
• python3-tk (for QoS patcher GUI)

• UDP multicast support (required for DDS communication)
• Firewall configuration to allow UDP multicast traffic
• Internet connection (for map tiles in demo dashboard, optional)

Clone the project repository to your local machine:

git clone https://github.com/mercethem/DDS
cd DDS

! NOTE: The AppImage and setup scripts are currently designed for Linux systems only. Windows native support is planned for future releases.

For Linux users, you have two options:

1. Download AppImage from GitHub releases (DDS-v1.0.0_beta-x86_64.AppImage)
2. Place it anywhere (Desktop, Downloads, etc.) - it's completely self-contained!
3. Make executable: chmod +x DDS-v1.0.0_beta-x86_64.AppImage
4. Run: ./DDS-v1.0.0_beta-x86_64.AppImage
   • Setup runs automatically on first run if needed
   • Tests and demo start automatically

See Quick Start: Run Tests and Demo section for detailed AppImage usage.

You can set up the project manually using the provided scripts:

# Install system dependencies
bash init/sh/install_system_dependencies.sh

# Run project setup
bash init/sh/project_setup.sh

# Run tests and demo
bash init/sh/run_tests_and_demo.sh

See Project Setup section for detailed manual setup instructions.

Run the automated dependency installer:

bash init/sh/install_system_dependencies.sh

This script will:

• Update system packages
• Install build tools (CMake, GCC, Make)
• Install Python 3 and Java JDK
• Install required libraries (OpenSSL, Fast-DDS, etc.)
• Configure environment variables
• Automatically run post-install build (Fast-DDS manual installation and monitoring application build via init/sh/post_install_build.sh)

! IMPORTANT: Do not run the installer script as root. It will request sudo privileges automatically when needed.

If the automated installer does not work or you prefer manual installation:

# Update system packages
sudo apt update

# Install build tools
sudo apt install -y build-essential cmake git pkg-config curl wget unzip

# Install Python and Java
sudo apt install -y python3 python3-pip python3-tk python3-dev openjdk-11-jdk

# Install library dependencies
sudo apt install -y libssl-dev libtinyxml2-dev libfoonathan-memory-dev libasio-dev libboost-dev

# Install Fast-DDS (Ubuntu 20.04+)
sudo apt install -y libfastcdr-dev libfastdds-dev fastddsgen

# Configure environment variables
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
export PATH=$PATH:$JAVA_HOME/bin
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

# Update library cache
sudo ldconfig

Install equivalent packages using your distribution's package manager. Fast-DDS may need to be built from source if packages are not available.

! NOTE: macOS is not currently supported. The AppImage and setup scripts are designed for Linux only. macOS support is planned for future releases.

If you want to attempt manual installation on macOS (not recommended, untested):

• Install Homebrew package manager
• Install build tools: brew install cmake git
• Install Python 3: brew install python3
• Install Java JDK 11: brew install openjdk@11
• Build Fast-DDS from source (no pre-built packages available)
• Install Node.js: brew install node
• Manually run setup scripts (may require modifications)

Recommended: Use Linux (native or via VM) or wait for official macOS support.

! NOTE: Windows native support is planned for future releases. Currently, manual installation required or use WSL 2 for Linux compatibility.

Install dependencies manually:

• Install Visual Studio 2017 or higher with C++ support
• Install CMake
• Install Python 3
• Install Java JDK 11
• Build Fast-DDS from source or use pre-built binaries
• Install Node.js

Recommended: Use WSL 2 (Windows Subsystem for Linux) to run Linux scripts, or wait for official Windows support.

Verify that all required tools are installed:

# Check CMake
cmake --version

# Check C++ compiler
g++ --version

# Check Python
python3 --version

# Check Java
java -version

# Check Fast-DDS generator
fastddsgen -version

# Check Node.js (for demo)
node --version
npm --version

All commands should return version information without errors.

! CRITICAL: If any tool is missing or shows an error, installation is incomplete. Resolve missing dependencies before proceeding.

Add environment variables to your shell configuration file:

# Add to ~/.bashrc (Linux) or ~/.zshrc (macOS)
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
export PATH=$PATH:$JAVA_HOME/bin
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH

Reload your shell configuration:

source ~/.bashrc

Or restart your terminal.

After installing dependencies, you must set up the project. The setup process configures certificates, generates code, applies patches, and builds executables.

Run the automated setup script:

bash init/sh/project_setup.sh

This script performs all setup steps automatically:

• Runs the complete DDS workflow
• Creates security certificates
• Generates IDL code
• Applies code patches
• Builds all IDL modules
• Builds monitoring application
• Verifies setup completion

! IMPORTANT: The setup script must be run from the project root directory. It will take several minutes to complete depending on your system performance.

If you prefer manual setup or need to troubleshoot, follow these steps in order:

Configure the project environment and detect tools:

cd scripts/sh
bash setup_environment.sh

This creates environment configuration files (export_dds_environment.sh and setup_dds_aliases.sh) in the init/sh directory.

Generate C++ code from IDL files:

cd scripts/sh
bash generate_idl_code.sh

This script:

• Scans all IDL files in the IDL/ directory
• Runs fastddsgen to generate C++ code
• Creates *_idl_generated/ directories for each module
• Downloads JSON library if missing

! CRITICAL: IDL generation must complete successfully before proceeding. Check for errors in the output.

Update domain IDs in generated code based on IDL file comments:

cd scripts/sh
bash update_domain_ids.sh

This ensures that generated applications use the correct DDS domain IDs specified in IDL files.

Create security certificates and apply security configuration:

cd scripts/sh
bash setup_security_certificates.sh

This script:

• Creates Root CA certificate if missing
• Creates PC-specific participant certificates
• Generates security policy documents
• Applies security patches to generated code

! IMPORTANT: Certificates are PC-specific based on hostname. Each machine needs its own certificates.

Apply code enhancements in the correct order:

5a. IDL Patcher (creates default data blocks):

python3 scripts/py/idl_default_data_patcher.py

5b. JSON Patcher (replaces blocks with JSON reading):

python3 scripts/py/json_reading_patcher.py

! CRITICAL: Patches must be applied in this exact order. Running JSON patcher before IDL patcher will fail.

5c. Security Patcher (adds security configuration):

python3 scripts/py/apply_security_settings.py

5d. Clean Duplicates (removes duplicate code blocks):

python3 scripts/py/clean_duplicate_code.py

5e. CMake Portability Fix (makes CMake files portable):

python3 scripts/py/fix_cmake_rpath.py

Build all IDL modules:

cd scripts/sh
bash build_idl_modules.sh

This script:

• Finds all *_idl_generated/ directories
• Configures CMake for each module
• Builds static libraries and executables
• Reports build results

Build output is placed in IDL/<Module>_idl_generated/build/ directories.

Build the unified monitoring application:

cd monitoring/build_monitoring
bash build_monitoring.sh

This creates the monitor executable at monitoring/build/monitor.

Install Node.js dependencies for the demo dashboard:

cd demo/build_demo
bash build_demo.sh

Or manually:

cd demo
npm install

This installs Express, Socket.IO, and other required packages.

After setup, verify that everything is configured correctly:

# Check for generated code
ls IDL/*_idl_generated/

# Check for certificates
ls secure_dds/CA/mainca_cert.pem
ls secure_dds/participants/$(hostname)/$(hostname)_cert.pem

# Check for built executables
ls IDL/CoreData_idl_generated/build/CoreDatamain
ls monitoring/build/monitor

# Check demo dependencies
ls demo/node_modules/

All checks should show files exist. If any are missing, review the setup steps and check for errors.

For Linux users, there are two ways to run the beta version:

The AppImage is completely self-contained and contains all project files!

1. Download AppImage from GitHub releases:

   • Go to the latest release page
   • Download DDS-v1.0.0_beta-x86_64.AppImage

2. Place AppImage anywhere you want:

   • Desktop
   • Downloads folder
   • Any directory
   • No project directory needed - it's completely independent!

3. Make executable and run:

   chmod +x DDS-v1.0.0_beta-x86_64.AppImage
   ./DDS-v1.0.0_beta-x86_64.AppImage

4. First Run:

   • AppImage extracts project files to ~/.dds-project-runtime automatically
   • PHASE 1: JavaScript/Node.js Setup (Priority)
     • Checks and installs FUSE2 (required for AppImage execution)
     • Checks and installs Node.js (Ubuntu repos or NodeSource)
     • Installs demo npm packages automatically
   • PHASE 2: System Dependencies
     • Automatically checks system dependencies (cmake, gcc, python3, java, Fast-DDS, etc.)
     • Automatically installs missing dependencies (via init/sh/install_system_dependencies.sh)
   • PHASE 3: Fast-DDS and Monitoring Build
     • Automatically runs Fast-DDS manual installation (via init/sh/fastdds_and_npm_auto_install.sh)
     • Automatically builds monitoring application (via monitoring/build_monitoring/build_monitoring.sh)
   • PHASE 4: System Readiness Check
     • Comprehensive check of all components:
       • Build tools (CMake, GCC/G++, Python3, Java)
       • Node.js and npm packages
       • Fast-DDS installation
       • IDL generated modules and executables
       • Monitoring executable
       • Security certificates
     • If any component is missing, runs setup automatically (via init/sh/project_setup.sh)
     • Re-checks system readiness after setup
   • PHASE 5: Start Tests and Demo (Only if system is ready)
     • Tests and demo will NOT start if any component is missing
     • Runs tests and demo automatically only when all components are ready (via init/sh/run_tests_and_demo.sh)
   • Subsequent runs skip setup and run directly if system is ready

Advantages of AppImage:

• ✅ Completely self-contained - contains all project files
• ✅ Portable - can be placed anywhere (Desktop, Downloads, etc.)
• ✅ No installation required - just download and run
• ✅ Automatic dependency installation - checks and installs system dependencies automatically
• ✅ JavaScript-first approach - Node.js and npm packages installed with priority
• ✅ Comprehensive system checks - verifies all components before running tests/demo
• ✅ Safe execution - tests and demo only start when all components are ready
• ✅ Automatic setup - detects and runs setup if needed
• ✅ Independent - no external project directory needed
• ✅ Double-click to run - GUI support
• ✅ Works like an .exe - single file, runs from anywhere

If you prefer to set up manually or want more control:

First, verify if the system is already set up:

# Check if executables exist
ls IDL/*_idl_generated/build/*main 2>/dev/null && ls monitoring/build/monitor 2>/dev/null

If the command shows files, the system is ready. Skip to Step 3.

If files are missing or you see errors, the system needs setup. Proceed with setup first.

If the system is not ready, run the setup scripts:

# Install system dependencies (requires sudo)
bash init/sh/install_system_dependencies.sh

# Run complete project setup
bash init/sh/project_setup.sh

These scripts will:

• Install all system dependencies (requires sudo password)
• Configure environment variables automatically
• Automatically run post-install build (Fast-DDS manual installation and monitoring application build)
• Run complete project setup
• Build all components (IDL modules and monitoring application)
• Verify installation

! IMPORTANT: Scripts should be run as a normal user (not root). They will request sudo privileges automatically when needed.

Wait for the setup to complete successfully before proceeding.

After setup is complete (or if the system was already ready), run tests and start the demo:

bash init/sh/run_tests_and_demo.sh

This script will:

• Start test publishers/subscribers for all IDL modules
• Launch the demo dashboard automatically
• Display real-time DDS data in the web interface

Press Ctrl+C to stop the demo when finished.

Using AppImage (Recommended):

# Download AppImage from https://github.com/mercethem/DDS/releases
chmod +x DDS-v1.0.0_beta-x86_64.AppImage
./DDS-v1.0.0_beta-x86_64.AppImage
# Setup runs automatically on first run if needed

Using Manual Scripts:

# First time setup:
bash init/sh/install_system_dependencies.sh  # Install dependencies
bash init/sh/project_setup.sh                # Setup and build everything
bash init/sh/run_tests_and_demo.sh           # Run tests and demo

# Subsequent runs (system already set up):
bash init/sh/run_tests_and_demo.sh           # Just run tests and demo

Building AppImage from Source:

If you want to build the AppImage yourself:

cd image
bash build.sh  # Single script that does everything
# Output: image/DDS-v1.0.0_beta-x86_64.AppImage

See image/README.md for detailed AppImage build instructions.

Publishers send data via DDS topics. Each IDL module has its own publisher executable.

Navigate to the module's build directory and run:

cd IDL/CoreData_idl_generated/build
./CoreDatamain publisher

Replace CoreData with the desired module name (CoreData2, CoreData3, CoreData4, Intelligence, Messaging).

Publishers run continuously until stopped (Ctrl+C). They will:

• Connect to the DDS domain specified in the IDL file
• Use security certificates for authentication
• Read data from JSON scenario files (if JSON patcher was applied)
• Publish samples at regular intervals

! IMPORTANT: Publishers must use the same domain ID as subscribers to communicate. Domain IDs are specified in IDL files with //domain=X comments.

Subscribers receive data from DDS topics.

cd IDL/CoreData_idl_generated/build
./CoreDatamain subscriber

Subscribers will:

• Connect to the DDS domain
• Subscribe to the topic
• Receive and display data samples
• Run until stopped (Ctrl+C)

The unified monitor subscribes to all topics across multiple domains:

cd monitoring/run_monitoring
bash run_monitoring.sh

Or directly:

./monitoring/build/monitor

Specify domains to monitor:

# Comma-separated domains
./monitoring/build/monitor "0,1,2"

# Range syntax
./monitoring/build/monitor "0-3"

# Mixed syntax
./monitoring/build/monitor "0,2-4,6"

# Environment variable
MONITOR_DOMAINS="0,1,2" ./monitoring/build/monitor

Default behavior monitors domains 0-5 if no specification is provided.

The demo dashboard provides web-based visualization of DDS data.

cd demo/run_demo
bash run_demo.sh

Or directly:

cd demo
node server.js

The server starts on port 3000 by default.

Open a web browser and navigate to:

http://localhost:3000

The dashboard will display:

• Interactive map with aircraft positions
• Real-time data panels
• Intelligence and messaging information
• System status indicators

Configure the dashboard using environment variables:

# Change server port
PORT=8080 node demo/server.js

# Specify monitor domains
MONITOR_DOMAINS="0,1,2" node demo/server.js

# Disable DDS-only mode (enable simulation)
DDS_ONLY=0 node demo/server.js

After starting the dashboard, start publishers in separate terminals:

# Terminal 1: CoreData Publisher
cd IDL/CoreData_idl_generated/build
./CoreDatamain publisher

# Terminal 2: Intelligence Publisher
cd IDL/Intelligence_idl_generated/build
./Intelligencemain publisher

# Terminal 3: Messaging Publisher
cd IDL/Messaging_idl_generated/build
./Messagingmain publisher

Or use the unified monitor:

cd monitoring/run_monitoring
bash run_monitoring.sh

The dashboard will automatically parse monitor output and display data.

! IMPORTANT: The dashboard requires an internet connection for map tiles. If maps do not load, check your internet connection and firewall settings.

For a complete system demonstration:

1. Start the demo dashboard server:

   cd demo/run_demo
   bash run_demo.sh

2. In separate terminals, start publishers:

   # Terminal 1
   cd IDL/CoreData_idl_generated/build
   ./CoreDatamain publisher
   
   # Terminal 2
   cd IDL/Intelligence_idl_generated/build
   ./Intelligencemain publisher
   
   # Terminal 3
   cd IDL/Messaging_idl_generated/build
   ./Messagingmain publisher

3. Access the dashboard in your browser:

   http://localhost:3000
   

4. Observe real-time data visualization on the dashboard.

Understanding the project structure helps with navigation and troubleshooting.

See README.mdd for the complete directory structure diagram.

Problem: Certificate creation fails during setup.

Solution:

1. Verify OpenSSL is installed: openssl version
2. Check write permissions on secure_dds/ directory
3. Ensure hostname is resolvable: hostname
4. Manually create certificates: python3 scripts/py/generate_security_certificates.py or run bash init/sh/project_setup.sh

Problem: fastddsgen fails or produces errors.

Solution:

1. Verify Java is installed: java -version
2. Check fastddsgen is in PATH: which fastddsgen
3. Verify IDL file syntax (check for errors)
4. Check Java version compatibility (JDK 11+ required)
5. Review fastddsgen output for specific error messages

Problem: CMake or compilation fails.

Solution:

1. Verify Fast-DDS is installed: pkg-config --modversion fastdds
2. Check CMake version: cmake --version (3.10+ required)
3. Verify compiler supports C++17: g++ -std=c++17 --version
4. Check library paths: echo $LD_LIBRARY_PATH
5. Clean build directories and rebuild: rm -rf IDL/*/build && bash scripts/sh/build_idl_modules.sh

Problem: No data exchange between publishers and subscribers.

Solution:

1. Verify domain IDs match (check IDL files and generated code)
2. Check security certificates exist and are valid
3. Verify topic names match exactly
4. Check network connectivity (DDS uses UDP multicast)
5. Review firewall settings (UDP multicast must be allowed)
6. Check DDS logs for connection errors

Problem: Dashboard loads but displays no data.

Solution:

1. Verify publishers are running and sending data
2. Check monitor is receiving data (check stdout)
3. Verify WebSocket connection (check browser console)
4. Ensure domain IDs match between publishers and monitor
5. Check server logs for parsing errors
6. Verify internet connection for map tiles

Problem: Security initialization fails with certificate errors.

Solution:

1. Verify certificates exist: ls secure_dds/CA/mainca_cert.pem
2. Check hostname matches certificate directory: hostname
3. Verify certificate validity: openssl x509 -in secure_dds/participants/$(hostname)/$(hostname)_cert.pem -text -noout
4. Recreate certificates if needed: python3 scripts/py/generate_security_certificates.py
5. Check file permissions on certificate files

Problem: Demo dashboard fails to start because port 3000 is in use.

Solution:

1. Find process using port: lsof -i :3000 or netstat -tulpn | grep 3000
2. Kill the process or use a different port: PORT=8080 node demo/server.js

Problem: Python patching scripts fail with errors.

Solution:

1. Verify Python 3 is installed: python3 --version
2. Check script permissions: ls -l scripts/py/*.py
3. Verify required Python modules are available
4. Check script execution order (IDL patcher before JSON patcher)
5. Review error messages for specific issues

If you encounter issues not covered here:

1. Check the detailed documentation in the docs/ directory
2. Review script documentation in docs/scripts/
3. Check error messages carefully for specific guidance
4. Verify all requirements are met
5. Review system logs for additional information

Comprehensive documentation is available:

• version_beta_readme.md: Beta release notes and installation guide for AppImage users
• docs/demo.md: Complete demo dashboard documentation
• docs/IDL.md: IDL system and code generation documentation
• docs/monitoring.md: Monitoring system documentation
• docs/scenarios.md: Scenario-based testing documentation
• docs/secure_dds.md: Security system documentation
• docs/scripts/: Individual script documentation

Each documentation file provides detailed information about its respective component, including architecture, usage, configuration, and troubleshooting.

See the LICENSE file for license information.

For issues, questions, or contributions, please refer to the project repository or contact the development team.