Thanks to visit codestin.com
Credit goes to github.com

Skip to content

samqbush/katabatic-analyzer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

15 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

๐Ÿช Morrison Wind Analyzer

A specialized wind analysis dashboard for wing foiling conditions in Morrison, Colorado. Analyzes morning wind patterns to identify ideal wing foiling sessions based on sustained average wind speeds.

๐ŸŽฏ Quick Start

# Setup
cp .env.example .env  # Add your Ecowitt credentials
pip install -r requirements.txt

# Run dashboard
streamlit run app.py

๐ŸŒช๏ธ Wing Foiling Conditions

  • ๐ŸŸฅ Strong Gear Day (โ‰ฅ20 mph avg): Use smaller wing/board
  • ๐ŸŸจ Perfect Foiling (15-19 mph avg): Ideal sustained winds
  • โฌœ Too Light (<15 mph avg): Below foiling minimum

๐Ÿ“š Documentation

๐Ÿš€ Features

  • Real-time wind analysis with 5-minute data resolution
  • Automatic data collection for today's data when accessed after 8am
  • Wing foiling condition detection based on sustained winds
  • Interactive charts showing time series and daily patterns
  • Gear recommendations for different wind conditions
  • Export functionality for detailed analysis and debugging support
  • Morrison timezone handling for accurate morning hours
  • ๐Ÿ”ฎ Prediction tracking with accuracy analysis and verification
  • ๐ŸŽฏ Model improvement through systematic forecast verification

๐Ÿ“Š Dashboard Sections

  1. Data Status - 7-day data availability overview
  2. Weather Analysis - Current katabatic scoring and conditions
  3. ๐Ÿ”ฎ Prediction Tracking - Forecast accuracy and verification
  4. Key Metrics - Weekly wind event summary
  5. Wind Charts - Time series and daily average patterns
  6. Foiling Analysis - Detailed condition comparison
  7. Data Tables - Events, measurements, and export tools

Built specifically for wing foiling enthusiasts who need reliable morning wind analysis in Morrison, Colorado.
Purpose: Detect and analyze strong wind events during morning hours

Features

  • ๐Ÿ“Š Data Collection: Retrieve wind history from Ecowitt API with 5-minute resolution
  • ๐Ÿ”„ Automatic Collection: Today's data collected automatically when accessed after 8am Morrison time
  • ๐ŸŒช๏ธ Katabatic Detection: Identify wind speeds >15mph between 6-8am local time
  • ๐Ÿ“ˆ Interactive Dashboard: Visual representation of wind patterns and events
  • ๐Ÿ—„๏ธ Local Storage: SQLite database for historical data
  • ๐Ÿ• Timezone Handling: All data normalized to Morrison, CO local time
  • ๐Ÿ“‹ Data Tables: Human-readable tabular data display
  • ๐Ÿ” Gap Management: Detect and fill missing data days

Quick Start

1. Setup Environment

# Clone and navigate to project
cd katabatic-analyzer

# Create and activate virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

2. Configure API Credentials

Use the interactive setup script:

# Run the interactive setup
python setup.py

Or configure manually:

# Copy environment template
cp .env.example .env

# Edit .env file with your Ecowitt API credentials
# Get credentials from: https://www.ecowitt.net/

Required environment variables in .env:

ECOWITT_APPLICATION_KEY=your_application_key_here
ECOWITT_API_KEY=your_api_key_here
ECOWITT_DEVICE_MAC=FF:FF:FF:FF:FF:FF

3. Test Installation

# Run foundation tests
python test_foundation.py

# Test API connection (requires credentials)
python test_api.py

4. Launch Dashboard

# Start Streamlit dashboard
streamlit run app.py

Open http://localhost:8501 in your browser.

Usage

Data Collection

The dashboard provides several data collection options:

  1. Collect Today: Get data for the current day (only available after 8am Morrison time)
  2. Collect Yesterday: Get data for the previous day
  3. Fill Missing Days: Automatically collect data for any missing days in the last week
  4. Custom Date Range: Collect data for a specific date range

Automatic Collection: When you open the dashboard after 8am Morrison time, the system will automatically check if today's data is available and collect it if needed. This ensures you always have current data for wind analysis.

Debug Data Export

For troubleshooting and analysis support, the dashboard includes comprehensive debug export tools:

Dashboard Export

Use the "Export Data" tab in the dashboard for:

  • Quick Export: Standard CSV exports with date ranges and filters
  • Debug Export: Comprehensive multi-file export including:
    • Raw measurements, morning data, and strong wind events
    • Daily summaries and data quality metrics
    • System configuration and database statistics

Command Line Export

# Quick 14-day debug export
python quick_export.py

# Custom debug export with options
python export_debug_data.py --days 30 --output-dir ./debug_exports

The debug export creates 8 files with complete system state for analysis:

  • *_raw_measurements.csv - All wind measurements
  • *_morning_hours.csv - 6-8am data only
  • *_strong_wind_events.csv - Events โ‰ฅ15mph during morning hours
  • *_daily_summaries.csv - Daily statistics and foiling conditions
  • *_data_quality.csv - Data coverage and gap analysis
  • *_config.json - System configuration and analysis settings
  • *_database_stats.json - Database schema and statistics
  • *_summary.json - Export overview and data counts

๐Ÿ’ก Perfect for sharing with support or detailed offline analysis!

๐Ÿ”ฎ Prediction Tracking System

The application now includes comprehensive prediction tracking to measure forecast accuracy and improve the katabatic wind model over time.

Key Features

  • Automatic Prediction Storage: All katabatic wind predictions are automatically stored when generated
  • Smart Verification: Predictions are verified against actual wind data after target dates pass
  • Accuracy Analytics: Detailed statistics show model performance by confidence level
  • Historical Analysis: Track prediction accuracy trends over time
  • Model Insights: Identify areas for prediction algorithm improvement

How It Works

  1. Prediction Generation: Weather analysis generates katabatic predictions daily
  2. Auto-Storage: Predictions are stored with timestamps, confidence levels, and expected conditions
  3. Verification Process: After target dates pass, predictions are compared with actual wind data
  4. Accuracy Scoring: Each prediction gets an accuracy score (0.0-1.0) based on multiple factors:
    • Peak wind detection accuracy
    • Wind direction accuracy within predicted ranges
    • Wind strength correlation with katabatic score
    • Confidence level calibration

Using Prediction Tracking

Dashboard View: Check the "๐Ÿ”ฎ Prediction Tracking & Accuracy" section to see:

  • Recent predictions with verification status
  • Accuracy statistics by confidence level
  • Model performance insights and recommendations
  • Manual verification controls

Automated Verification:

# Run daily verification (can be automated via cron)
python3 daily_verification.py

Key Metrics Tracked:

  • Overall prediction accuracy
  • Accuracy by confidence level (HIGH, MODERATE, LOW, VERY LOW)
  • Peak detection rate (did predicted winds actually occur?)
  • Direction accuracy (were winds from predicted directions?)
  • Model calibration (do confidence levels match actual accuracy?)

Dashboard Features

  • Automatic Data Collection: Today's data is automatically collected when accessing the dashboard after 8am Morrison time
  • Data Collection Panel: One-click data collection with progress indicators
  • Data Status: Visual calendar showing available data for the last 7 days with gap detection
  • Weather Analysis: Real-time katabatic scoring with current conditions analysis
  • ๐Ÿ”ฎ Prediction Tracking: Forecast accuracy analysis with verification controls
  • Key Metrics: Summary of strong wind events, strongest winds, and recent activity
  • Interactive Charts: Time series plots of morning wind speeds with strong wind event highlights
  • Data Tables: Detailed views of strong wind events, recent measurements, and daily summaries
  • Export Functionality: Download data as CSV files with customizable filters, plus comprehensive debug exports for analysis support

Analysis Features

The system automatically:

  • Converts all timestamps to Morrison, CO local time (America/Denver)
  • Flags measurements during morning hours (6-8am)
  • Identifies strong wind events (wind speed >15mph during morning hours)
  • Calculates daily, weekly, and trend statistics
  • Validates data quality and filters invalid readings
  • Provides comprehensive error handling and user feedback

Project Structure

katabatic-analyzer/
โ”œโ”€โ”€ README.md                 # This file
โ”œโ”€โ”€ requirements.txt          # Python dependencies
โ”œโ”€โ”€ .env.example             # Environment template
โ”œโ”€โ”€ config.py               # Configuration settings
โ”œโ”€โ”€ app.py                  # Main Streamlit dashboard
โ”œโ”€โ”€ data_collector.py       # Ecowitt API integration
โ”œโ”€โ”€ analyzer.py            # Wind analysis and katabatic detection
โ”œโ”€โ”€ database.py            # SQLite database operations
โ”œโ”€โ”€ timezone_utils.py      # Morrison, CO timezone handling
โ”œโ”€โ”€ test_foundation.py     # Foundation validation script
โ”œโ”€โ”€ test_api.py           # API connection test script
โ”œโ”€โ”€ setup.py              # Interactive setup script
โ”œโ”€โ”€ reset_database.py     # Database reset/migration script
โ”œโ”€โ”€ data/
โ”‚   โ”œโ”€โ”€ wind_data.db       # SQLite database (created automatically)
โ”‚   โ””โ”€โ”€ logs/              # Application logs
โ”œโ”€โ”€ tests/                 # Unit tests
โ””โ”€โ”€ docs/                  # Documentation

Configuration

Analysis Settings

Default katabatic detection parameters (configurable in config.py):

  • Threshold: 15 mph wind speed
  • Morning Hours: 6:00 AM - 8:00 AM (Morrison time)
  • Timezone: America/Denver

API Settings

The application uses Ecowitt's device history API:

  • Resolution: 5-minute intervals for recent data
  • Data Fields: Wind speed, wind gust, wind direction
  • Rate Limits: Respectful request timing with delays

Database Schema

The SQLite database stores wind measurements with these key fields:

  • datetime_local: Morrison, CO local time
  • wind_speed: Wind speed in mph
  • wind_gust: Wind gust in mph
  • wind_direction: Wind direction in degrees
  • is_morning_hours: Boolean flag for 6-8am period
  • is_strong_wind_event: Boolean flag for strong wind events

Development

Running Tests

# Run unit tests
pytest tests/

# Run with coverage
pytest tests/ --cov=.

# Run foundation tests
python test_foundation.py

Code Quality

# Format code
black .

# Lint code
flake8 .

Debugging

Logs are written to ./data/logs/wind_analyzer.log with configurable verbosity.

API Documentation

This project uses the Ecowitt Weather API. Key endpoints:

  • Device History: /api/v3/device/history
  • Documentation: See docs/api-docs/Get-Device-History.md

API Rate Limits

  • Respectful request timing (1 second between requests)
  • Daily data collection in single-day chunks
  • Automatic retry with exponential backoff

Troubleshooting

Common Issues

  1. "Configuration error": Check that your .env file has valid API credentials
  2. "No data collected": Verify your device MAC address and API permissions
  3. "Import errors": Ensure all dependencies are installed: pip install -r requirements.txt

Getting Help

  1. Check the logs in ./data/logs/ for detailed error messages
  2. Run python test_foundation.py to validate setup
  3. Verify your Ecowitt device is online and reporting data

Data Storage

  • Local Only: All data stored locally in SQLite database
  • No Cloud: No data transmitted to external services (except Ecowitt API)
  • Backup: Database file at ./data/wind_data.db can be backed up manually

Contributing

This is a personal analysis project for Morrison, CO wind patterns. The codebase is designed to be modular and extensible.

Key Design Principles

  • Local Operation: No cloud dependencies beyond API calls
  • Manual Control: User-triggered data collection and analysis
  • Timezone Accuracy: All data properly normalized to Morrison time
  • Data Integrity: Comprehensive validation and error handling

License

This project is for personal use analyzing local wind patterns in Morrison, Colorado.

Changelog

Version 1.0.0 (July 4, 2025)

  • โœ… Foundation Implementation - Complete core architecture
  • โœ… Data Collection - Ecowitt API integration with proper error handling
  • โœ… Database - SQLite schema with wind data storage and indexing
  • โœ… Dashboard - Streamlit interface with data collection controls
  • โœ… Katabatic Detection - Algorithm for identifying >15mph morning winds
  • โœ… Timezone Handling - Morrison, CO timezone conversion and validation
  • โœ… Testing - Comprehensive unit tests and validation scripts
  • โœ… Documentation - User guide and setup instructions
  • โœ… Export Features - CSV data export with filtering options
  • โœ… Enhanced UI - Progress bars, status indicators, and error feedback
  • โœ… Setup Tools - Interactive configuration and testing scripts

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages