Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Latest commit

 

History

146 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

IncidentRelay

IncidentRelay

Self-hosted on-call, alert routing, event orchestration and incident response for teams that want to keep operational control in their own infrastructure.

IncidentRelay receives alerts from monitoring systems, normalizes and orchestrates them, routes them to the responsible service and team, groups related signals into incidents, applies priority and escalation rules, and delivers notifications to on-call responders.

It is designed for SRE, DevOps, platform, infrastructure and operations teams that need PagerDuty-style building blocks without depending on a hosted incident-management platform.

Alerts

What IncidentRelay provides

Alert intake, routing and orchestration

  • route-based intake tokens and source-specific webhook endpoints;
  • native integrations for Alertmanager, AWS SNS / CloudWatch, Azure Monitor, Datadog, Grafana, LibreNMS, Nagios, New Relic, RMON, Sentry, Uptime Kuma and Zabbix;
  • generic webhook intake with PagerDuty Events API v2-compatible trigger, acknowledge and resolve events;
  • Global and Service Event Orchestration;
  • orchestration Builder and JSON editor, validation, simulation, shadow mode, replay, immutable published versions and rollback;
  • orchestration actions for routing, service selection, labels, severity, priority, grouping, suppression, pause/drop decisions, policy selection and asynchronous webhooks;
  • configurable Explain Trace detail and retention.

Incident management

  • alert groups with child alerts, deduplication and grouping;
  • acknowledge, resolve and temporary alert shelving workflows;
  • incident priorities and priority policies;
  • responders, stakeholders and comments;
  • reminders and escalation policies;
  • silences and maintenance windows;
  • notification center and audit history;
  • dependency-aware alert correlation and Explain Trace.

Service catalog and impact

  • technical services with ownership, criticality, tier, links and runbooks;
  • matcher-based service routing and reusable matcher presets;
  • service dependencies and blast-radius analysis;
  • Business Services and component impact;
  • current service impact plus historical impact snapshots;
  • Service Standards and readiness checks;
  • SLI/SLO definitions and measurements;
  • service events and default stakeholders;
  • Heartbeats with expected and auto-discovered instances.

On-call and notification delivery

  • groups, teams and RBAC-style group/team roles;
  • rotations, rotation layers, restrictions and temporary overrides;
  • on-call calendar, ICS feeds and CalDAV access;
  • route channels and service Notification Policies;
  • Mattermost, Slack, Telegram, Discord, Microsoft Teams, email and generic webhook delivery;
  • interactive ACK / Resolve / Shelve actions for supported chat providers;
  • temporary per-AlertGroup shelving with automatic expiry, notification/escalation pause and responder-audited Unshelve;
  • profile-level browser/PWA push notifications;
  • personal notification rules for browser push, email and voice-call follow-up;
  • pluggable self-hosted voice providers.

Access and administration

  • OIDC and SAML 2.0 SSO with mapping rules;
  • personal API tokens with scopes;
  • Swagger/OpenAPI documentation;
  • configurable retention for resolved alerts and diagnostic/orchestration history;
  • SQLite for small single-node installations and PostgreSQL for larger deployments;
  • Docker Compose, Helm/Kubernetes, RPM and manual systemd installation paths.

Alert flow

A simplified IncidentRelay 2.x flow looks like this:

Monitoring system
      ↓
Integration authentication + normalization
      ↓
Global Event Orchestration
      ↓
Route / Service / Team selection
      ↓
Service Event Orchestration
      ↓
Grouping + correlation + priority
      ↓
Rotation / escalation + notification policy
      ↓
Shared channels + personal notification rules
      ↓
ACK / Resolve / Shelve / responders / stakeholders

Routes control how alerts enter IncidentRelay and provide the security boundary for intake tokens. Services describe what is affected. Notification delivery can come directly from route channels, from a service Notification Policy, or from both, depending on route configuration. Event Orchestration can override routing, service and policy decisions for matching events.

Browser/PWA push and personal notification rules are evaluated separately from shared route/service channels.


Event Orchestration

Event Orchestration lets incoming events be transformed and routed before the normal alert lifecycle finishes processing them.

Typical rules can:

IF labels.environment == production
AND severity == critical
THEN
  select Payments service
  set priority P1
  use Critical escalation policy
  normalize labels

Global orchestration can make group-wide routing decisions. Service orchestration runs after a service is known and can apply service-specific logic. Published definitions support simulation, shadow evaluation, replay, version history and rollback.

Event Orchestration

Explain Trace shows why an alert was routed, grouped, prioritized, suppressed or notified the way it was.

Alert Explain Trace

Read more: Event Orchestration and Explain Trace.


Supported integrations

Incoming alert sources

Source Endpoint Documentation
Alertmanager POST /api/integrations/alertmanager Alertmanager
AWS SNS / CloudWatch POST /api/integrations/aws-sns/<route_id> AWS SNS / CloudWatch
Azure Monitor POST /api/integrations/azure-monitor Azure Monitor
Datadog POST /api/integrations/datadog Datadog
Grafana POST /api/integrations/grafana Grafana
LibreNMS POST /api/integrations/librenms LibreNMS
Nagios POST /api/integrations/nagios Nagios
New Relic POST /api/integrations/new-relic New Relic
RMON POST /api/integrations/rmon RMON
Sentry POST /api/integrations/sentry/<route_id> Sentry
Uptime Kuma POST /api/integrations/uptime-kuma Uptime Kuma
Zabbix POST /api/integrations/zabbix Zabbix
Generic / PagerDuty Events API v2 POST /api/integrations/webhook Generic webhook

Incoming integrations use route intake credentials. The generated endpoint/help text on the Routes page shows the authentication form supported by each source.

Shared notification channels

Channel Notes
Mattermost Incoming webhook or Bot API; Bot API supports interactive ACK / Resolve / Shelve actions and message updates
Slack Incoming webhook or Bot API; interactive ACK / Resolve / Shelve actions can use HTTP callbacks or Socket Mode worker
Telegram Bot notifications with optional ACK / Resolve / Shelve action buttons
Discord Webhook delivery
Microsoft Teams Webhook delivery
Email Delivered through global SMTP configuration
Webhook Generic outbound webhook

Browser/PWA push is profile-level rather than a shared channel. Personal notification rules can deliver through browser push, email or voice call to the assigned user's profile contacts.

Read more: Notification channels and Notification Policies.


Installation

Docker Compose

Docker Compose is the fastest way to run a small self-hosted installation. The repository Compose file uses the published IncidentRelay image and starts the web service, scheduler, Telegram worker and Slack Socket Mode worker.

cd docker
docker compose up -d

Open:

http://SERVER_IP:8080/login

Use PostgreSQL with the supplied override:

docker compose \
  -f docker-compose.yml \
  -f docker-compose.postgres.yml \
  up -d

Read more: Docker installation.

Kubernetes / Helm

The Helm chart is published as an OCI artifact in GHCR:

helm install incidentrelay \
  oci://ghcr.io/roxy-wi/incidentrelay-charts/incidentrelay \
  --version 2.2.0 \
  --set-string config.main.secret_key="$(openssl rand -hex 32)"

The chart defaults to the ghcr.io/roxy-wi/incidentrelay:2.2 application image. Configuration can be rendered from config.* values or supplied through existingConfigSecret.

Read more: Kubernetes installation.

RHEL / Rocky Linux / AlmaLinux / CentOS Stream

sudo dnf install -y curl
sudo curl -fsSL \
  https://repo.incidentrelay.io/incidentrelay.repo \
  -o /etc/yum.repos.d/incidentrelay.repo
sudo dnf makecache
sudo dnf install -y incidentrelay

Read more: RPM installation.

Manual systemd installation

Use the manual installation path when running directly from a source checkout or when you manage the Python environment yourself.

Read more: Systemd installation.


Runtime services

A full installation can run these processes:

incidentrelay.service                  # HTTP API, UI and incoming webhooks
incidentrelay-scheduler.service        # reminders, escalations and periodic jobs
incidentrelay-telegram-worker.service  # optional Telegram callbacks / polling
incidentrelay-slack-worker.service     # optional Slack Socket Mode interactions

The scheduler should run as a dedicated process rather than once per web worker.

Common RPM/systemd paths:

/var/www/incidentrelay
/var/www/incidentrelay/venv
/etc/incidentrelay/incidentrelay.conf
/var/lib/incidentrelay
/var/log/incidentrelay
/usr/local/lib/incidentrelay/voice_providers

Configuration

IncidentRelay reads the configuration file path from:

INCIDENTRELAY_CONFIG_FILE

Example:

export INCIDENTRELAY_CONFIG_FILE=/etc/incidentrelay/incidentrelay.conf

For production, set the public base URL used for links and provider callbacks:

[server]
public_base_url = https://incidentrelay.example.com

SQLite is suitable for a small single-node installation:

[database]
type = sqlite
name = /var/lib/incidentrelay/incidentrelay.db

[sqlite]
wal = true
busy_timeout = 5000

PostgreSQL is recommended for larger installations and multi-worker deployments:

[database]
type = postgresql
host = 127.0.0.1
port = 5432
name = incidentrelay
user = incidentrelay
password = change-me

Important operational settings include retention, outbound HTTP network policy and Explain Trace detail. See Configuration for the canonical reference.


First setup

For RPM/systemd installations, run migrations and create the first administrator:

sudo -u incidentrelay \
  INCIDENTRELAY_CONFIG_FILE=/etc/incidentrelay/incidentrelay.conf \
  /var/www/incidentrelay/venv/bin/python \
  /var/www/incidentrelay/manage.py migrate
sudo -u incidentrelay \
  INCIDENTRELAY_CONFIG_FILE=/etc/incidentrelay/incidentrelay.conf \
  /var/www/incidentrelay/venv/bin/python \
  /var/www/incidentrelay/manage.py create-admin \
  --username admin \
  --password 'change-me-123' \
  --email [email protected]

For Docker Compose, the web container can run migrations automatically. Create the first administrator with:

cd docker
docker compose exec incidentrelay \
  python manage.py create-admin \
  --username admin \
  --password 'change-me-123' \
  --email [email protected]

Change the example password before production use.

A typical UI setup is:

1. Create a group and users
2. Create a team and assign team roles
3. Create a rotation and on-call members
4. Create technical services
5. Add runbooks, links and dependencies as needed
6. Create shared notification channels
7. Optionally create notification / priority / escalation policies
8. Create a route and select its service/channel mode
9. Optionally configure Global or Service Event Orchestration
10. Copy the route intake credential into the monitoring system
11. Send a test alert and verify ACK / Resolve / Shelve

Detailed guide: First login and initial setup.


Example Alertmanager request

curl -X POST http://127.0.0.1:8080/api/integrations/alertmanager \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer ROUTE_TOKEN' \
  -d '{
    "status": "firing",
    "alerts": [
      {
        "status": "firing",
        "labels": {
          "alertname": "DiskFull",
          "severity": "critical",
          "team": "infra",
          "instance": "host1"
        },
        "annotations": {
          "summary": "Disk is full",
          "description": "/var is 95% full"
        },
        "fingerprint": "disk-full-host1-var"
      }
    ]
  }'

Read more: Alertmanager integration.


API

Swagger UI:

/docs

OpenAPI JSON:

/api/openapi.json

Personal API tokens can be created from the user profile and restricted by scope.

Read more: API documentation and Profile/API tokens.


Documentation

Area Documentation
Getting started Getting started
Configuration Configuration
Groups and RBAC Groups and RBAC
Teams, rotations and routes Teams, rotations and routes
Alerts and incidents Alerts
Alert shelving Shelving
Event Orchestration Event Orchestration
Explain Trace Explain Trace
Incident priorities Priorities
Responders / stakeholders Responders / Stakeholders
Services Services
Business Services Business Services
Dependency correlation Alert correlation
Service SLI/SLO SLI/SLO
Service standards Standards and events
Heartbeats Heartbeats
Notification Policies Notification Policies
Maintenance windows Maintenance Windows
On-call calendar Calendar
Browser/PWA push Browser Push
SSO OIDC and SAML
Data retention Data retention
Integrations Integrations
Troubleshooting Troubleshooting

Demo and development checks

Create demo data:

python manage.py demo-data

After migrations, verify that the configured database matches the Peewee models:

python app/check_schema.py

See Demo data, Schema check and CONTRIBUTING.md.


License

IncidentRelay is source-available under the Elastic License 2.0 (Elastic-2.0). Self-hosted and internal use is available subject to ELv2. Offering IncidentRelay, or a service exposing a substantial set of its functionality, to third parties as a hosted or managed service requires rights permitted by ELv2 or a separate commercial license.

See LICENSING.md for usage examples and COMMERCIAL_LICENSE.md for commercial licensing. Contributions are subject to CLA.md.

Historical versions and copies previously released under MIT remain licensed under the MIT License that accompanied those versions.

About

Open-source self-hosted on-call scheduling, alert routing, escalation, and incident response platform for SRE and DevOps teams.

Topics

Resources

Contributing

Stars

191 stars

Watchers

6 watching

Forks

Releases

Packages

Contributors

Languages