netdata · kanelatechnical · Oct 28, 2025 · Nov 21, 2025 · Nov 21, 2025 · Nov 21, 2025
@@ -165,6 +165,35 @@ cloud_notifications_integrations,,,,
 https://github.com/netdata/netdata/edit/master/src/health/REFERENCE.md,Alert Configuration Reference,Published,Alerts & Notifications,
 https://github.com/netdata/netdata/edit/master/src/web/api/health/README.md,Health API Calls,Published,Alerts & Notifications,
 ,,,,
+https://github.com/netdata/netdata/edit/master/docs/alerts/understanding-alerts.md,Understanding Alerts,Published,Alerts & Notifications/The Book/Understanding Alerts,
+https://github.com/netdata/netdata/edit/master/docs/alerts/understanding-alerts/what-is-a-netdata-alert.md,What is a Netdata Alert,Published,Alerts & Notifications/The Book/Understanding Alerts,
+https://github.com/netdata/netdata/edit/master/docs/alerts/understanding-alerts/alert-types-alarm-vs-template.md,Alert Types: alarm versus template,Published,Alerts & Notifications/The Book/Understanding Alerts,
+https://github.com/netdata/netdata/edit/master/docs/alerts/understanding-alerts/where-alerts-live.md,"Where Alerts Live (Files, Agent, Cloud)",Published,Alerts & Notifications/The Book/Understanding Alerts,
+https://github.com/netdata/netdata/edit/master/docs/alerts/creating-alerts-pages/creating-alerts.md,Creating and Managing Alerts,Published,Alerts & Notifications/The Book/Creating and Managing Alerts,
+https://github.com/netdata/netdata/edit/master/docs/alerts/creating-alerts-pages/quick-start-create-your-first-alert.md,Quick Start: Create Your First Alert,Published,Alerts & Notifications/The Book/Creating and Managing Alerts,
+https://github.com/netdata/netdata/edit/master/docs/alerts/creating-alerts-pages/creating-and-editing-alerts-via-config-files.md,Creating and Editing Alerts via Configuration Files,Published,Alerts & Notifications/The Book/Creating and Managing Alerts,
+https://github.com/netdata/netdata/edit/master/docs/alerts/creating-alerts-pages/creating-and-editing-alerts-via-cloud.md,Creating and Editing Alerts via Netdata Cloud (Alerts Configuration Manager),Published,Alerts & Notifications/The Book/Creating and Managing Alerts,
+https://github.com/netdata/netdata/edit/master/docs/alerts/creating-alerts-pages/managing-stock-vs-custom-alerts.md,Managing Stock versus Custom Alerts,Published,Alerts & Notifications/The Book/Creating and Managing Alerts,
+https://github.com/netdata/netdata/edit/master/docs/alerts/creating-alerts-pages/reloading-and-validating-alert-configuration.md,Reloading and Validating Alert Configuration,Published,Alerts & Notifications/The Book/Creating and Managing Alerts,
+https://github.com/netdata/netdata/edit/master/docs/alerts/alert-configuration-syntax/alert-configuration-syntax.md,Alert Configuration Syntax,Published,Alerts & Notifications/The Book/Alert Configuration Syntax,
+https://github.com/netdata/netdata/edit/master/docs/alerts/alert-configuration-syntax/alert-definition-lines.md,Alert Definition Lines (Minimal and Full Forms),Published,Alerts & Notifications/The Book/Alert Configuration Syntax,
+https://github.com/netdata/netdata/edit/master/docs/alerts/alert-configuration-syntax/lookup-and-time-windows.md,Lookup and Time Windows,Published,Alerts & Notifications/The Book/Alert Configuration Syntax,
+https://github.com/netdata/netdata/edit/master/docs/alerts/alert-configuration-syntax/calculations-and-transformations.md,"Calculations and Transformations (calc, absolute, percentage, anomaly-bit)",Published,Alerts & Notifications/The Book/Alert Configuration Syntax,
+https://github.com/netdata/netdata/edit/master/docs/alerts/alert-configuration-syntax/expressions-operators-functions.md,"Expressions, Operators, and Functions",Published,Alerts & Notifications/The Book/Alert Configuration Syntax,
+https://github.com/netdata/netdata/edit/master/docs/alerts/alert-configuration-syntax/variables-and-special-symbols.md,"Variables and Special Symbols",Published,Alerts & Notifications/The Book/Alert Configuration Syntax,
+https://github.com/netdata/netdata/edit/master/docs/alerts/alert-configuration-syntax/optional-metadata.md,"Optional Metadata: class, type, component, and tags",Published,Alerts & Notifications/The Book/Alert Configuration Syntax,
+,,,,
+https://github.com/netdata/netdata/edit/master/docs/alerts/controlling-alerts-noise/index.md,Controlling Alerts and Noise,Published,Alerts & Notifications/The Book/Controlling Alerts and Noise,
+https://github.com/netdata/netdata/edit/master/docs/alerts/receiving-notifications/index.md,Receiving Notifications,Published,Alerts & Notifications/The Book/Receiving Notifications,
+https://github.com/netdata/netdata/edit/master/docs/alerts/alert-examples/index.md,Alert Examples and Common Patterns,Published,Alerts & Notifications/The Book/Alert Examples and Common Patterns,
+https://github.com/netdata/netdata/edit/master/docs/alerts/troubleshooting-alerts/index.md,Troubleshooting Alert Behaviour,Published,Alerts & Notifications/The Book/Troubleshooting Alert Behaviour,
+https://github.com/netdata/netdata/edit/master/docs/alerts/advanced-techniques/index.md,Advanced Alert Techniques,Published,Alerts & Notifications/The Book/Advanced Alert Techniques,
+https://github.com/netdata/netdata/edit/master/docs/alerts/apis-alerts-events/index.md,APIs for Alerts and Events,Published,Alerts & Notifications/The Book/APIs for Alerts and Events,
+https://github.com/netdata/netdata/edit/master/docs/alerts/cloud-alert-features/index.md,Netdata Cloud Alert and Events Features,Published,Alerts & Notifications/The Book/Netdata Cloud Alert and Events Features,
+https://github.com/netdata/netdata/edit/master/docs/alerts/built-in-alerts.md,Built-In Alerts Reference,Published,Alerts & Notifications/The Book/Built-In Alerts Reference,
+https://github.com/netdata/netdata/edit/master/docs/alerts/best-practices.md,Best Practices for Alerting,Published,Alerts & Notifications/The Book/Best Practices for Alerting,
+https://github.com/netdata/netdata/edit/master/docs/alerts/architecture.md,Alerts and Notifications Architecture,Published,Alerts & Notifications/The Book/Alerts and Notifications Architecture,
+,,,,
 https://github.com/netdata/netdata/edit/master/docs/category-overview-pages/machine-learning-and-assisted-troubleshooting.md,Netdata AI,Published,Netdata AI,
 https://github.com/netdata/netdata/edit/master/docs/ml-ai/ai-insights.md,Insights,Published,Netdata AI/Insights,
 https://github.com/netdata/netdata/edit/master/docs/netdata-ai/insights/infrastructure-summary.md,Infrastructure Summary,Published,Netdata AI/Insights,

@@ -0,0 +1,37 @@
+# 8.4 Custom Actions with `exec`
+
+## 8.4.1 Basic exec Syntax
+
+```conf
+template: service_down
+    on: health.service
+lookup: average -1m of status
+    every: 1m
+     crit: $this == 0
+       exec: /usr/local/bin/alert-handler.sh
+       to: ops-team
+```
+
+## 8.4.2 Environment Variables
+
+| Variable | Description |
+|----------|-------------|
+| `NETDATA_ALARM_NAME` | Alert name |
+| `NETDATA_HOST` | Hostname |
+| `NETDATA_STATUS` | Current status |
+| `NETDATA_VALUE` | Current value |
+
+## 8.4.3 Example: PagerDuty Integration
+
+```bash
+#!/bin/bash
+if [ "$NETDATA_STATUS" = "CRITICAL" ]; then
+    curl -X POST https://events.pagerduty.com/v2/enqueue \
+        -H 'Content-Type: application/json' \
+        -d "{\"routing_key\": \"YOUR_KEY\", \"event_action\": \"trigger\", \"dedup_key\": \"$NETDATA_ALARM_NAME\"}"
+fi
+```
+
+## 8.4.4 Related Sections
+
+- **5.2.6 Custom Scripts** for additional examples
@@ -0,0 +1,29 @@
+# 8.1 Hysteresis and Status-Based Conditions
+
+## 8.1.1 The Problem with Simple Thresholds
+
+```conf
+warn: $this > 80
+```
+
+This causes flapping when CPU hovers around 80%.
+
+## 8.1.2 Hysteresis Solution
+
+```conf
+template: cpu_hysteresis
+    on: system.cpu
+lookup: average -5m of user,system
+    every: 1m
+     warn: ($this > 80) && ($status != WARNING)
+     crit: ($this > 95) && ($status != CRITICAL)
+```
+
+Behavior:
+- Enter WARNING at 80%, but only clear when below 70%
+- Enter CRITICAL at 95%, but only clear when below 85%
+
+## 8.1.3 Related Sections
+
+- **4.4 Reducing Flapping** for delay techniques
+- **7.3 Alert Flapping** for debugging
@@ -0,0 +1,18 @@
+# 8. Advanced Alert Techniques
+
+This chapter covers advanced patterns for experienced users.
+
+## What You'll Find in This Chapter
+
+| Section | Technique |
+|---------|-----------|
+| **8.1 Hysteresis** | Status-dependent thresholds |
+| **8.2 Multi-Dimensional** | Target specific dimensions |
+| **8.3 Label-Based Targeting** | Host and chart labels |
+| **8.4 Custom Actions** | `exec` for automation |
+| **8.5 Performance** | Efficient alert sets |
+
+## What's Next
+
+- **8.1 Hysteresis and Status-Based Conditions** for stable thresholds
+- **8.4 Custom Actions with `exec`** for automation
@@ -0,0 +1,25 @@
+# 8.3 Host, Chart, and Label-Based Targeting
+
+## 8.3.1 Understanding Labels
+
+Netdata supports:
+- **Host labels**: `env:production`, `role:database`
+- **Chart labels**: `mount_point:/data`
+
+## 8.3.2 Label-Based Alert Scope
+
+```conf
+template: db_cpu_high
+    on: system.cpu
+lookup: average -5m of user,system
+    every: 1m
+     warn: $this > 80
+   calc: if($host_labels.role == "database" && $host_labels.env == "production", $this, 0)
+```
+
+This targets only production database servers.
+
+## 8.3.3 Related Sections
+
+- **3.6 Optional Metadata** for labels documentation
+- **10.4 Room-Based Alerting** for Cloud label usage
@@ -0,0 +1,29 @@
+# 8.2 Multi-Dimensional and Per-Instance Alerts
+
+## 8.2.1 Dimension Selection
+
+```conf
+template: network_quality
+    on: net.net
+lookup: average -1m of errors,dropped
+    every: 1m
+     warn: $errors > 10
+```
+
+## 8.2.2 Scaling to All Instances
+
+Templates automatically create one alert per matching chart.
+
+```conf
+template: interface_errors
+    on: net.net
+lookup: average -5m of errors
+    every: 1m
+     warn: $this > 10
+```
+
+Creates alerts for: `interface_errors-eth0`, `interface_errors-eth1`, etc.
+
+## 8.2.3 Related Sections
+
+- **1.2 Alert Types: alarm vs template** for conceptual understanding
@@ -0,0 +1,30 @@
+# 8.5 Performance Considerations
+
+## 8.5.1 What Affects Performance
+
+| Factor | Impact | Recommendation |
+|--------|--------|----------------|
+| `every` frequency | Higher = more CPU | Use 1m for most alerts |
+| `lookup` window | Longer windows need more processing | Match to needs |
+| Number of alerts | More alerts = more evaluation | Disable unused alerts |
+
+## 8.5.2 Efficient Configuration
+
+```conf
+# Efficient: 1-minute evaluation
+template: system_health
+    on: system.cpu
+lookup: average -5m of user,system
+    every: 1m     # Good: 60 evaluations/hour
+     warn: $this > 80
+
+# Inefficient: 10-second evaluation
+template: system_fast
+    on: system.cpu
+lookup: average -1m of user,system
+    every: 10s    # Bad: 360 evaluations/hour
+```
+
+## 8.5.3 Related Sections
+
+- **13.1 Evaluation Architecture** for internal behavior
@@ -0,0 +1,86 @@
+# 3. Alert Configuration Syntax (Reference)
+
+This chapter is a precise, task-oriented reference for writing and understanding Netdata alert definitions in configuration files.
+
+:::tip
+
+Refer to this chapter when you:
+- Are editing `.conf` files under `health.d` and need to know what each line means
+- Want to understand or safely tweak a stock alert shipped by Netdata
+- Need to write custom, advanced alert rules beyond what the UI exposes
+- Are debugging an alert expression and need to know which variables, operators, or functions are available
+
+:::
+
+## Alert Configuration Structure Overview
+
+Netdata alerts follow a simple pipeline:
+
+```mermaid
+%%{init: {'theme':'base', 'themeVariables': { 'fontSize':'18px', 'fontFamily':'arial'}}}%%
+graph LR
+    A("<b>Define</b><br/>name, chart<br/>&nbsp;<br/>&nbsp;")
+    B("<b>Get Data</b><br/>lookup §3.2<br/>&nbsp;<br/>&nbsp;")
+    C("<b>Transform</b><br/>calc §3.3<br/>&nbsp;<br/>&nbsp;")
+    D("<b>Evaluate</b><br/>warn/crit §3.4<br/>&nbsp;<br/>&nbsp;")
+    E("<b>Notify</b><br/>to: recipient<br/>&nbsp;<br/>&nbsp;")
+
+    F("<b>Variables §3.5</b><br/>$this, $status<br/>&nbsp;<br/>&nbsp;")
+    G("<b>Metadata §3.6</b><br/>class, type<br/>&nbsp;<br/>&nbsp;")
+
+    A --> B
+    B --> C
+    C --> D
+    D --> E
+
+    F -.-> D
+    G -.-> E
+
+    style A fill:#4caf50,stroke:#666666,stroke-width:3px,color:#000000,font-size:16px
+    style B fill:#ffeb3b,stroke:#666666,stroke-width:3px,color:#000000,font-size:16px
+    style C fill:#2196F3,stroke:#666666,stroke-width:3px,color:#ffffff,font-size:16px
+    style D fill:#ff9800,stroke:#666666,stroke-width:3px,color:#000000,font-size:16px
+    style E fill:#4caf50,stroke:#666666,stroke-width:3px,color:#000000,font-size:16px
+    style F fill:#e0e0e0,stroke:#666666,stroke-width:3px,stroke-dasharray: 5 5,color:#000000,font-size:16px
+    style G fill:#e0e0e0,stroke:#666666,stroke-width:3px,stroke-dasharray: 5 5,color:#000000,font-size:16px
+```
+
+**How it works:**
+
+1. **Define** (§3.1): Name the alert and target a chart
+2. **Get Data** (§3.2): Use `lookup` to read metrics over time
+3. **Transform** (§3.3): Optionally use `calc` to compute derived values
+4. **Evaluate** (§3.4): Apply `warn`/`crit` conditions using variables (§3.5)
+5. **Notify**: Send to recipients with metadata (§3.6)
+
+**Key:** Solid lines show the evaluation pipeline; dotted lines show supporting elements that inform evaluation or notifications.
+
+## What You'll Find in This Chapter
+
+Each section mirrors Netdata's `REFERENCE.md`, but is reorganized by what you're trying to do, not by internal implementation details.
+
+| Section | What It Covers |
+|---------|----------------|
+| **3.1 Alert Definition Lines (Minimal and Full Forms)** | The structure of an alert block. Shows the essential lines (`alarm`/`template`, `on`, `lookup`, `every`, `warn`, `crit`) and optional ones (`to`, `summary`, `info`, `delay`, `exec`, etc.), with valid values and defaults. Helps you read or assemble a complete alert from top to bottom. |
+| **3.2 Lookup and Time Windows** | The `lookup` line in detail: how to choose aggregation functions (`average`, `min`, `max`, `sum`, etc.), time windows (`1m`, `10m`, `1h`, etc.), and dimensions, and how these lookups interact with Netdata's time-series storage (dbengine). This is where you learn what data your alert actually sees. |
+| **3.3 Calculations and Transformations (`calc`)** | How to transform lookup results before comparing them to thresholds. Documents the `calc` expression and how to build percentages and rate-of-change calculations on top of raw metrics. |
+| **3.4 Expressions, Operators, and Functions** | The expression language used in `warn`, `crit`, and related lines: arithmetic operators, comparisons, logical operators, and any supported helper functions. This section explains how to write conditions that evaluate to true/false. |
+| **3.5 Variables and Special Symbols** | All the variables you can use inside expressions: `$this`, `$status`, `$now`, chart and dimension variables, context-related variables, and status constants. Also shows how to discover available variables using the `alarm_variables` API. |
+| **3.6 Optional Metadata: class, type, component, and labels** | Metadata lines that don't change when an alert fires, but control how alerts are grouped, filtered, and displayed in UIs and notifications. Explains how to keep metadata consistent so you can build clean views, silencing rules, and routing later. Covers `class`, `type`, `component`, `host labels`, and `chart labels`. |
+
+## How to Navigate This Chapter
+
+- If you're reading or editing a specific alert, start at 3.1, then jump to the subsection that matches the line you're unsure about.
+- If you're designing a new rule, skim 3.1 → 3.2 → 3.3 → 3.4 to understand the core building blocks, then use 3.5–3.6 to add variables and metadata.
+- For API details about querying alerts and variables, use this chapter together with Chapter 9: APIs for Alerts and Events.
+
+:::note
+
+This chapter is not a workflow guide. It won't tell you when to create alerts or how to deploy them across nodes. Instead, it's the place you come back to whenever you need a precise answer to: "What does this line/condition actually mean?"
+
+:::
+
+## What's Next
+
+- **3.1 Alert Definition Lines (Minimal and Full Forms)** covers the structure of a single alert block line by line (start here).
+- Or, if you're already comfortable with the basics and only care about what data the alert sees, skip ahead to **3.2 Lookup and Time Windows**.