Netdata v2.5.1 is a patch release to address issues discovered since v2.5.0.

This patch release provides the following bug fixes and updates:

• Fixed obsolete chart cleanup to properly handle virtual nodes (#20254, @ilyam8)
• Fixed SNMP collector to use 32-bit counters for network interfaces when 64-bit counters aren't available (#20249, @ilyam8)
• Fixed SNMP collector to fall back to interface description (ifDescr) when interface name (ifName) is empty (#20248, @ilyam8)
• Fixed SNMP discovery by correcting SNMPv3 credential parameter names to match expected values (#20247, #20256, @ilyam8)
• Fixed compilation on older distributions by removing uv_sleep function call that isn't available in older libuv versions (#20243, @stelfrag)
• Fixed claiming in Docker by improving detection of localhost environments and providing correct claim command instructions (#20240, @stelfrag)
• Added user configuration option to override default thread stack size (#20236, @stelfrag)
• Fixed CouchDB collector to use correct units (bytes instead of KiB) for database size charts (#20235, @ilyam8)

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Make use of the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!

Table of Contents

• Summary
• Acknowledgments
• Contributions
  • Collectors
  • Packaging/Installation
  • Documentation
  • Other Notable Changes
• Deprecation notice
• Support options

Release Summary

Netdata v2.5.0 continues our commitment to stability with significant improvements to system robustness. This release focuses on eliminating potential crashes, resolving memory issues, and enhancing thread management across the codebase. We've implemented comprehensive deadlock detection, improved resource cleanup procedures, and added protection against corrupted data files.

Acknowledgments

• @barracuda156 for fixing compilation on macOS versions earlier than 11.
• @luiizaferreirafonseca for fixing grammar in the main README file.
• @rhoriguchi for fixing filtering of systemd-nspawn container payload in cgroups monitoring.

Contributions

Collectors

Packaging/Installation

Documentation

Other Notable Changes

Table of Contents

• Summary
• Acknowledgments
• Contributions
  • Collectors
  • Packaging/Installation
  • Documentation
  • Other Notable Changes
• Deprecation notice
• Support options

Release Summary

Netdata v2.4.0 is a stability-focused release that addresses many issues that were identified thanks to the new agent reporting system introduced in v2.3.0. This release significantly improves reliability by fixing multiple crash scenarios and memory leaks throughout the codebase.

Key Highlights

Acknowledgments

• @dave818 for fixing a cron job syntax error in the updater script by correcting the time format.
• @ycdtosa for adding missing --offline-install-source option documentation to kickstart script usage information, adding Synology-specific user and group creation commands to kickstart script for improved DSM compatibility, and updating Synology installation documentation to clearly differentiate steps required for older DSM versions.

Contributions

Collectors

Packaging/Installation

Documentation

Other Notable Changes

Netdata v2.3.2 is a patch release to address issues discovered since v2.3.1.

This patch release provides the following bug fixes and updates:

• Fixed journal file creation reliability with improved error handling and simplified allocation process (#20018, @ktsaou)
• Fixed leakage of build environment identifiers by blacklisting GitHub runner machine IDs (#20016, @ktsaou)
• Fixed potential memory access violations by adding validation for journal file headers and page boundaries (#20013, @stelfrag)
• Fixed a rare crash condition by properly reinitializing data collection for obsolete or archived dimensions (#20007, @ktsaou)
• Fixed MegaCLI collector to properly handle missing battery backup units (#20008, @ilyam8)
• Changed UUID generation to use version 4 format for better uniqueness (#20002, @ktsaou)
• Added detection for additional CI environment variables to automatically disable telemetry (#19999, @ktsaou)
• Fixed Agent status system to handle null UUIDs and improved tracking of shutdown time, crash counts, and connection states (#19996, #20003, #20011 @ktsaou)
• Added detailed worker thread status information and enhanced crash diagnostics capabilities (#19992, @ktsaou)
• Fixed potential crash in Windows perflib collector when handling null pointers (#19985, @ktsaou)
• Fixed error reporting to preserve errno values during out-of-memory conditions (#19984, @ktsaou)
• Fixed potential crashes when handling empty data arrays (#19983, @ktsaou)
• Fixed Agent shutdown by properly joining ACLK and metadata threads before closing database connections (#19980, @stelfrag)
• Fixed random crashes during shutdown by avoiding precompiled database statements for host metadata (#19978, @stelfrag)
• Limited maximum database file size to 1GB to optimize memory usage during file operations (#19977, @stelfrag)
• Fixed crash in variable lookup function when processing search results with scores (#19972, @ktsaou)
• Fixed ACLK synchronization thread shutdown with better termination sequence and timeout handling for stuck operations (#19966, @stelfrag)
• Improved Parent node startup performance by preloading UUIDs into metrics registry for faster initialization (#19964, @ktsaou)
• Fixed Windows installer to properly manage configuration files and handle upgrades correctly (#19962, @thiagoftsm)
• Fixed potential crash in health alarm cleanup when unlinking alerts from charts (#19956, @stelfrag)
• Fixed buffer overflow when processing cloud rooms during Agent claiming on startup (#19954, @stelfrag)
• Updated Agent status reporting system with enhanced crash diagnostics, anonymized stack traces, and ACLK connection status tracking (#19953, #19957, #19959 @ktsaou)
• Fixed thread creation issues by adding retry logic when system resource limits are temporarily reached (#19951, @stelfrag)
• Added monitoring of IIS Application Pool metrics to Windows collector (#19950, @thiagoftsm)
• Improved metadata thread stability with better shutdown handling and enhanced event loop management (#19929, @stelfrag)
• Fixed potential deadlocks by processing alert configuration database operations asynchronously through the metadata thread (#19885, @stelfrag)
• Reworked shared memory management in eBPF plugin for more reliable interprocess communication (#19844, @thiagoftsm)

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Make use of the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!

Netdata v2.3.1 is a patch release to address issues discovered since v2.3.0.

This patch release provides the following bug fixes and updates:

• Fixed debug information handling by including it in default builds while disabling separate debuginfo packages for Debian-based distributions (#19946, #19948 @Ferroin)
• Fixed static build configuration to avoid unnecessary libunwind compilation (#19939, @Ferroin)
• Improved detection of low memory conditions with more aggressive monitoring (#19938, @ktsaou)
• Fixed installation path for updater script crontab configuration (#19935, @ralphm)
• Fixed validation of database page size limits for 32-bit compression format (#19932, @stelfrag)
• Fixed compilation issues when building without database engine support or with address sanitizer enabled (#19930, @stelfrag)
• Added additional system resource metrics to status file including memory usage and enhanced out-of-memory protection information (#19928, #19937 @ktsaou)
• Fixed security issue by preventing exposure of absolute file paths in web server responses (#19925, @ktsaou)
• Fixed security vulnerability in daemon status file handling by using file descriptor-based permissions to prevent race conditions (#19924, @Ferroin)
• Removed insecure SVG generation endpoint to prevent potential code injection vulnerabilities (#19919, @ilyam8)
• Fixed unaligned memory access in socket message buffer by properly aligning memory structures (#19917, @vkalintiris)
• Fixed ACLK synchronization by ensuring thread initialization completes before proceeding with startup (#19916, @stelfrag)
• Fixed issue where commands could be queued before ACLK initialization was complete (#19914, @ktsaou)
• Fixed potential crash when database engine encounters null data files during range operations (#19913, @ktsaou)
• Fixed Agent status reporting to handle first-run scenarios when no previous status file exists (#19912, @ktsaou)
• Added initial implementation of libbacktrace for improved crash diagnostics (#19910, @ktsaou)
• Fixed reliability calculation to properly handle normal Agent exit cases (#19909, @ktsaou)
• Added enhanced shutdown diagnostics with timeouts and improved system information in crash reports including cloud provider details (#19903, @ktsaou)

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Make use of the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!

Table of Contents

• Netdata Growth
• Summary
• Highlights
• Acknowledgments
• Contributions
  • Collectors
  • Packaging/Installation
  • Documentation
  • Other Notable Changes
• Deprecation notice
• Support options

Netdata Growth

• 1.5 million downloads per day
• 73.7k GitHub stars!
• 656.1M Docker Hub pulls!

Netdata continues to experience phenomenal growth, with over 1.5 million downloads daily through Cloudflare and Docker Hub, fueling user observability worldwide.

Thanks to your unwavering support ❤️, Netdata is the leader in the observability category in the CNCF landscape, ahead of all other solutions, including Elasticsearch, Grafana, and Prometheus, in GitHub stars. This demonstrates the trust and admiration of our community.

This success drives rapid adoption among enterprises, reflecting the growing recognition of Netdata as the go-to observability solution for both cloud-native and on-premises environments. Our commitment remains steadfast: to deliver cutting-edge, AI-powered observability with unmatched performance and simplicity—all while being significantly more affordable.

We are also proud to see our users and customers experience high-scale setups, achieving reliable multi-million samples/s setups, effortlessly, streamlining their operations with Netdata.

As we evolve, our focus on empowering businesses with higher-fidelity AI insights ensures Netdata remains the easiest and fastest way to optimize infrastructure and applications at any scale. 🚀

Do you like Netdata? Give Netdata a ⭐ too, on GitHub!

Release Summary

Netdata 2.3 delivers significant enhancements to monitoring reliability and scalability:

• Crash Handling & Reporting: A zero-sampling system that captures and analyzes agent crashes with complete diagnostic information, significantly improving reliability across diverse environments.
• Extreme Cardinality Protection: Automatic safeguards that maintain performance in high-scale environments with millions of time series while intelligently managing metadata retention.
• Nodes Ephemerality & Streaming Alerts: A sophisticated approach to handling node connections in distributed environments, reducing alert noise by distinguishing between permanent and ephemeral nodes.
• SNMP Service Discovery: A new system automatically finds and monitors SNMP-enabled devices on configured networks, eliminating manual configuration.

Release Highlights

Nodes Ephemerality & Streaming Alerts

Netdata 2.3 implements a more sophisticated approach to handling node connections in distributed environments. We now define ephemeral nodes as "nodes that are expected to disconnect without raising alerts", enabling smarter monitoring of dynamic infrastructure.

Learn more about managing ephemeral nodes.

Extreme Cardinality Protection

Netdata 2.3 introduces automatic protection against extreme cardinality issues when combining high-dimensional metrics with long retention periods. This system:

This protection maintains Netdata's performance even in high-scale environments with millions of time series, while still allowing unlimited cardinality for high-resolution data. Learn more about configuring this feature.

Crash Handling & Reporting

We've implemented a powerful, zero-sampling crash monitoring system that captures and analyzes agent restarts and crashes with complete diagnostic information. This solution leverages systemd's journal for flexible, scalable event tracking without additional licensing costs. With anonymous telemetry enabled, this system helps us identify critical issues across diverse environments, significantly improving Netdata's reliability for all users. Read more about our approach in this blog post.

SNMP Discovery

Netdata 2.3 adds an SNMP service discovery system that automatically finds and monitors SNMP-enabled devices on your networks.

Netdata v2.2.6 is a patch release to address issues discovered since v2.2.5.

This patch release provides the following bug fixes and updates:

• Fixed log2journal to properly handle empty JSON arrays and objects in log parsing (#19685, @ktsaou)
• Fixed MariaDB collector to handle CPU time values reported in microseconds since version 10.11.11 (#19683, @ilyam8)
• Added user action logging to dyncfg while ensuring sensitive information is hidden when dashboard is in anonymous mode (#19674, @ktsaou)
• Disabled legacy Go sensors collector in preparation for removal, as metrics are now collected by the C implementation (#19667, @ilyam8)
• Added user action logging to go.d.plugin dyncfg (#19666, #19672, #19673, #19676 @ilyam8)
• Added periodic event loop processing to prevent database engine starvation (#19661, @ktsaou)
• Added deployment metrics collection to KubeState collector, including replica status and availability alerts (#19657, #19662, #19664 @ilyam8)
• Added agent timezone information to host labels (#19656, @ktsaou)
• Added fallback to standard memory allocation when onewayalloc allocator fails (#19646, @ktsaou)
• Fixed Windows log collection to append new fields instead of renumbering them (#19640, @ktsaou)
• Improved error handling in Windows perflib collector to be more robust (#19631, @ktsaou)
• Prevented Sentry report flooding during database initialization failures (#19628, @ktsaou)
• Added a new alert to detect system reboots (#19624, @ilyam8)
• Fixed SSL certificate handling in statically linked libcurl by properly configuring certificate paths during build (#19622, @ktsaou)
• Added additional context information to Sentry crash reports for better debugging (#19608, @vkalintiris)
• Added automatic detection of system DNS servers from /etc/resolv.conf in DNS query collector (#19562, @ilyam8)

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Make use of the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!

Netdata v2.2.5 is a patch release to address issues discovered since v2.2.4.

This patch release provides the following bug fixes and updates:

• Fixed MariaDB collector to handle CPU time values reported in microseconds since version 11.4.5 (#19618, @ilyam8)
• Changed clickhouse_replicated_readonly_tables info to clarify read-only state conditions for replicated tables (#19616, @ilyam8)
• Fixed indirect cloud access to function properly when an Agent is claimed but temporarily offline by using Parent's claim ID (#19611, @ktsaou)
• Added data interpolation to median and percentile calculations in the stats.d plugin (#19606, @ktsaou)
• Fixed potential database access issues during shutdown by properly closing the database when startup errors occur (#19600, @stelfrag)
• Fixed a memory corruption issue by properly dequeuing contexts from hub before deletion (#19599, @ktsaou)
• Fixed various issues in journal watcher, including improved event parsing, corrected path handling, and resolved memory-related bugs (#19592, @ktsaou)
• Fixed NVMe device discovery to properly handle device paths in nvme-cli v2.11 (#19577, @ilyam8)
• Fixed MySQL collector to properly respect the global variables refresh interval instead of collecting them on every iteration (#19549, @arkamar)
• Fixed several CI/build system issues, including static build conditions, POWER8+ build hosts, Ubuntu runner versions, and compiler flags (#19575, #19579, #19614, #19619, @Ferroin)
• Fixed Amazon Linux aarch64 package builds by migrating to GitHub's ARM CI runners (#19427, @Ferroin)

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Make use of the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!

Netdata v2.2.4 is a patch release to address issues discovered since v2.2.3.

This patch release provides the following bug fixes and updates:

• Fixed go.d plugin to properly clean up charts for stopped and removed jobs to prevent stale alerts (#19573, @ilyam8)
• Fixed a memory leak in the health component (#19569, @stelfrag)
• Added a new agent status to indicate when connection to the Cloud is in progress (#19564, @stelfrag)
• Fixed Windows collector to properly monitor multiple MSSQL instances running simultaneously (#19559, @thiagoftsm)

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Make use of the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!

Netdata v2.2.3 is a patch release to address issues discovered since v2.2.2.

This patch release provides the following bug fixes and updates:

• Fixed an issue with newly created and claimed agents not immediately appearing in the cloud (#19542 @stelfrag)
• Fixed database engine to properly handle files larger than 4GB by correcting 32-bit variable limitations (#19538, #19539 @ktsaou)
• Fixed an issue where contexts remained in a non-live state after child reconnections due to incorrectly cached collection flags (#19536 @ktsaou)

Support options

As we grow, we stay committed to providing the best support ever seen from an open-source solution. Should you encounter an issue with any of the changes made in this release or any feature in the Netdata Agent, feel free to contact us through one of the following channels:

• Netdata Learn: Find documentation, guides, and reference material for monitoring and troubleshooting your systems with Netdata.
• GitHub Issues: Make use of the Netdata repository to report bugs or open a new feature request.
• GitHub Discussions: Join the conversation around the Netdata development process and be a part of it.
• Community Forums: Visit the Community Forums and contribute to the collaborative knowledge base.
• Discord Server: Jump into the Netdata Discord and hang out with like-minded sysadmins, DevOps, SREs, and other troubleshooters. More than 2000 engineers are already using it!