What's Changed

Timeseries Labeling Change

• As part of the KOF 1.2.0 overhaul of metrics collection and representation, we switched from the victoria-metrics-k8s-stack metrics and dashboards to opentelemetry-kube-stack metrics and kube-prometheus-stack dashboards.

• Some of the previously collected metrics have slightly different labels.

• If consistency of timeseries labeling is important, users are advised to conduct relabeling of the corresponding timeseries in the metric storage by running a retroactive relabeling procedure of their preference.

• A possible reference solution here would be to use Rules backfilling via vmalert.

• The labels that would require renaming are these:

  • Replace job="integrations/kubernetes/kubelet" with job="kubelet", metrics_path="/metrics".
  • Replace job="integrations/kubernetes/cadvisor" with job="kubelet", metrics_path="/metrics/cadvisor".
  • Replace job="prometheus-node-exporter" with job="node-exporter".

• test: add support bundle to troubleshoot CI issues by @gmlexx in k0rdent#363

• feat: Speedup kof release workflow by @AndrejsPon00 in k0rdent#365

• docs: add data collection recipes by @gmlexx in k0rdent#379

• fix: Do not replace expr with int zero 0 if expr is not overridden by @denis-ryzhkov in k0rdent#380

• fix: batch processor order by @gmlexx in k0rdent#381

• feat: Add cluster filter to Victoria Logs dashboard by @AndrejsPon00 in k0rdent#382

• docs: add data sending customization recipe by @gmlexx in k0rdent#383

• fix: update dev-child-coredns setup to wait ingress ip provisioning by @gmlexx in k0rdent#386

• chore: Use kgst 1.0.0 by @denis-ryzhkov in k0rdent#385

• feat: Option to allow regional cluster to be in another namespace than the child cluster by @denis-ryzhkov in k0rdent#390

• feat: add dashboard to monitor OpenTelemetry Collectors metrics across all clusters by @AndrejsPon00 in k0rdent#391

• fix: Moved PromxyServerGroup and GrafanaDatasource to namespace of ClusterDeployment by @denis-ryzhkov in k0rdent#394

• feat: Show trend insights for collectors metrics in KOF dashboard by @AndrejsPon00 in k0rdent#395

• feat: Add collectors list page to KOF dashboard by @AndrejsPon00 in k0rdent#398

• feat: Switch Metric Collectors to Opentelemetry-kube-stack by @aglarendil in k0rdent#273

• feat: optional regional cluster by @gmlexx in k0rdent#396

• feat: Added the clusterNamespace metrics label as cluster name may be not unique by @denis-ryzhkov in k0rdent#401

• fix: Added letter v to cert-manager:v1.16.4 for compatibility with all registries by @denis-ryzhkov in k0rdent#404

• fix: patch up incorrect alertmanager rules job label variable by @aglarendil in k0rdent#409

• fix: Add debug info to kof-operator build to prevent auto-instrumentation crash by @AndrejsPon00 in k0rdent#414

• feat: Collect internal metrics from victoria metrics/logs services by @AndrejsPon00 in k0rdent#403

• fix: Enabled "self metrics" of kube-state-metrics by @denis-ryzhkov in k0rdent#415

• fix: change cluster filter label in dashboards by @gmlexx in k0rdent#417

• fix: Increase collectors memory to prevent OOM by @AndrejsPon00 in k0rdent#408

• fix: Prevent regional clusters selection from cluster-deployments-events dashboard by @AndrejsPon00 in k0rdent#418

• feat: add handler to fetch internal metrics from collectors by @AndrejsPon00 in k0rdent#387

• fix: Incorrect routing fallback for KOF UI on server side by @AndrejsPon00 in k0rdent#419

• chore: update standalone cluster templates version to match the latest kcm by @gmlexx in k0rdent#420

• fix: k8s events processor by @aglarendil in k0rdent#421

• fix: fix collectors observability a bit by @aglarendil in k0rdent#424

• feat: Add ability to extract metrics port from annotation by @AndrejsPon00 in k0rdent#423

• fix: Made opentelemetry-go-instrumentation path and version compatible with all image registries by @denis-ryzhkov in k0rdent#426

• chore: Update dependencies to resolve vulnerabilities in KOF UI by @AndrejsPon00 in k0rdent#422

• chore: update charts version for release 1.2.0 by @gmlexx in k0rdent#427

Full Changelog: k0rdent/kof@v1.1.0...v1.2.0-rc2