Thanks to visit codestin.com
Credit goes to GitHub.com

Skip to content

Add GC duration histogram#1854

Open
zeitlinger wants to merge 12 commits intomainfrom
gniadeck/gc-pause-histogram
Open

Add GC duration histogram#1854
zeitlinger wants to merge 12 commits intomainfrom
gniadeck/gc-pause-histogram

Conversation

@zeitlinger
Copy link
Member

Supercedes #1738

introduces a Prometheus histogram metric jvm_gc_duration_seconds to record JVM garbage collection pause durations. existing metrics provide limited visibility into long GC pauses, making it difficult to detect latency spikes. this change leverages the already registered GC notifications (as used in JvmMemoryPoolAllocationMetrics) to capture pause durations without additional instrumentation.

the histogram uses 0.01, 0.1, 1, 10 buckets and includes labels for gc name, action, and cause, enabling detailed monitoring of both short and long GC pauses. this addresses the lack of visibility highlighted in community discussions such as this one. buckets are also defined according to the opentelemetry semantic conventions spec

Example result:

# HELP jvm_gc_duration_seconds JVM GC pause duration histogram.
# TYPE jvm_gc_duration_seconds histogram
jvm_gc_duration_seconds_bucket{action="end of minor GC",cause="G1 Evacuation Pause",gc="G1 Young Generation",le="0.01"} 806
jvm_gc_duration_seconds_bucket{action="end of minor GC",cause="G1 Evacuation Pause",gc="G1 Young Generation",le="0.1"} 806
jvm_gc_duration_seconds_bucket{action="end of minor GC",cause="G1 Evacuation Pause",gc="G1 Young Generation",le="1.0"} 806
jvm_gc_duration_seconds_bucket{action="end of minor GC",cause="G1 Evacuation Pause",gc="G1 Young Generation",le="10.0"} 806
jvm_gc_duration_seconds_bucket{action="end of minor GC",cause="G1 Evacuation Pause",gc="G1 Young Generation",le="+Inf"} 806
jvm_gc_duration_seconds_count{action="end of minor GC",cause="G1 Evacuation Pause",gc="G1 Young Generation"} 806
jvm_gc_duration_seconds_sum{action="end of minor GC",cause="G1 Evacuation Pause",gc="G1 Young Generation"} 0.7360000000000005

gniadeck and others added 11 commits February 6, 2026 17:06
Signed-off-by: gniadeck <[email protected]>
Signed-off-by: Gregor Zeitlinger <[email protected]>
…o list

Replace the boolean property with a comma-separated list of OTel metric
names, giving users fine-grained control over which metrics use
OpenTelemetry Semantic Conventions. Use "*" to enable all metrics.

Signed-off-by: Gregor Zeitlinger <[email protected]>
@zeitlinger zeitlinger self-assigned this Feb 6, 2026
Signed-off-by: Gregor Zeitlinger <[email protected]>
public class JvmGarbageCollectorMetrics {

private static final String JVM_GC_COLLECTION_SECONDS = "jvm_gc_collection_seconds";
private static final String JVM_GC_DURATION = "jvm.gc.duration";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we be consistent with the way the other metric is defined?

Suggested change
private static final String JVM_GC_DURATION = "jvm.gc.duration";
private static final String JVM_GC_DURATION = "jvm_gc_duration";

Comment on lines +115 to +124
if (!GarbageCollectionNotificationInfo.GARBAGE_COLLECTION_NOTIFICATION.equals(
notification.getType())) {
return;
}

GarbageCollectionNotificationInfo info =
GarbageCollectionNotificationInfo.from(
(CompositeData) notification.getUserData());

observe(gcDurationHistogram, info);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

try/catch here to avoid crashing the JVM is the notification listener throws something

Suggested change
if (!GarbageCollectionNotificationInfo.GARBAGE_COLLECTION_NOTIFICATION.equals(
notification.getType())) {
return;
}
GarbageCollectionNotificationInfo info =
GarbageCollectionNotificationInfo.from(
(CompositeData) notification.getUserData());
observe(gcDurationHistogram, info);
try {
if (!GarbageCollectionNotificationInfo.GARBAGE_COLLECTION_NOTIFICATION.equals(
notification.getType())) {
return;
}
GarbageCollectionNotificationInfo info =
GarbageCollectionNotificationInfo.from(
(CompositeData) notification.getUserData());
observe(gcDurationHistogram, info);
} catch (Exception e) {
logger.warning(
"Exception while processing garbage collection notification: " + e.getMessage());
}

.name(JVM_GC_DURATION)
.unit(Unit.SECONDS)
.help("Duration of JVM garbage collection actions.")
.labelNames("jvm.gc.action", "jvm.gc.name", "jvm.gc.cause")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jvm.gc.cause is still experimental, should we hold off on that one until it stabilizes?

registerNotificationListener(gcDurationHistogram);
}

private void registerNotificationListener(Histogram gcDurationHistogram) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

taking a look at the OTel implementation, they also track the listeners and do a cleanup, not sure if it makes sense for us to implement something like that, by making JvmMetrics an AutoClosable. Looks like it ends up requiring a lot of changes, not sure if its worth it

Another thing they do is check to see if the class exists first , which I think would be good to do here too

}

@Test
public void testNonOtelMetricsAbsentWhenUseOtelEnabled() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public void testNonOtelMetricsAbsentWhenUseOtelEnabled() {
void testNonOtelMetricsAbsentWhenUseOtelEnabled() {


@Test
@SuppressWarnings("rawtypes")
public void testGCDurationHistogramLabels() throws Exception {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
public void testGCDurationHistogramLabels() throws Exception {
void testGCDurationHistogramLabels() throws Exception {

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants