Thanks to visit codestin.com
Credit goes to github.com

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/documentation/release_notes/topics/26_6_0.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,20 @@ These properties are shared among the available OpenTelemetry components - logs,

For more details, see the link:{telemetryguide_link}[{telemetryguide_name}] guide.

= Graceful shutdown of HTTP stack

To allow for rolling updates for configuration changes or version updates, a graceful shutdown of {project_name} nodes prevents users seeing error responses when logging in or refreshing their tokens when nodes shut down.

Starting with this version, {project_name} supports a graceful shutdown of the HTTP stack.
This includes delaying a shutdown after receiving a termination signal, connection draining for HTTP/1.1 and HTTP/2 connections during that period, and a shutdown timeout to finish ongoing requests.

The defaults are a shutdown delay and a shutdown timeout of one second each.
This should be a good fit for setups where the reverse proxy is using TLS edge termination or re-encrypt, and the reverse proxy is notified about the Keycloak node shutting down at the same time as the Keycloak node.
This is a common setup for example in Kubernetes environments.

Users should adjust those values depending on their proxy setup.
See the section https://www.keycloak.org/server/reverseproxy#graceful-http-shutdown[Graceful HTTP shutdown] in the reverse proxy guide for more information.

= Custom request headers for OpenTelemetry

It is now possible to set request headers for exporting telemetry via OpenTelemetry Protocol (OTLP).
Expand Down
3 changes: 2 additions & 1 deletion docs/documentation/tests/src/test/resources/ignored-links
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,5 @@ https://docs.kantarainitiative.org*
https://saml.xml.org*

# To be removed once KC 26.6.0 is released
https://www.keycloak.org/securing-apps/dpop
https://www.keycloak.org/securing-apps/dpop
https://www.keycloak.org/server/reverseproxy#graceful-http-shutdown
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,13 @@ When running the server in dev mode on a platform other than Windows Subsystem F
This ensures your dev instance won't be accessible from
other machines. If you want the previous behavior of binding to all interfaces, then explicitly set `http-host` to `0.0.0.0`.

=== Graceful shutdown of HTTP stack

Starting with this version, {project_name} will apply a shutdown delay and a shutdown timeout of one second each to allow a graceful termination.

Users should adjust those values depending on their proxy setup.
See the section https://www.keycloak.org/server/reverseproxy#graceful-http-shutdown[Graceful HTTP shutdown] in the reverse proxy guide for more information.

=== `X-Forwarded-Prefix` Header is now supported

With `proxy-headers` set to `xforwarded`, the server can determine the proxy context path from the `X-Forwarded-Prefix` header.
Expand Down
10 changes: 9 additions & 1 deletion docs/guides/observability/health.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ These endpoints respond with HTTP status `200 OK` on success or `503 Service Una
{
"status": "UP",
"checks": [
{
"name": "Graceful Shutdown",
"status": "UP"
},
{
"name": "Keycloak cluster health check",
"status": "UP"
Expand Down Expand Up @@ -120,9 +124,13 @@ The table below shows the available checks.
|Returns the status of the cluster (network partitions).
|No

|Graceful shutdown
|Will start to return "DOWN" once the pre-shutdown phase started.
|No

|===

For some checks, you'll need to also enable metrics as indicated by the `Requires Metrics` column. To enable metrics
For some checks, you'll need to also enable metrics as indicated by the *Requires Metrics* column. To enable metrics
use the `metrics-enabled` option as follows:

<@kc.build parameters="--health-enabled=true --metrics-enabled=true"/>
Expand Down
138 changes: 134 additions & 4 deletions docs/guides/server/reverseproxy.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@
<@tmpl.guide
title="Configuring a reverse proxy"
summary="Configure {project_name} with a reverse proxy, API gateway, or load balancer."
includedOptions="proxy-* hostname hostname-admin http-relative-path">

includedOptions="proxy-* hostname hostname-admin http-relative-path shutdown-delay shutdown-timeout">

Distributed environments frequently require the use of a reverse proxy. {project_name} offers several options to securely integrate with such environments.

Expand Down Expand Up @@ -53,7 +54,7 @@ By default {project_name} is exposed through the root context path (`/`). If the
- Use a simple hostname for the `hostname` option, `xforwarded` for the `proxy-headers` option, and have the proxy set the `X-Forwarded-Prefix` header.
- Use a full URL for the `hostname` option including the proxy context path, for example using `--hostname=https://my.keycloak.org/auth` if {project_name} is exposed through the reverse proxy on `/auth`.
- Change the context path of {project_name} itself to match the context path for the reverse proxy using the `http-relative-path` option.

For more details on exposing {project_name} on different hostname or context path incl. Administration REST API and Console, see <@links.server id="hostname"/>.

== Enable sticky sessions
Expand Down Expand Up @@ -254,8 +255,6 @@ The NGINX SSL/TLS module does not expose the client certificate chain. {project_
If you are using this provider, see <@links.server id="keycloak-truststore"/> for how
to configure a {project_name} Truststore.

</@tmpl.guide>

=== Configuring the rfc9440 provider

If you stick to the header names mentioned in RFC 9440, you do not need to configure any additional options after selecting the `rfc9440` provider.
Expand All @@ -278,3 +277,134 @@ The default values of the options are as follows:

If your certificate chain is longer than the given default, you must define the option with an appropriate number.
Otherwise, the provider will discard the request.

[[graceful-http-shutdown]]
== Graceful HTTP shutdown

When running {project_name} behind a reverse proxy or load balancer, graceful shutdown ensures that in-flight requests complete successfully during server termination, preventing connection errors for clients.

{project_name} enables graceful HTTP shutdown by default with configurable timeouts.

=== Understanding shutdown phases

The shutdown process consists of two phases:

*Pre-shutdown delay*:: During this phase, {project_name} signals to load balancers and proxies that it is preparing to shut down.
The server's readiness endpoint returns a "not ready" status, allowing the load balancer to stop routing new requests to this instance.
Existing TLS and HTTP keepalive connections are allowed to drain naturally.
The server continues to process existing requests.

*Shutdown timeout*:: After the pre-shutdown delay, {project_name} stops accepting new requests and waits for in-flight HTTP requests to complete.
If requests are still running after the timeout expires, the server shuts down regardless.

=== Default behavior

By default, {project_name} is configured with a 1-second pre-shutdown delay and a 1-second shutdown timeout.
These defaults work well for most standard deployments where:

* The load balancer reconfigures quickly (within 1 second)
* Most requests complete within 1 second
* The reverse proxy uses edge termination or re-encryption (not TLS passthrough)

=== Configuring shutdown timeouts

Advanced users can adjust the shutdown timeouts using CLI options based on their deployment characteristics.

[source,bash]
----
bin/kc.sh start --shutdown-delay=<duration> --shutdown-timeout=<duration>
----

Available options:

`--shutdown-delay`::
Length of the pre-shutdown phase during which the server prepares for shutdown.
This period allows for load balancer reconfiguration and draining of TLS/HTTP keepalive connections.
Default: `1s`

`--shutdown-timeout`::
The shutdown period waiting for currently running HTTP requests to finish.
Default: `1s`

Both values accept duration formats: `1s` (seconds), `500ms` (milliseconds), `2m` (minutes), etc.

=== When to adjust shutdown timeouts

Consider adjusting these values based on your deployment configuration with the following example Scenarios:

[%autowidth,cols="2,>1,>1,3a"]
|===
|Scenario |Delay |Timeout |Reason

|Load balancer polls readiness probe
|16s
|
|Assumptions:

* 5s poll interval and a load balancer reconfiguring after two successive failed probes.
* 1s for reconfiguring the proxy.
* Proxy using TLS re-encrypt or edge termination.

Calculation:

* Allow three poll cycles for the load balancer to detect shutdown and the extra time for the load balancer to perform the reconfiguration:
+
`shutdown delay = 3 * 5s + 1s`

|TLS passthrough configuration
|10‑30s
|
|Longer delay allows keepalive connections to drain naturally and receive connection close signals.

|Long-running admin API requests
|
|10‑30s
|Admin operations may take longer than typical user requests.

|Test environments / quick restarts
|0s
|500ms
|Minimize shutdown time when graceful draining is not needed.

|Deployment orchestration reconfigures the proxy and drains connections before Pod termination
|0s
|
|No pre-shutdown delay needed if proxy is already reconfigured

|Combined: TLS passthrough plus polled readiness
|26‑56s
|
|Delays add up: time for load balancer detection + connection draining

|===

==== Example configurations

For production with TLS passthrough:

[source,bash]
----
<@kc.start parameters="--shutdown-delay=30s --shutdown-timeout=1s"/>
----

For load balancers that poll readiness:

[source,bash]
----
<@kc.start parameters="--shutdown-delay=16s --shutdown-timeout=1s"/>
----

For test environments:

[source,bash]
----
<@kc.start parameters="--shutdown-delay=0s --shutdown-timeout=500ms"/>
----

[NOTE]
====
The shutdown delay affects the minimum time required for a complete server restart.
In Kubernetes environments, ensure your `terminationGracePeriodSeconds` is longer than the sum of `shutdown-delay` and `shutdown-timeout` to prevent forced termination.
====

</@tmpl.guide>
Original file line number Diff line number Diff line change
Expand Up @@ -150,4 +150,16 @@ public enum ClientAuth {
.defaultValue(Boolean.FALSE)
.build();

public static final Option<String> SHUTDOWN_TIMEOUT = new OptionBuilder<>("shutdown-timeout", String.class)
.category(OptionCategory.HTTP)
.description("The shutdown period waiting for currently running HTTP requests to finish. " + DURATION_DESCRIPTION)
.defaultValue("1s")
.build();

public static final Option<String> SHUTDOWN_DELAY = new OptionBuilder<>("shutdown-delay", String.class)
.category(OptionCategory.HTTP)
.description("Length of the pre-shutdown phase during which the server prepares for shutdown. " + DURATION_DESCRIPTION +
" This period allows for loadbalancer reconfiguration and draining of TLS/HTTP keepalive connections.")
.defaultValue("1s")
.build();
}
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,14 @@
import java.io.IOException;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.Duration;
import java.util.List;
import java.util.Optional;

import org.keycloak.common.Profile;
import org.keycloak.common.crypto.FipsMode;
import org.keycloak.config.HttpOptions;
import org.keycloak.config.OptionsUtil;
import org.keycloak.config.SecurityOptions;
import org.keycloak.quarkus.runtime.Environment;
import org.keycloak.quarkus.runtime.Messages;
Expand All @@ -19,6 +21,7 @@
import org.keycloak.quarkus.runtime.cli.command.AbstractCommand;

import io.quarkus.runtime.LaunchMode;
import io.quarkus.runtime.configuration.DurationConverter;
import io.quarkus.runtime.util.ClassPathUtils;
import io.quarkus.vertx.http.runtime.options.TlsUtils;
import io.smallrye.config.ConfigSourceInterceptorContext;
Expand Down Expand Up @@ -182,8 +185,19 @@ public List<PropertyMapper<?>> getPropertyMappers() {
.to("quarkus.rest.jackson.optimization.enable-reflection-free-serializers")
.build(),
fromOption(HttpOptions.HTTP_ACCEPT_NON_NORMALIZED_PATHS)
.build(),
fromOption(HttpOptions.SHUTDOWN_TIMEOUT)
.to("quarkus.shutdown.timeout")
.paramLabel("timeout")
.validator(HttpPropertyMappers::validateShutdownDuration)
.build(),
fromOption(HttpOptions.SHUTDOWN_DELAY)
.to("quarkus.shutdown.delay")
.paramLabel("delay")
.validator(HttpPropertyMappers::validateShutdownDuration)
.build()
);

}

@Override
Expand Down Expand Up @@ -242,4 +256,15 @@ private static String resolveMaxThreads(String value,
}
return value;
}

private static void validateShutdownDuration(String value) {
try {
Duration duration = DurationConverter.parseDuration(value);
if (duration == null || duration.isNegative()) {
throw new PropertyException("Invalid duration '%s'. Duration must be zero or positive.".formatted(value));
}
} catch (IllegalArgumentException e) {
throw new PropertyException("Invalid duration format '%s'. %s".formatted(value, OptionsUtil.DURATION_DESCRIPTION));
}
}
}
5 changes: 3 additions & 2 deletions quarkus/runtime/src/main/resources/application.properties
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ mp.openapi.extensions.smallrye.duplicateOperationIdBehavior=FAIL
# Disable Error messages from smallrye.openapi
# related issue: https://github.com/keycloak/keycloak/issues/41871
quarkus.log.category."io.smallrye.openapi.runtime.scanner.dataobject".level=off

# 3.31.1 appears to have a hibernate bug
quarkus.log.category."org.hibernate.orm.sql.exec".level=info
quarkus.log.category."org.hibernate.orm.sql.exec".level=info
# Enable shutdown delay by default as it is a built-time property, and timeout are then configured at runtime
quarkus.shutdown.delay-enabled=true
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ void testNonBlockingProbes() {
.statusCode(200);
when().get("/health/ready").then()
.statusCode(200)
.body("checks.size()", equalTo(2));
.body("checks.size()", equalTo(3));
when().get("/lb-check").then()
.statusCode(404);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,4 +108,18 @@ public void httpStoreTypeValidation(KeycloakDistribution dist) {
result.assertExitCode(-1);
result.assertMessage("ERROR: No trust store password provided");
}

@Test
@Launch({"start-dev", "--shutdown-delay=1s", "--shutdown-timeout=0s"})
public void testShutdownParametersValidValues() {
// Test that valid shutdown parameters are accepted (including 0s)
when().get("/realms/master").then().statusCode(200);
}

@Test
public void testShutdownParametersNegativeValue(KeycloakDistribution dist) {
// Test that negative values are rejected
CLIResult result = dist.run("start-dev", "--shutdown-delay=-1s");
result.assertError("Invalid duration '-1s'. Duration must be zero or positive");
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -282,6 +282,16 @@ HTTP(S):
The type of the trust store file. If not given, the type is automatically
detected based on the file extension. If 'fips-mode' is set to 'strict' and
no value is set, it defaults to 'BCFKS'.
--shutdown-delay <delay>
Length of the pre-shutdown phase during which the server prepares for
shutdown. May be an ISO 8601 duration value, an integer number of seconds,
or an integer followed by one of [ms, h, m, s, d]. This period allows for
loadbalancer reconfiguration and draining of TLS/HTTP keepalive connections.
Default: 1s.
--shutdown-timeout <timeout>
The shutdown period waiting for currently running HTTP requests to finish. May
be an ISO 8601 duration value, an integer number of seconds, or an integer
followed by one of [ms, h, m, s, d]. Default: 1s.

HTTP Access log:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -361,6 +361,16 @@ HTTP(S):
The type of the trust store file. If not given, the type is automatically
detected based on the file extension. If 'fips-mode' is set to 'strict' and
no value is set, it defaults to 'BCFKS'.
--shutdown-delay <delay>
Length of the pre-shutdown phase during which the server prepares for
shutdown. May be an ISO 8601 duration value, an integer number of seconds,
or an integer followed by one of [ms, h, m, s, d]. This period allows for
loadbalancer reconfiguration and draining of TLS/HTTP keepalive connections.
Default: 1s.
--shutdown-timeout <timeout>
The shutdown period waiting for currently running HTTP requests to finish. May
be an ISO 8601 duration value, an integer number of seconds, or an integer
followed by one of [ms, h, m, s, d]. Default: 1s.

HTTP Access log:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -330,6 +330,16 @@ HTTP(S):
The type of the trust store file. If not given, the type is automatically
detected based on the file extension. If 'fips-mode' is set to 'strict' and
no value is set, it defaults to 'BCFKS'.
--shutdown-delay <delay>
Length of the pre-shutdown phase during which the server prepares for
shutdown. May be an ISO 8601 duration value, an integer number of seconds,
or an integer followed by one of [ms, h, m, s, d]. This period allows for
loadbalancer reconfiguration and draining of TLS/HTTP keepalive connections.
Default: 1s.
--shutdown-timeout <timeout>
The shutdown period waiting for currently running HTTP requests to finish. May
be an ISO 8601 duration value, an integer number of seconds, or an integer
followed by one of [ms, h, m, s, d]. Default: 1s.

HTTP Access log:

Expand Down
Loading
Loading