Thanks to visit codestin.com
Credit goes to github.com

Skip to content

WaitFor: Add health checks to remaining resources #5645

@mitchdenny

Description

@mitchdenny

With the merge of #5394 and some of the subsequent enhancements around health check integration (#5515) its now time to scale out the support for WaitFor for all of our resources. Essentially each AddXYZ method for a resource needs to register a health check with DI and add a matching HealthCheckAnnotation. Here are two examples:

Redis

public static IResourceBuilder<RedisResource> AddRedis(this IDistributedApplicationBuilder builder, string name, int? port = null)
{
ArgumentNullException.ThrowIfNull(builder);
var redis = new RedisResource(name);
string? connectionString = null;
builder.Eventing.Subscribe<ConnectionStringAvailableEvent>(redis, async (@event, ct) =>
{
connectionString = await redis.GetConnectionStringAsync(ct).ConfigureAwait(false);
});
var healthCheckKey = $"{name}_check";
builder.Services.AddHealthChecks().AddRedis(sp => connectionString ?? throw new InvalidOperationException("Connection string is unavailable"), name: healthCheckKey);
return builder.AddResource(redis)
.WithEndpoint(port: port, targetPort: 6379, name: RedisResource.PrimaryEndpointName)
.WithImage(RedisContainerImageTags.Image, RedisContainerImageTags.Tag)
.WithImageRegistry(RedisContainerImageTags.Registry)
.WithHealthCheck(healthCheckKey);
}

Postgres

public static IResourceBuilder<PostgresServerResource> AddPostgres(this IDistributedApplicationBuilder builder,
string name,
IResourceBuilder<ParameterResource>? userName = null,
IResourceBuilder<ParameterResource>? password = null,
int? port = null)
{
ArgumentNullException.ThrowIfNull(builder);
ArgumentNullException.ThrowIfNull(name);
var passwordParameter = password?.Resource ?? ParameterResourceBuilderExtensions.CreateDefaultPasswordParameter(builder, $"{name}-password");
var postgresServer = new PostgresServerResource(name, userName?.Resource, passwordParameter);
string? connectionString = null;
builder.Eventing.Subscribe<ConnectionStringAvailableEvent>(postgresServer, async (@event, ct) =>
{
connectionString = await postgresServer.GetConnectionStringAsync(ct).ConfigureAwait(false);
var lookup = builder.Resources.OfType<PostgresDatabaseResource>().ToDictionary(d => d.Name);
foreach (var databaseName in postgresServer.Databases)
{
if (!lookup.TryGetValue(databaseName.Key, out var databaseResource))
{
throw new DistributedApplicationException($"Database resource '{databaseName}' under Postgres server resource '{postgresServer.Name}' not in model.");
}
var connectionStringAvailableEvent = new ConnectionStringAvailableEvent(databaseResource, @event.Services);
await builder.Eventing.PublishAsync<ConnectionStringAvailableEvent>(connectionStringAvailableEvent, ct).ConfigureAwait(false);
var beforeResourceStartedEvent = new BeforeResourceStartedEvent(databaseResource, @event.Services);
await builder.Eventing.PublishAsync(beforeResourceStartedEvent, ct).ConfigureAwait(false);
}
});
var healthCheckKey = $"{name}_check";
builder.Services.AddHealthChecks().AddNpgSql(sp => connectionString ?? throw new InvalidOperationException("Connection string is unavailable"), name: healthCheckKey, configure: (connection) =>
{
// HACK: The Npgsql client defaults to using the username in the connection string if the database is not specified. Here
// we override this default behavior because we are working with a non-database scoped connection string. The Aspirified
// package doesn't have to deal with this because it uses a datasource from DI which doesn't have this issue:
//
// https://github.com/npgsql/npgsql/blob/c3b31c393de66a4b03fba0d45708d46a2acb06d2/src/Npgsql/NpgsqlConnection.cs#L445
//
connection.ConnectionString = connection.ConnectionString + ";Database=postgres;";
});
return builder.AddResource(postgresServer)
.WithEndpoint(port: port, targetPort: 5432, name: PostgresServerResource.PrimaryEndpointName) // Internal port is always 5432.
.WithImage(PostgresContainerImageTags.Image, PostgresContainerImageTags.Tag)
.WithImageRegistry(PostgresContainerImageTags.Registry)
.WithEnvironment("POSTGRES_HOST_AUTH_METHOD", "scram-sha-256")
.WithEnvironment("POSTGRES_INITDB_ARGS", "--auth-host=scram-sha-256 --auth-local=scram-sha-256")
.WithEnvironment(context =>
{
context.EnvironmentVariables[UserEnvVarName] = postgresServer.UserNameReference;
context.EnvironmentVariables[PasswordEnvVarName] = postgresServer.PasswordParameter;
})
.WithHealthCheck(healthCheckKey);
}

Remaining resources

Pri0

P1

Some of these resources don't have health checks and possibly won't need them but we should go through the list and very each one and create issues. We also need test coverage for the WaitFor behavior to verify that it is working as expected for each resource (example):

public async Task VerifyWaitForOnRedisBlocksDependentResources()
{
var cts = new CancellationTokenSource(TimeSpan.FromMinutes(3));
using var builder = TestDistributedApplicationBuilder.CreateWithTestContainerRegistry(testOutputHelper);
// We use the following check added to the Redis resource to block
// dependent reosurces from starting. This means we'll have two checks
// associated with the redis resource ... the built in one and the
// one that we add here. We'll manipulate the TCS to allow us to check
// states at various stages of the execution.
var healthCheckTcs = new TaskCompletionSource<HealthCheckResult>();
builder.Services.AddHealthChecks().AddAsyncCheck("blocking_check", () =>
{
return healthCheckTcs.Task;
});
var redis = builder.AddRedis("redis")
.WithHealthCheck("blocking_check");
var dependentResource = builder.AddRedis("dependentresource")
.WaitFor(redis); // Just using another redis instance as a dependent resource.
using var app = builder.Build();
var pendingStart = app.StartAsync(cts.Token);
var rns = app.Services.GetRequiredService<ResourceNotificationService>();
// What for the Redis server to start.
await rns.WaitForResourceAsync(redis.Resource.Name, KnownResourceStates.Running, cts.Token);
// Wait for the dependent resource to be in the Waiting state.
await rns.WaitForResourceAsync(dependentResource.Resource.Name, KnownResourceStates.Waiting, cts.Token);
// Now unblock the health check.
healthCheckTcs.SetResult(HealthCheckResult.Healthy());
// ... and wait for the resource as a whole to move into the health state.
await rns.WaitForResourceAsync(redis.Resource.Name, (re => re.Snapshot.HealthStatus == HealthStatus.Healthy), cts.Token);
// ... then the dependent resource should be able to move into a running state.
await rns.WaitForResourceAsync(dependentResource.Resource.Name, KnownResourceStates.Running, cts.Token);
await pendingStart; // Startup should now complete.
// ... but we'll shut everything down immediately because we are done.
await app.StopAsync();
}

Metadata

Metadata

Assignees

Labels

area-integrationsIssues pertaining to Aspire Integrations packages

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions