Could this go out in a minor version release or will OTel be looked at further down the line towards Npgsql 10?

@kerams we don't generally do minor releases - yeah, this will have to wait for 10.

Bump. .NET 10 (and supposedly npgsql) is around the corner and I'd hate to see this going to waste for another year.

The new multiplexing tests are randomly failing in local (often) and in CI (sometimes).
It seems to happen when an extra "connection open" tracing activity is added when initiating a copy operation.
Any clue on a possible explanation?

The new multiplexing tests are randomly failing in local (often) and in CI (sometimes).
It seems to happen when an extra "connection open" tracing activity is added when initiating a copy operation.
Any clue on a possible explanation?

Ye, I do have an idea. You have another query just before initiating COPY, and with multiplexing there is a race between a reader from that query being disposed (this is when we signal the connector that it's free to handle another query) and starting COPY operation. Specifically, the issue is that after the reader is disposed, connector first checks whether it has other queries, and if there are none, then it returns to the pool. But because it races with COPY, what happens is that that connector is not yet in the pool and so for COPY we create another connector. Limiting the pool size to 1 should do the trick.

Limiting the pool size to 1 should do the trick.

I tried, but the test hangs while renting a new connector.

I have found another solution, by setting EnablePhysicalOpenTracing to false.

@vonzshik FYI I have already prepared a branch to instrument other COPY operations.
See main...manandre:npgsql:otel-text

@vonzshik FYI I have already prepared a branch to instrument other COPY operations. See main...manandre:npgsql:otel-text

@manandre that's actually great! While I didn't check every single thing, overall it seems quite good. Main thing left there is it add callbacks to NpgsqlTracingOptionsBuilder. Overall that should be 3 more callbacks, one for enrichment, one for span name and one for filtering. To differentiate between different copy operations we'll use an enum. So something like:

public enum CopyOperationType
{
    BinaryImport,
    BinaryExport,
    TextImport,
    TextExport,
    RawBinary
}

public NpgsqlTracingOptionsBuilder ConfigureCopyOperationFilter(Func<string, CopyOperationType, bool>? copyOperationFilter)
{
	// TODO
	return this;
}

public NpgsqlTracingOptionsBuilder ConfigureCopyOperationEnrichmentCallback(Action<Activity, string, CopyOperationType>? copyOperationEnrichmentCallback)
{
	// TODO
	return this;
}

public NpgsqlTracingOptionsBuilder ConfigureCopyOperationSpanNameProvider(Func<string, CopyOperationType, string?>? copyOperationSpanNameProvider)
{
	// TODO
	return this;
}

UPD: discussed with @roji, we probably do not want enum at all, so just 3 callbacks and reuse them for every COPY operation.

@manandre as we'd like to release Npgsql 10, I think I'll take over this PR to bring it to completion ASAP (unless I run into issues). Can you please comment here in the next couple hours if you're available to do work on it?

@roji I am on it, I have already implemented the new callbacks, I still have to add an unit test.
I have just pushed my latest changes.
I have planned to continue working on it this evening. But feel free to work on it if you need it to be done on shorter term.
Keep me posted!

Thank you, sirs.

@manandre I think this is almost the last thing left that we want to get in for 10. Let me know where you're at and how you'd like to finish this - I'll get a bit to eat and will be back in ~1 hour to review etc.

Unit test added.
@roji Do you want I add other copy operations in this PR or in another one?

Let's wrap everything up in this single PR. It's very likely we'll be releasing Npgsql 10.0, I'll be available from the morning to review and iterate over it.

@roji Everything added and rebased on latest main branch

@roji I am not available before this evening. You can take it over.

FYI am starting to work on this now.

await using var dataSource = CreateDataSource(ds => ds.ConfigureTracing(o => o.EnablePhysicalOpenTracing(false)));

@roji It must be applied in all copy tests to avoid random failures.

It must be applied in all copy tests to avoid random failures.

@manandre you're right, I forgot about multiplexing when working on the tests.