The current metrics have in my tests always the uri="UNKNOWN". With this setup I can't distinguish different target hosts, which makes this metric difficult to use to monitor the latency of a specific endpoint.

At the same time the tag should be low cardinality, so I'd suggest to use only the hostname and port there, maybe the path, but never any query parameters.

We can map the URI using:

                      MicrometerHttpRequestExecutor.builder(Metrics.globalRegistry)
                            .uriMapper(request -> {
                                // TODO
                                return "";
                            })
                            .build()

It's simple enough to record only the hostname and port, however including the path is trickier as this needs to be defined on a case-by-case basis. A workaround is to make use of a custom header and then use this in the uri mapper. For example:

// Request code
HttpGet request = new HttpGet("http://service/api/v1/users/123");
request.addHeader("X-Metrics-Template", "/api/v1/users/{id}");

// Mapper
.uriMapper(request -> {
    Header header = request.getFirstHeader("X-Metrics-Template");
    return (header != null) ? header.getValue() : "UNKNOWN"; // This could also default to just the host+port
})

The downside to this is that we will need to define the X-Metrics-Template throughout our code.

Adding a template is possibly unrealistic, and IMHO shouldn't be done as part of this PR. Let's go with hostname and port, and still add some prevention for a maximum number of tags.

The Internet claims that there is some MeterFilter#maximumAllowableTags but I wouldn't know how to apply it.

Do we really want to limit the number of host and port tags? I don't think it would be common to need more than 100, but I would imagine those metrics should always be desired instead of some hosts being arbitrarily ignored.

For the uri tag, unless a uriMapper is defined, the uri is always UNKNOWN, which is why the MetricsDistTest is failing.

If we want the 100 limit to apply to all uri fields then we would need to do:

.uriMapper(request -> request.getRequestLine().getUri());

However, if I understand the MeterFilter correctly, I think that would limit the total number of unique uri tags across all hosts to 100, which doesn't make sense if you consider that the client might be reaching out to multiple different hosts each of which have endpoints like /user/{id} where id changes each call.

My suggestion would be that we remove the tag limit and define the following mapper:

.uriMapper(request -> {
    Header header = request.getFirstHeader("X-Metrics-Template");
    return (header != null) ? header.getValue() : "UNKNOWN"; // This could also default to just the host+port
})

Users of the client can then add the X-Metrics-Template if desired. We could also have a follow-up task to add X-Metrics-Template throughout the code.

This was supposed to be a small change, but the effort we're putting into this is getting out of hand.

There is AFAIK a default URI mapper in place that applies the template. We're not using the template, maybe a 3rd party library does. I don't have any intentions to appliy the X-Metrics-Template in our code.

Limiting is a recommended practice, otherwise you can run out of memory. We don't know how many different URLs are called - it can become a lot if there are a lot of realms.

If we think it is not useful as it is, we could also divert to not implementing it at all: If you can't distinguish the targets, you can't make sense of the metrics. At the moment you can't even distinguish the sources (which realm, which for example client configuration).

So would we skip this one as we think it will not be very useful?

Happy to jump on a call tomorrow.

Following up from our call earlier, I have implemented a default uriMapper which uses the value of the X-Metrics-Template header if present for the uri tag, otherwise the value will be "UNKNOWN".

I have updated the various identity providers to set the X-Metrics-Template where possible. In the case of user configurable URLs, I have used the url path as the template value when the URL is not expected to contain specific resource identifiers in the path.

I have also made it so that the maximum number of tags can be defined via the CLI option http-client-metrics-tag-limit, which currently defaults to 100.

👍 thanks for the change

We may still need a UriMapper though, see e9ba1b9#r2695215811

The URI Mapper is not doing what you think it is doing. When I tested it, the request line didn't include a host.