feat(http): add support for mapping headers to request fields #1005

toumorokoshi · 2025-03-29T20:03:48Z

Hello! This is primarily a way to start a conversation, but I wanted to illustrate a somewhat clear proposal as well.

This proposes a method to map HTTP headers to requests and responses in a protobuf request.

This is useful for implementing IETF RFCs (such as 9110's if-match header) in HTTP+JSON apis that
are exposed in gRPC-JSON transcoding.

This was also brought up in a GitHub discussion: (see #749)

rofrankel

+100 for the spec having a way to express this, thanks for drafting a proposal @toumorokoshi!

Added some comments on the details.

google/api/http.proto

rofrankel · 2025-03-31T18:35:23Z

google/api/http.proto

+  // response header with the name of the key in the map will be populated
+  // with that value.
+  //
+  // NOTE: the referred field must be present at the top-level of the request


Suggested change

// NOTE: the referred field must be present at the top-level of the request

// NOTE: the referred field must be present at the top-level of the request

(same two comments from above apply here)

rofrankel · 2025-03-31T18:37:11Z

google/api/http.proto

  string response_body = 12;

+  // The name of fields whose values are mapped to HTTP headers in a response.
+  // If a field is in the response that matches a value in this map, an HTTP


Same comment as above, but inverted: what if multiple fields are mapped to the same header? (The use case for mapping the same field value to multiple headers also seems like a stretch, but at least the behavior is better defined...)

This issue already exists today. For example, ?a=1&b=2&a=3&b=4

The current implementation, hence the semantics, is the last one wins.

Query params are ordered, so "last one wins" is well-defined. But map entries are not ordered, so even if we wanted "last one wins" semantics, I'm not sure we could have them.

Inverting the maps seems to make this problem go away entirely. Do you see any downsides to doing so?

Can you spell out the details and let's see if it can fully address ambiguity?

I think it's better to just state that the keys and values must all be unique here, allowing only 1-1 mappings.

Is there really a good reason for allowing many-to-one in either direction? I can't think of one that wouldn't just lead to more esoteric behavior (e.g. if there's two mappings, which one wins?)

Updated to require all keys/values to be unique. thoughts?

Can you spell out the details and let's see if it can fully address ambiguity?

Are the details in the corresponding comment on the request field enough? I didn't write it all out twice, but the details are pretty much the same.

I think it's better to just state that the keys and values must all be unique here, allowing only 1-1 mappings.

There's protocol-level enforcement of key uniqueness, but value uniqueness needs to be implemented separately (e.g. by a linter). Not the end of the world, of course, just noting that it makes compliance a bit more complicated (not everyone is necessarily going to use Google's API linter or the AEP linter).

Is there really a good reason for allowing many-to-one in either direction? I can't think of one that wouldn't just lead to more esoteric behavior (e.g. if there's two mappings, which one wins?)

If you allow a header to be mapped to multiple request fields, or a response field to be mapped to multiple headers, then doesn't the "if there's two mappings, which one wins?" question go away?

wora · 2025-04-02T00:12:50Z

google/api/http.proto

+  // in the header.
+  //
+  // If a field is specified in this map, it will not be mapped to a query
+  // parameter.


The current implementation already allows a parameter be duplicated by body and HTTP query parameters. We cannot suddenly block a parameter from query parameter. That is a break changing.

If an existing field was added to this map, then yes, I think that would be a breaking change. I imagined that, even once this mapping exists, then existing fields would not map to it.

I don't think the alternative is desirable: that would imply that any field not in the body would always be able to be provided as a query parameter, and then optionally a header parameter (by adding it to the map).

There would be no way to specify that a field is exclusively honored in the header.

will not be => must not be

Note: This doesn't work for Google APIs. Every Google API request field can be specified by an HTTP request parameter, regardless whether it is mentioned by google.api.http. Please verify other implementations of gRPC transcoding. The current implementation is copy every value from request parameters into body blindly.

google/api/http.proto

wora · 2025-04-02T00:15:38Z

google/api/http.proto

  string response_body = 12;

+  // The name of fields whose values are mapped to HTTP headers in a response.
+  // If a field is in the response that matches a value in this map, an HTTP


Can you spell out the details and let's see if it can fully address ambiguity?

This proposes a method to map HTTP headers to requests and responses in a protobuf request. This is useful for implementing IETF RFCs (such as 9110's if-match header) in HTTP+JSON apis that are exposed in gRPC-JSON transcoding.

Co-authored-by: Richard Frankel <[email protected]>

toumorokoshi · 2025-04-02T05:51:10Z

@jskeet @noahdietz hi! we were discussing offline and felt you might be the right people to loop into this discussion.

We feel that a way to enable proto-HTTP header mappings is necessary for enabling gRPC APIs which can be transcoded to those that adhere to IETF RFCs (such as 9110's if-match).

This PR itself probably will not got anywhere, but I'm wondering if you have suggestions on how we could move this conversation forward.

noahdietz · 2025-04-02T17:46:43Z

Hi hi!

@jskeet @noahdietz hi! we were discussing offline and felt you might be the right people to loop into this discussion.

I've engaged with the folks that own this proto and the transcoding infra and will hopefully get someone from there involved.

This PR itself probably will not got anywhere

Right, as you know, we'd need to update the internal proto and mirror it here.

I'm wondering if you have suggestions on how we could move this conversation forward.

Let's keep discussing here, and I will try to get others to weigh in.

toumorokoshi · 2025-04-03T15:52:53Z

Hi hi!

@jskeet @noahdietz hi! we were discussing offline and felt you might be the right people to loop into this discussion.

I've engaged with the folks that own this proto and the transcoding infra and will hopefully get someone from there involved.

This PR itself probably will not got anywhere

Right, as you know, we'd need to update the internal proto and mirror it here.

I'm wondering if you have suggestions on how we could move this conversation forward.

Let's keep discussing here, and I will try to get others to weigh in.

Awesome! thank you for the help.

noahdietz · 2025-04-07T17:31:14Z

This was also brought up in a GitHub discussion: (see #749)

@toumorokoshi I'm trying to get a better understanding of where this will ultimately be used. Have you contacted Envoy or grpc-gateway, two of the major OSS projects that implement grpc-HTTP transcoding, yet?

I'm not getting much traction myself and of course most teams are already committed to work for the foreseeable future...

toumorokoshi · 2025-04-10T15:47:15Z

@noahdietz got it! Thanks for pushing.

Our main use case is to adopt IETF RFCs, specifically things like If-Match. https://datatracker.ietf.org/doc/html/rfc7232#section-3.1. We plan to recommend it for aep.dev: aep-dev/aeps#289

We'll reach out to envoy and grpc-gateway too - thanks again!

toumorokoshi · 2025-04-15T20:47:53Z

The level of effort required to support this change in the aformentioned codebases comes down to modifying these two repositories:

https://github.com/grpc-ecosystem/grpc-httpjson-transcoding for envoy.
https://github.com/grpc-ecosystem/grpc-gateway for grpc-gateway (it looks like modifying the code generation).

So it's not an intractable amount to get it working. Regarding not getting traction: @noahdietz is it possible to discuss this live with some group? I'd be great to see if there's some middle ground we can find here: both projects have said that having the annotation supported in google.api.http is a baseline requirement.

wora · 2025-04-16T01:13:16Z

google/api/http.proto

  // message type.
  string body = 7;

+  // A mapping from HTTP headers to request fields. If


A mapping from HTTP request headers to gRPC request fields. The map key corresponds to an HTTP header name (case-insensitive), and the map value corresponds to an gRPC request field path (using the same syntax as the path template).

wora · 2025-04-16T01:17:51Z

google/api/http.proto


+  // A mapping from HTTP headers to request fields. If
+  // a header has a case-insensitive match to a key in this map, the field in
+  // the request that matches the value will have its value set to the value


that matches the value => the gRPC request field that is specified by the map value will ...

wora · 2025-04-16T01:19:08Z

google/api/http.proto

+  // A mapping from HTTP headers to request fields. If
+  // a header has a case-insensitive match to a key in this map, the field in
+  // the request that matches the value will have its value set to the value
+  // in the header.


to the value in the header => to the value of the HTTP header.

We need to specify the precise encoding for string, bytes, number, etc.

wora · 2025-04-16T01:20:49Z

google/api/http.proto

+  //
+  // All keys and values of this map **must** be unique.
+  //
+  // If a field is specified in this map, it will not be mapped to a query


a field => a request field

wora · 2025-04-16T01:23:20Z

google/api/http.proto

+  // in the header.
+  //
+  // If a field is specified in this map, it will not be mapped to a query
+  // parameter.


will not be => must not be

Note: This doesn't work for Google APIs. Every Google API request field can be specified by an HTTP request parameter, regardless whether it is mentioned by google.api.http. Please verify other implementations of gRPC transcoding. The current implementation is copy every value from request parameters into body blindly.

wora · 2025-04-16T01:24:12Z

google/api/http.proto

+  // If a field is specified in this map, it will not be mapped to a query
+  // parameter.
+  //
+  // The referred field is specified using the field path syntax in a path


I don't think this is necessary if we adopt the suggestion above.

wora · 2025-04-16T01:26:24Z

google/api/http.proto

  // message type.
  string response_body = 12;

+  // A mapping from response fields to HTTP headers in the response. If a field


Similar comments as above.

CEikermann · 2025-11-18T08:42:08Z

What is the current status on this?

toumorokoshi · 2025-12-02T12:41:20Z

I haven't had a lot of success getting traction, unfortunately.

For my use case we've just forged ahead with third party header support and adding that wiring separately. For example via using context and metadata for GRPC:

rofrankel suggested changes Mar 31, 2025

View reviewed changes

wora reviewed Apr 2, 2025

View reviewed changes

toumorokoshi force-pushed the headers branch from 7c39d79 to ad009fb Compare April 2, 2025 05:23

toumorokoshi and others added 3 commits April 1, 2025 22:32

Apply suggestions from code review

a042feb

Co-authored-by: Richard Frankel <[email protected]>

make all keys/values in these maps unique

e30d200

using path template syntax for map

ce7fc16

wora reviewed Apr 16, 2025

View reviewed changes

maxgtr816 approved these changes Apr 30, 2025

View reviewed changes

	// NOTE: the referred field must be present at the top-level of the request
	// NOTE: the referred field must be present at the top-level of the request

feat(http): add support for mapping headers to request fields #1005

Are you sure you want to change the base?

feat(http): add support for mapping headers to request fields #1005

Uh oh!

Conversation

toumorokoshi commented Mar 29, 2025

Uh oh!

rofrankel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rofrankel Apr 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

toumorokoshi commented Apr 2, 2025

Uh oh!

noahdietz commented Apr 2, 2025

Uh oh!

toumorokoshi commented Apr 3, 2025

Uh oh!

noahdietz commented Apr 7, 2025

Uh oh!

toumorokoshi commented Apr 10, 2025

Uh oh!

toumorokoshi commented Apr 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

CEikermann commented Nov 18, 2025

Uh oh!

toumorokoshi commented Dec 2, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

rofrankel Apr 2, 2025 •

edited

Loading