Implemented encoding null in key and value of a map in Protobuf #2910

shanshin · 2025-01-23T13:37:30Z

Also improved null encoding error messages.

Resolves #2760

Also improved `null` encoding error messages. Resolves #2760

shanshin · 2025-01-23T13:40:00Z

The open question is whether we want to allow encoding null in the key.

This change may also make the encoded messages incompatible with some decoders created using protoc.

sandwwraith · 2025-01-23T15:48:38Z

The open question is whether we want to allow encoding null in the key.

How will it look like? A value without a key? In that case, we possibly can decode it automatically into a null: value map pair

sandwwraith · 2025-01-23T16:06:55Z

...ats/protobuf/commonMain/src/kotlinx/serialization/protobuf/internal/ProtobufTaggedEncoder.kt

                NullableMode.OPTIONAL -> "'null' is not supported for optional properties in ProtoBuf"
-                NullableMode.COLLECTION -> "'null' is not supported for collection types in ProtoBuf"
+                NullableMode.COLLECTION -> "'null' is not supported as the value of collection types in ProtoBuf"
+                NullableMode.LIST_ELEMENT -> "'null' is not supported as the value of a list element in ProtoBuf"


Is LIST_ELEMENT added for the purpose of proper error message only? I do not see this string being tested anywhere

Yes, only the accepted or not flag is sufficient to throw an error. The enumeration is created only for printong different messages

Then add a test for this message pls

Tests added

shanshin · 2025-01-23T16:25:55Z

How will it look like? A value without a key?

Each map declaration in protobuf structually is repeated field with Entry message

message NullableMap {
    map<string, int32> m = 1;
}

is the same as

message NullableMap {
    message m_Entry {
        optional string key = 1;
        optional int32 value = 2;
    }

    repeated m_Entry m = 1;
}

see the protobuf doc

Therefore, we can skip either the key or the value.

In that case, we possibly can decode it automatically into a null: value map pair

It's already implemented in this PR, see encodedKeyNull in testNullMap.
Question: should we leave this opportunity

sandwwraith · 2025-01-23T16:51:39Z

I think it is OK to have it if the message itself is not malformed

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [io.modelcontextprotocol:kotlin-sdk](https://github.com/modelcontextprotocol/kotlin-sdk) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `0.5.0` -> `0.6.0` | | [org.jetbrains.kotlinx:kotlinx-serialization-json](https://github.com/Kotlin/kotlinx.serialization) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.7.3` -> `1.9.0` | | [org.jetbrains.kotlinx:kotlinx-serialization-core](https://github.com/Kotlin/kotlinx.serialization) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.7.3` -> `1.9.0` | | [com.google.apis:google-api-services-storage](http://nexus.sonatype.org/oss-repository-hosting.html) ([source](http://svn.sonatype.org/spice/tags/oss-parent-7)) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `v1-rev20250718-2.0.0` -> `v1-rev20250814-2.0.0` | | [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.32.23` -> `2.32.24` | | [software.amazon.awssdk:sqs](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.32.23` -> `2.32.24` | | [software.amazon.awssdk:regions](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.32.23` -> `2.32.24` | | [software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.32.23` -> `2.32.24` | | [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.32.23` -> `2.32.24` | | [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.32.23` -> `2.32.24` | | [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.32.23` -> `2.32.24` | | [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.32.23` -> `2.32.24` | --- ### Release Notes <details> <summary>modelcontextprotocol/kotlin-sdk (io.modelcontextprotocol:kotlin-sdk)</summary> ### [`v0.6.0`](https://github.com/modelcontextprotocol/kotlin-sdk/releases/tag/0.6.0) [Compare Source](modelcontextprotocol/kotlin-sdk@0.5.0...0.6.0) #### What's Changed - Update jreleaser to fix publication issue by [@e5l](https://github.com/e5l) in modelcontextprotocol/kotlin-sdk#91 - Disable configuration cache to fix jreleaser issue by [@e5l](https://github.com/e5l) in modelcontextprotocol/kotlin-sdk#92 - feat: Add audio type according to 2025-03-26 spec by [@SeanChinJunKai](https://github.com/SeanChinJunKai) in modelcontextprotocol/kotlin-sdk#68 - fix(client): serialize inputSchema as input\_schema by [@shiqicao](https://github.com/shiqicao) in modelcontextprotocol/kotlin-sdk#97 - fix(client) add encodeDefault for field with non spec default value by [@shiqicao](https://github.com/shiqicao) in modelcontextprotocol/kotlin-sdk#99 - Make McpJson public to allow flexible protocol development by [@parnurzeal](https://github.com/parnurzeal) in modelcontextprotocol/kotlin-sdk#103 - fix: apply `requestBuilder` headers when sending rpc messages by [@dead8309](https://github.com/dead8309) in modelcontextprotocol/kotlin-sdk#96 - fix: Remove [@SerialName](https://github.com/SerialName) annotation for inputSchema by [@adamglin0](https://github.com/adamglin0) in modelcontextprotocol/kotlin-sdk#105 - add ios and wasm targets by [@devcrocod](https://github.com/devcrocod) in modelcontextprotocol/kotlin-sdk#81 - Add dependabot by [@StefMa](https://github.com/StefMa) in modelcontextprotocol/kotlin-sdk#121 - Bump org.jetbrains.kotlinx:kotlinx-serialization-json from 1.7.3 to 1.8.1 by [@dependabot](https://github.com/dependabot)\[bot] in modelcontextprotocol/kotlin-sdk#127 - Bump io.github.oshai:kotlin-logging from 7.0.0 to 7.0.7 by [@dependabot](https://github.com/dependabot)\[bot] in modelcontextprotocol/kotlin-sdk#126 - Bump org.jetbrains.kotlinx.binary-compatibility-validator from 0.17.0 to 0.18.0 by [@dependabot](https://github.com/dependabot)\[bot] in modelcontextprotocol/kotlin-sdk#125 - update kotlin to 2.2.0 and ktor to 3.1.3 by [@devcrocod](https://github.com/devcrocod) in modelcontextprotocol/kotlin-sdk#120 - Add client roots addition/removal API and listRoots handler by [@ptitjes](https://github.com/ptitjes) in modelcontextprotocol/kotlin-sdk#118 - Bump org.slf4j:slf4j-simple from 2.0.16 to 2.0.17 by [@dependabot](https://github.com/dependabot)\[bot] in modelcontextprotocol/kotlin-sdk#131 - Bump org.jetbrains.kotlinx:kotlinx-serialization-json from 1.8.1 to 1.9.0 by [@dependabot](https://github.com/dependabot)\[bot] in modelcontextprotocol/kotlin-sdk#132 - Bump io.mockk:mockk from 1.13.13 to 1.14.4 by [@dependabot](https://github.com/dependabot)\[bot] in modelcontextprotocol/kotlin-sdk#133 - Remove fixed suffix /sse by [@adamglin0](https://github.com/adamglin0) in modelcontextprotocol/kotlin-sdk#108 - sse server does not process endpoint correctly when sse path is not a directory by [@shendaxia-sm](https://github.com/shendaxia-sm) in modelcontextprotocol/kotlin-sdk#43 - Add labels to dependabot configuration for Kotlin and GitHub Actions by [@devcrocod](https://github.com/devcrocod) in modelcontextprotocol/kotlin-sdk#135 - feat: add tool annotations according to 2025-03-26 spec by [@SeanChinJunKai](https://github.com/SeanChinJunKai) in modelcontextprotocol/kotlin-sdk#71 - Bump ktor from 3.1.3 to 3.2.1 by [@dependabot](https://github.com/dependabot)\[bot] in modelcontextprotocol/kotlin-sdk#140 - Bump gradle/actions from 4.0.0 to 4.4.1 by [@dependabot](https://github.com/dependabot)\[bot] in modelcontextprotocol/kotlin-sdk#122 - Bump org.jreleaser from 1.17.0 to 1.19.0 by [@dependabot](https://github.com/dependabot)\[bot] in modelcontextprotocol/kotlin-sdk#141 - refactor `SseClientTransport` by [@devcrocod](https://github.com/devcrocod) in modelcontextprotocol/kotlin-sdk#142 - atomic and persistent collections for thread safety by [@devcrocod](https://github.com/devcrocod) in modelcontextprotocol/kotlin-sdk#143 - Bump org.gradle.toolchains.foojay-resolver-convention from 0.8.0 to 1.0.0 by [@dependabot](https://github.com/dependabot)\[bot] in modelcontextprotocol/kotlin-sdk#130 - Add support for elicitation by [@ptitjes](https://github.com/ptitjes) in modelcontextprotocol/kotlin-sdk#138 - Add support for tool structured content and output schema by [@ptitjes](https://github.com/ptitjes) in modelcontextprotocol/kotlin-sdk#146 - Add streamable http client transport by [@devcrocod](https://github.com/devcrocod) in modelcontextprotocol/kotlin-sdk#147 - Refactor JSON processing to exclude "method" in serialization by [@devcrocod](https://github.com/devcrocod) in [#157](modelcontextprotocol/kotlin-sdk#157) - Release 0.6.0 by [@devcrocod](https://github.com/devcrocod) in modelcontextprotocol/kotlin-sdk#149 #### New Contributors - [@shiqicao](https://github.com/shiqicao) made their first contribution in modelcontextprotocol/kotlin-sdk#97 - [@parnurzeal](https://github.com/parnurzeal) made their first contribution in modelcontextprotocol/kotlin-sdk#103 - [@dead8309](https://github.com/dead8309) made their first contribution in modelcontextprotocol/kotlin-sdk#96 - [@adamglin0](https://github.com/adamglin0) made their first contribution in modelcontextprotocol/kotlin-sdk#105 - [@StefMa](https://github.com/StefMa) made their first contribution in modelcontextprotocol/kotlin-sdk#121 - [@dependabot](https://github.com/dependabot)\[bot] made their first contribution in modelcontextprotocol/kotlin-sdk#127 - [@ptitjes](https://github.com/ptitjes) made their first contribution in modelcontextprotocol/kotlin-sdk#118 - [@shendaxia-sm](https://github.com/shendaxia-sm) made their first contribution in modelcontextprotocol/kotlin-sdk#43 **Full Changelog**: modelcontextprotocol/kotlin-sdk@0.5.0...0.6.0 </details> <details> <summary>Kotlin/kotlinx.serialization (org.jetbrains.kotlinx:kotlinx-serialization-json)</summary> ### [`v1.9.0`](https://github.com/Kotlin/kotlinx.serialization/blob/HEAD/CHANGELOG.md#190--2025-06-27) \================== This release updates Kotlin version to 2.2.0, includes several bugfixes and provides serializers for kotlin.time.Instant. #### Add kotlin.time.Instant serializers Instant class was moved from kotlinx-datetime library to Kotlin standard library. As a result, kotlinx-datetime 0.7.0 no longer has serializers for the Instant class. To use new kotlin.time.Instant class in your [@Serializable](https://github.com/Serializable) classes, you can use this 1.9.0 kotlinx-serialization version (Kotlin 2.2 is required). You can choose between default `InstantSerializer` which uses its string representation, or specify `InstantComponentSerializer` that represents instant as its components. See details in the [PR](Kotlin/kotlinx.serialization#2945). #### Other bugfixes - Fix resize in JsonPath ([#2995](Kotlin/kotlinx.serialization#2995)) - Fixed proguard rules for obfuscation to work correctly ([#2983](Kotlin/kotlinx.serialization#2983)) ### [`v1.8.1`](https://github.com/Kotlin/kotlinx.serialization/blob/HEAD/CHANGELOG.md#181--2025-03-31) \================== This release updates Kotlin version to 2.1.20, while also providing several important improvements and bugfixes. #### Improvements - Implemented encoding null in key and value of a map in Protobuf ([#2910](Kotlin/kotlinx.serialization#2910)) - Make type argument in JsonTransformingSerializer nullable ([#2911](Kotlin/kotlinx.serialization#2911)) - Use SPDX identifier in POMs ([#2936](Kotlin/kotlinx.serialization#2936)) (thanks to [Leon Linhart](https://github.com/TheMrMilchmann)) - Add watchosDeviceArm64 to Okio integration module ([#2920](Kotlin/kotlinx.serialization#2920)) (thanks to [Daniel Santiago](https://github.com/danysantiago)) - Update kotlinx-io version to 0.6.0 ([#2933](Kotlin/kotlinx.serialization#2933)) (thanks to [Piotr Krzemiński](https://github.com/krzema12)) #### Bugfixes - Fix incorrect enum coercion during deserialization from JsonElement ([#2962](Kotlin/kotlinx.serialization#2962)) - Supply proper equals(), hashCode(), and toString() for SerialDescriptor() wrapper ([#2942](Kotlin/kotlinx.serialization#2942)) - Do not encode empty packed collections in protobuf ([#2907](Kotlin/kotlinx.serialization#2907)) ### [`v1.8.0`](https://github.com/Kotlin/kotlinx.serialization/blob/HEAD/CHANGELOG.md#180--2025-01-06) \================== This release contains all of the changes from 1.8.0-RC. Kotlin 2.1.0 is used as a default, while upcoming 2.1.10 is also supported. Also added small bugfixes, including speedup of ProtoWireType.from ([#2879](Kotlin/kotlinx.serialization#2879)). </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Never, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: 40e2f07ecd958e92e64c571d1351e91f96e71ce7

Implemented encoding null in key and value of a map in Protobuf

283217c

Also improved `null` encoding error messages. Resolves #2760

shanshin requested a review from sandwwraith January 23, 2025 13:37

sandwwraith reviewed Jan 23, 2025

View reviewed changes

~ added tests on messages

cc78388

shanshin requested a review from sandwwraith January 23, 2025 17:50

sandwwraith approved these changes Jan 28, 2025

View reviewed changes

shanshin merged commit f2b0258 into dev Jan 28, 2025
4 checks passed

shanshin deleted the protobuf-null-in-map branch January 28, 2025 13:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Implemented encoding null in key and value of a map in Protobuf #2910

Implemented encoding null in key and value of a map in Protobuf #2910

Uh oh!

shanshin commented Jan 23, 2025

Uh oh!

shanshin commented Jan 23, 2025

Uh oh!

sandwwraith commented Jan 23, 2025

Uh oh!

sandwwraith Jan 23, 2025

Uh oh!

shanshin Jan 23, 2025

Uh oh!

sandwwraith Jan 23, 2025

Uh oh!

shanshin Jan 23, 2025

Uh oh!

shanshin commented Jan 23, 2025 •

edited

Loading

Uh oh!

sandwwraith commented Jan 23, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Implemented encoding null in key and value of a map in Protobuf #2910

Implemented encoding null in key and value of a map in Protobuf #2910

Uh oh!

Conversation

shanshin commented Jan 23, 2025

Uh oh!

shanshin commented Jan 23, 2025

Uh oh!

sandwwraith commented Jan 23, 2025

Uh oh!

sandwwraith Jan 23, 2025

Choose a reason for hiding this comment

Uh oh!

shanshin Jan 23, 2025

Choose a reason for hiding this comment

Uh oh!

sandwwraith Jan 23, 2025

Choose a reason for hiding this comment

Uh oh!

shanshin Jan 23, 2025

Choose a reason for hiding this comment

Uh oh!

shanshin commented Jan 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sandwwraith commented Jan 23, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

shanshin commented Jan 23, 2025 •

edited

Loading