xref: kubernetes/kubernetes#132312

For the pull-structured-merge-diff-test failure, please add this to fix the error @inteon

diff --git a/internal/cli/main_test.go b/internal/cli/main_test.go
index 3e409ede0673..8beed5c6cf55 100644
--- a/internal/cli/main_test.go
+++ b/internal/cli/main_test.go
@@ -21,6 +21,7 @@ import (
        "encoding/json"
        "io/ioutil"
        "path/filepath"
+       "strings"
        "testing"
 )

@@ -135,7 +136,7 @@ func (tt *testCase) checkOutput(t *testing.T, got []byte) {
                t.Fatalf("couldn't read expected output %q: %v", tt.expectedOutputPath, err)
        }

-       if a, e := string(got), string(want); a != e {
+       if a, e := strings.TrimSpace(string(got)), strings.TrimSpace(string(want)); a != e {
                t.Errorf("output didn't match expected output: got:\n%v\nwanted:\n%v\n", a, e)
        }
 }

For the pull-structured-merge-diff-test failure, please add this to fix the error @inteon
...

I fixed the test failure.

/assign @BenTheElder @liggitt

/assign @jpbetz
who is the primary apimachinery approver on this bit and was deeply involved in the initial performance-driven use of json-iterator in these bits

For the pull-structured-merge-diff-test failure, please add this to fix the error @inteon

I suspect using a json marshal function (like MarshalWrite) that doesn't append a newline would be a more efficient way to accomplish that

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: inteon
Once this PR has been reviewed and has the lgtm label, please ask for approval from jpbetz. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

The .../deserialize... benchmarks actually don't look terrible now... I'd be willing to accept that performance drop in pursuit of correctness / safety.

The serialize benchmarks still look pretty rough. Need to see what we can improve there.

did you run the full set of benchmarks to see how we looked across all of them?

Thanks for the updates, how are the overall benchmarks looking (not just the subset in the description)?

As you were adjusting the implementation, were there any unit tests it would make sense to add to catch edges the previous implementations handled we want to ensure the new one does as well? I'm thinking specifically of things like:

• handling of extra data in the input bytes/buffer when decoding/deserializing (e.g. "somevalue"extrastuff or {"key":"value"}extrastuff)
• handling of special characters in strings that need escaping where the raw bytes would not be the same as the decoded or encoded/escaped bytes
• handling of ignorable whitespace when decoding

First off- it's amazing to see this happening and the benchmarks are VERY promising. Thanks @inteon!

To get this to the finish line, and merge, what should our criteria be?

I chatted offline with @liggitt briefly and some of the criteria we discussed was:

• Golang releases a stable json/v2 (The alternative would be to add an internal copy of json-experiment to this repo like kube-openapi has, but I don't know if it's worth it given how close json/v2 is to stable)
• github.com/kubernetes/kubernetes CI test stability is not negatively impacted
• This passes a scale test (SIG instrumentation)
• We are confident on the correctness (triple check the implementation, shore up with additional functional tests)

Intuitively, it seems like the deserialization is already sufficiently fast. I suspect we need to optimize serialization a bit further since we serialize managed fields on all updates (not just patches). That said, I'm willing to be data driven here. If we can show downstream scale and performance is acceptable, I'm willing to accept a higher serialization perf regression in order to migrate to json/v2.

Thoughts, concerns?

Update check the new numbers in my PR description, upgrading encoding/json/v2 did improve performance!

Did some further tuning and got the # allocations lower than on master.

Golang releases a stable json/v2 (The alternative would be to add an internal copy of json-experiment to this repo like kube-openapi has, but I don't know if it's worth it given how close json/v2 is to stable

even with stable json/v2, we might need to temporarily use a fork, otherwise we kubernetes branches that aren't on that go version yet can't update SMD.

but we should encapsulate it and plan to eliminate it when we're ready to require that minimum go version

even kubernetes master isn't on 1.25 yet

Did some further tuning and got the # allocations lower than on master.

Am I reading correctly that B/op and allocs/op are ~equivalent or better than master on pretty much all benchmarks? If so, that's amazing progress!

Paired with a close review and functional/correctness test coverage to make sure the new approach behaves identically to the old version (especially in terms of what it accepts/rejects/produces in edge cases like leading/trailing/non-normalized/invalid inputs), this looks really promising.

Amazing work @inteon in reducing the number of allocs per operation to 1. I did a similar analysis over your change, and saw similar performance. The change should have zero to negligible impact on Kube API server performance. We just have to make sure this new library behaves the same as the existing implementation functionally, which I think existing tests should be able to do(?).

We just have to make sure this new library behaves the same as the existing implementation functionally, which I think existing tests should be able to do(?).

I'm not sure how detailed the existing tests are at all the edge cases of valid and invalid variants on input (handling of escaped values in keys, whitespace before/after/between tokens, valid and invalid syntax, etc), and byte-for-byte assertions about output. Since this needed to effectively rewrite some of the encoding/decoding paths, we need to make sure we have test coverage for those things.

We just have to make sure this new library behaves the same as the existing implementation functionally, which I think existing tests should be able to do(?).

I'm not sure how detailed the existing tests are at all the edge cases of valid and invalid variants on input (handling of escaped values in keys, whitespace before/after/between tokens, valid and invalid syntax, etc), and byte-for-byte assertions about output. Since this needed to effectively rewrite some of the encoding/decoding paths, we need to make sure we have test coverage for those things.

There are not enough tests for unicode and escape characters . I have added those tests in #300. Including these new tests, We should detect regression in Serialization and Deserialization code in future.

Excellent, #300 looks like a great step forward for test coverage of normalized encoding. We'll probably want similar additions for:

1. decoding of valid-but-non-normalized values working properly (insignificant leading / trailing / interspersed whitespace, or non-canonical escaped values, etc) and capturing what in-memory values are produced by the decoding
2. decoding of invalid values erroring properly

After upgrading github.com/go-json-experiment/json and rerunning the benchmarks, all benchmarks now outperform the benchmarks on master.

Huh… did something change on s-m-d master? The latest benchmark update looks like some of the relative improvement came from master getting worse...

oh, maybe the test changes in #300 impacted the master benchmark numbers

k/k master is at golang v1.25.1 fyi