[release/6.0] Manually backporting #74599 #75648
Conversation
* Manually backporting dotnet#74599 to 7.0 for RC2. * Fix a couple mis-copied lines of code and a couple nits.
|
@StephenMolloy please fill out the template, add the |
|
Why is this needed in release/6.0? We don't ship 6.0 for any big endian platforms, do we?
Is IBM building bits directly from dotnet/runtime's release/6.0 and that's why this is needed there? |
Perhaps I'm mis-interpreting this announcement... but it feels like if we "support" big-endian architectures, then our code should be correct on big-endian architectures. Especially if there exists an implementation. But that's a simple take. If we know that IBM builds and ships their support from a different branch and can patch their 6.0 implementation, then I suppose this isn't needed. This was just one of the issues we fixed in 7.0 that we reviewed and wanted to look at for backporting to 6.0 since it is a pretty serious data-loss issue - albeit only when communicating with big-endian machines. |
|
@uweigand, could you take a look at this PR and verify that it does the right thing on s390x for dotnet 6.0? |
|
/azp run runtime-community |
Thanks for the backport, and sorry for the delay! From looking over the code, the PR looks good to me. I tried to start the "runtime-community" CI pipeline, which includes a build on s390x, but apparently I don't have the required privileges in this repo. Maybe you can try to issue a |
|
Commenter does not have sufficient privileges for PR 75648 in repo dotnet/runtime |
|
/azp run runtime-community |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
Thanks @uweigand! I had not realized we had s390x builds available in the community pipeline. Good to know. Run kicked off now. :) |
|
@StephenMolloy how's this looking? Kind reminder that this still needs the |
| pi[0] = GetInt32(offset + 12); // flags | ||
| pi[1] = GetInt32(offset + 8); // hi32 | ||
| pi[2] = GetInt32(offset); // bottom-of-low64; | ||
| pi[3] = GetInt32(offset + 4); // top-of-low64; |
| pi[0] = GetInt32(offset + 12); // flags | ||
| pi[1] = GetInt32(offset + 8); // hi32 | ||
| pi[2] = GetInt32(offset); // bottom-of-low64; | ||
| pi[3] = GetInt32(offset + 4); // top-of-low64; |
There was a problem hiding this comment.
This looks wrong, which probably explains the failures in the decimal tests on s390x.
Upstream code uses the decimal-from-int-array constructor, which uses a different ordering compared to the in-memory layout. Is there any reason for not just backporting this from upstream as-is?
If you do want to use the "int *" unsafe method, the destination needs to match the definition of the decimal type:
private readonly int _flags;
private readonly uint _hi32;
private readonly ulong _lo64;
so we have flags, hi32, top-of-low64, bottom-of-low64 in that order on a BE host system.
On the other hand, the source also seems to be wrong: this has to match the LE in-memory layout of the decimal type, i.e. flags, hi32, bottom-of-low64, top-of-low64. Overall, it looks like this should be:
pi[0] = GetInt32(offset); // flags
pi[1] = GetInt32(offset + 4); // hi32
pi[2] = GetInt32(offset + 12); // top-of-low64;
pi[3] = GetInt32(offset + 8); // bottom-of-low64;
unless I'm missing something ...
|
/azp run runtime-community |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
@StephenMolloy what's the status on this change? Still needs the |
|
@StephenMolloy reminder that today's the last day for merging into 6.0 if you want this fix to go into November servicing. |
|
Any updates, @StephenMolloy? There is time for this to go into the December release. |
|
/azp run runtime-staging |
|
No pipelines are associated with this pull request. |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run runtime-community |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Backporting #75366 which was a backport of #74599 to 7.0 for RC2 from 8.0.
Customer Impact
DataContractSerializer has not been properly accounting for big-or-little endian-ness when writing a few types "to the wire." This has not previously been an issue, since to my knowledge there have not been any commonly used big-endian platforms with .Net. DCS writes little-endian on the wire, so not accounting for endian-ness has simply worked because little-endian platforms write little-endian on the wire as expected. IBM helped bring .Net to their big-endian s360 platform in 6.0 however, causing failures when serializing between different platforms.
Testing
Tests were added to check the correct little-endian output for affected types and arrays of types.
Regression
No... but kind of yes. Strictly speaking, the code has always missed this -endian handling for these types and was not regressed. But the introduction of big-endian implementations of dotnet in 6.0 has exposed this issue. So in a sense, DCS was fine in 5.0 and now is not correct on some platforms in 6.0 and later.
Risk
Low. The change is faithful to current execution for little-endian platforms and only does different things on a big-endian platform... which is currently broken without this fix.