Thanks to visit codestin.com
Credit goes to github.com

Skip to content

AccessViolationException from net10-rc1 NativeAOT builds only (win-x64 and ios-arm64 tested). #119571

@james-allison

Description

@james-allison

Description

My project is published with NativeAOT and makes use of the ZStdSharp nuget package.

The application is now throwing AccessViolationExceptions when built with NativeAOT. It runs with no problems without NativeAOT on net10, and also runs fine with and without NativeAOT on net9 on both windows and ios.

I've included a reproducible program.cs for a console app.

I've raised an issue in the ZStdSharp repo too, though this feels like a runtime issue - oleg-st/ZstdSharp#58

Reproduction Steps

ZStdSharpRepro.csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <PublishAot>true</PublishAot>
    <InvariantGlobalization>true</InvariantGlobalization>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="ZstdSharp.Port" Version="0.8.6" />
  </ItemGroup>

</Project>

Program.cs:

using System.Runtime.InteropServices;
using ZstdSharp.Unsafe;

var sourceBytes = new byte[1024*4];
var out1 = new byte[256];
var compressor = new ZStdCompressor(out1.Length);
var random = new Random(0);
random.NextBytes(sourceBytes);
var loopNo = 0;
while (++loopNo <= 1000)
{
    var length1 = random.Next(out1.Length);
    var start1 = random.Next(sourceBytes.Length - length1);
    var toCompress1 = sourceBytes.AsSpan(start1, length1);
    var compressedBytes = compressor.Compress(toCompress1, out1);
    Console.WriteLine($"{loopNo:D4}: {length1}b => {(compressedBytes == 0 ? length1 : compressedBytes)}b");
}

public unsafe class ZStdCompressor
{
    private readonly ZSTD_CCtx_s* _context;
    private readonly nint _scratchPad;
    private const int CCompressionLevel = 19;
    private const int ScratchPadSize = 8 * 1024;
    private int _scratchPadPos = 0;
    private readonly int _scratchPadSafeLimit;
    
    public ZStdCompressor(int maxPacketSize = 256)
    {
        _context = Methods.ZSTD_createCCtx();
        _scratchPad = Marshal.AllocHGlobal(ScratchPadSize);
        Console.WriteLine($"ContextPtr: {(ulong)_context}, ScratchPadPtr: {(ulong)_scratchPad}");
        Window.Clear();
        _scratchPadSafeLimit = ScratchPadSize - maxPacketSize;
        Methods.ZSTD_compressBegin(_context, CCompressionLevel);
    }

    public void Reset()
    {
        Window.Clear();
        _scratchPadPos = 0;
        Methods.ZSTD_CCtx_reset(_context, ZSTD_ResetDirective.ZSTD_reset_session_only);
        Methods.ZSTD_compressBegin(_context, CCompressionLevel);
    }

    public Span<byte> Window => new((void*)_scratchPad, ScratchPadSize);
    public int Compress(ReadOnlySpan<byte> input, Span<byte> output)
    {
        var rbBuffer = new Span<byte>((void*)(_scratchPad + _scratchPadPos), input.Length);
        _scratchPadPos += input.Length;
        if (_scratchPadPos >= _scratchPadSafeLimit) _scratchPadPos = 0;
        input.CopyTo(rbBuffer);
        fixed (void* dstPtr = output)
        fixed (void* srcPtr = rbBuffer)
        {
            var numBytes = (int)Methods.ZSTD_compressBlock(_context, dstPtr, (nuint)output.Length, srcPtr, (nuint)input.Length);
            if (numBytes == 0) 
            {
                rbBuffer.CopyTo(output);
            }

            return numBytes;
        }
    }

    public void Dispose()
    {
        Methods.ZSTD_freeCCtx(_context);
        Marshal.FreeHGlobal(_scratchPad);
    }
}

Expected behavior

ContextPtr: 2372961007840, ScratchPadPtr: 2372961013136
0001: 128b => 128b
0002: 3b => 3b
0003: 174b => 174b
0004: 140b => 19b
0005: 181b => 181b
0006: 62b => 62b
----------------------
0994: 158b => 74b
0995: 77b => 9b
0996: 96b => 9b
0997: 35b => 8b
0998: 12b => 8b
0999: 169b => 9b
1000: 179b => 15b

Actual behavior

ContextPtr: 1369886658560, ScratchPadPtr: 1369886663856
0001: 128b => 128b
0002: 3b => 3b
0003: 174b => 174b
Process terminated. Access Violation: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. The application will be terminated since this platform does not support throwing an AccessViolationException.
   at System.RuntimeExceptionHelpers.FailFast(String, Exception, String, RhFailFastReason, IntPtr, IntPtr) + 0x137
   at System.RuntimeExceptionHelpers.GetRuntimeException(ExceptionIDs) + 0x165
   at System.Runtime.EH.GetClasslibException(ExceptionIDs, IntPtr) + 0x34
   at ZstdSharp.Unsafe.Methods.ZSTD_litLengthPrice(UInt32, optState_t*, Int32) + 0x35
   at ZstdSharp.Unsafe.Methods.ZSTD_compressBlock_opt_generic(ZSTD_MatchState_t*, SeqStore_t*, UInt32*, Void*, UIntPtr, Int32, ZSTD_dictMode_e) + 0x30f
   at ZstdSharp.Unsafe.Methods.ZSTD_compressBlock_btultra2(ZSTD_MatchState_t*, SeqStore_t*, UInt32*, Void*, UIntPtr) + 0x81
   at ZstdSharp.Unsafe.Methods.ZSTD_buildSeqStore(ZSTD_CCtx_s*, Void*, UIntPtr) + 0x1c5
   at ZstdSharp.Unsafe.Methods.ZSTD_compressBlock_internal(ZSTD_CCtx_s*, Void*, UIntPtr, Void*, UIntPtr, UInt32) + 0x32
   at ZstdSharp.Unsafe.Methods.ZSTD_compressContinue_internal(ZSTD_CCtx_s*, Void*, UIntPtr, Void*, UIntPtr, UInt32, UInt32) + 0x213
   at ZstdSharp.Unsafe.Methods.ZSTD_compressBlock_deprecated(ZSTD_CCtx_s*, Void*, UIntPtr, Void*, UIntPtr) + 0x4b
   at ZStdCompressor.Compress(ReadOnlySpan`1, Span`1) + 0x91
   at Program.<Main>$(String[] args) + 0x2d2

Regression?

This works in NativeAOT on net9, also on regular (non NativeAOT) builds with net10.
The crash is 100% repeatable on both win-x64 and ios-arm64 (I've not tried other runtimes).

Known Workarounds

No response

Configuration

net10-rc1 - NativeAOT - win-x64 and ios-arm64

Windows 11: 24H2
Ios: 18.6.2

Only seen in NativeAOT builds.

Other information

No response

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

Status

No status

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions