-
Notifications
You must be signed in to change notification settings - Fork 5.2k
Description
Description
My project is published with NativeAOT and makes use of the ZStdSharp nuget package.
The application is now throwing AccessViolationExceptions when built with NativeAOT. It runs with no problems without NativeAOT on net10, and also runs fine with and without NativeAOT on net9 on both windows and ios.
I've included a reproducible program.cs for a console app.
I've raised an issue in the ZStdSharp repo too, though this feels like a runtime issue - oleg-st/ZstdSharp#58
Reproduction Steps
ZStdSharpRepro.csproj:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<PublishAot>true</PublishAot>
<InvariantGlobalization>true</InvariantGlobalization>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="ZstdSharp.Port" Version="0.8.6" />
</ItemGroup>
</Project>
Program.cs:
using System.Runtime.InteropServices;
using ZstdSharp.Unsafe;
var sourceBytes = new byte[1024*4];
var out1 = new byte[256];
var compressor = new ZStdCompressor(out1.Length);
var random = new Random(0);
random.NextBytes(sourceBytes);
var loopNo = 0;
while (++loopNo <= 1000)
{
var length1 = random.Next(out1.Length);
var start1 = random.Next(sourceBytes.Length - length1);
var toCompress1 = sourceBytes.AsSpan(start1, length1);
var compressedBytes = compressor.Compress(toCompress1, out1);
Console.WriteLine($"{loopNo:D4}: {length1}b => {(compressedBytes == 0 ? length1 : compressedBytes)}b");
}
public unsafe class ZStdCompressor
{
private readonly ZSTD_CCtx_s* _context;
private readonly nint _scratchPad;
private const int CCompressionLevel = 19;
private const int ScratchPadSize = 8 * 1024;
private int _scratchPadPos = 0;
private readonly int _scratchPadSafeLimit;
public ZStdCompressor(int maxPacketSize = 256)
{
_context = Methods.ZSTD_createCCtx();
_scratchPad = Marshal.AllocHGlobal(ScratchPadSize);
Console.WriteLine($"ContextPtr: {(ulong)_context}, ScratchPadPtr: {(ulong)_scratchPad}");
Window.Clear();
_scratchPadSafeLimit = ScratchPadSize - maxPacketSize;
Methods.ZSTD_compressBegin(_context, CCompressionLevel);
}
public void Reset()
{
Window.Clear();
_scratchPadPos = 0;
Methods.ZSTD_CCtx_reset(_context, ZSTD_ResetDirective.ZSTD_reset_session_only);
Methods.ZSTD_compressBegin(_context, CCompressionLevel);
}
public Span<byte> Window => new((void*)_scratchPad, ScratchPadSize);
public int Compress(ReadOnlySpan<byte> input, Span<byte> output)
{
var rbBuffer = new Span<byte>((void*)(_scratchPad + _scratchPadPos), input.Length);
_scratchPadPos += input.Length;
if (_scratchPadPos >= _scratchPadSafeLimit) _scratchPadPos = 0;
input.CopyTo(rbBuffer);
fixed (void* dstPtr = output)
fixed (void* srcPtr = rbBuffer)
{
var numBytes = (int)Methods.ZSTD_compressBlock(_context, dstPtr, (nuint)output.Length, srcPtr, (nuint)input.Length);
if (numBytes == 0)
{
rbBuffer.CopyTo(output);
}
return numBytes;
}
}
public void Dispose()
{
Methods.ZSTD_freeCCtx(_context);
Marshal.FreeHGlobal(_scratchPad);
}
}
Expected behavior
ContextPtr: 2372961007840, ScratchPadPtr: 2372961013136
0001: 128b => 128b
0002: 3b => 3b
0003: 174b => 174b
0004: 140b => 19b
0005: 181b => 181b
0006: 62b => 62b
----------------------
0994: 158b => 74b
0995: 77b => 9b
0996: 96b => 9b
0997: 35b => 8b
0998: 12b => 8b
0999: 169b => 9b
1000: 179b => 15b
Actual behavior
ContextPtr: 1369886658560, ScratchPadPtr: 1369886663856
0001: 128b => 128b
0002: 3b => 3b
0003: 174b => 174b
Process terminated. Access Violation: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. The application will be terminated since this platform does not support throwing an AccessViolationException.
at System.RuntimeExceptionHelpers.FailFast(String, Exception, String, RhFailFastReason, IntPtr, IntPtr) + 0x137
at System.RuntimeExceptionHelpers.GetRuntimeException(ExceptionIDs) + 0x165
at System.Runtime.EH.GetClasslibException(ExceptionIDs, IntPtr) + 0x34
at ZstdSharp.Unsafe.Methods.ZSTD_litLengthPrice(UInt32, optState_t*, Int32) + 0x35
at ZstdSharp.Unsafe.Methods.ZSTD_compressBlock_opt_generic(ZSTD_MatchState_t*, SeqStore_t*, UInt32*, Void*, UIntPtr, Int32, ZSTD_dictMode_e) + 0x30f
at ZstdSharp.Unsafe.Methods.ZSTD_compressBlock_btultra2(ZSTD_MatchState_t*, SeqStore_t*, UInt32*, Void*, UIntPtr) + 0x81
at ZstdSharp.Unsafe.Methods.ZSTD_buildSeqStore(ZSTD_CCtx_s*, Void*, UIntPtr) + 0x1c5
at ZstdSharp.Unsafe.Methods.ZSTD_compressBlock_internal(ZSTD_CCtx_s*, Void*, UIntPtr, Void*, UIntPtr, UInt32) + 0x32
at ZstdSharp.Unsafe.Methods.ZSTD_compressContinue_internal(ZSTD_CCtx_s*, Void*, UIntPtr, Void*, UIntPtr, UInt32, UInt32) + 0x213
at ZstdSharp.Unsafe.Methods.ZSTD_compressBlock_deprecated(ZSTD_CCtx_s*, Void*, UIntPtr, Void*, UIntPtr) + 0x4b
at ZStdCompressor.Compress(ReadOnlySpan`1, Span`1) + 0x91
at Program.<Main>$(String[] args) + 0x2d2
Regression?
This works in NativeAOT on net9, also on regular (non NativeAOT) builds with net10.
The crash is 100% repeatable on both win-x64 and ios-arm64 (I've not tried other runtimes).
Known Workarounds
No response
Configuration
net10-rc1 - NativeAOT - win-x64 and ios-arm64
Windows 11: 24H2
Ios: 18.6.2
Only seen in NativeAOT builds.
Other information
No response
Metadata
Metadata
Assignees
Labels
Type
Projects
Status