Thanks to visit codestin.com
Credit goes to github.com

Skip to content

AVX512 masking supportΒ #87097

@tannergooding

Description

@tannergooding

Summary

While implementing the API surface for Expose VectorMask to support generic masking for Vector, various considerations were found that necessitated taking a step back and reconsidering how it works.

Most of these issues were found foremost in the additional complexity and throughput hit that was going to be required for the JIT to integrate the type. However, it also impacted the way users interacted with the types and the public API surface we were to expose. Namely that existing user code would not benefit and it would nearly double the API surface we're currently exposing for the XArch and cross-platform intrinsics.

These considerations were raised with @dotnet/avx512-contrib and an alternative design was proposed where the JIT would do pattern recognition in lowering instead to limit the throughput hit and provide light-up to existing user code. This does not preclude the ability to expose VectorMask in the future and we can revisit the type and its design as appropriate.

Conceptual Differences

Previously, we would have defined the following and this would have expanded to effectively all existing intrinsics exposed. This would nearly double or triple our API surface taking us from the ~1900 APIs we have today up to at least ~3800 APIs. Arm64, as a corallary example, currently has ~2100 APIs.

namespace System.Runtime.Intrinsics.X86;

public static partial class Avx512F
{
    // Existing API
    public static Vector512<float> Add(Vector512<float> left, Vector512<float> right);

    // New mask API
    public static Vector512<float> Add(Vector512<float> mergeValues, Vector512Mask<float> mergeMask, Vector512<float> left, Vector512<float> right);

    // Potentially handled by just the above overload where `mergeValues: Vector512<float>.Zero`
    public static Vector512<float> Add(Vector512Mask<float> zeroMask, Vector512<float> left, Vector512<float> right);

    public static partial class VL
    {
        // New mask API
        public static Vector512<float> Add(Vector128<float> mergeValues, Vector128Mask<float> mergeMask, Vector128<float> left, Vector128<float> right);
        public static Vector512<float> Add(Vector256<float> mergeValues, Vector256Mask<float> mergeMask, Vector256<float> left, Vector256<float> right);

        // Potentially handled by just the above overload where `mergeValues: Vector512<float>.Zero`
        public static Vector512<float> Add(Vector128Mask<float> zeroMask, Vector128<float> left, Vector128<float> right);
        public static Vector512<float> Add(Vector256Mask<float> zeroMask, Vector256<float> left, Vector256<float> right);
    }
}

Pattern Recognition

Rather than exposing these overloads of APIs that take VectorMask<T> and allowing users to explicitly utilize masking, we will instead recognize a few key patterns and transform those in the JIT instead.

We would of also had some intrinsics such as public static Vector512Mask<float> CompareEqual(Vector512<float> left, Vector512<float> right) which produce a mask and various other ways to produce a mask as well. Developers then would've been able to consume this by passing down the mask to the API. For example, in the following we find all additions involving NaN and ensure those elements become 0 in the result.

Vector512Mask<float> nanMask = Avx512F.CompareNotEqual(left, left) | Avx512F.CompareNotEqual(right, right);
return Avx512F.Add(Vector512<float>.Zero, ~nanMask, left, right);

If a user wanted to do that today where masking doesn't exist, they'd actually do a functionally similar thing:

Vector256<float> nanMask = Avx.CompareNotEqual(left, left) | Avx.CompareNotEqual(right, right);
Vector256<float> result = Avx.Add(left, right);
return Vector256.ConditionalSelect(~nanMask, result, Vector256<float>.Zero);

Thus, by instead recognizing these patterns we can light up existing code and avoid exploding the API surface while also ensuring that the code users aim to write is consistent regardless of whether they are on hardware with native hardware masking or not.

A sampling of the set of patterns we want to recognize include, but are not limited to:

  • {k1} - ConditionalSelect(mask1, resultVector, mergeVector)
  • {k1}{z} - ConditionalSelect(mask1, resultVector, Vector.Zero)
  • kadd k1, k2 - mask1.ExtractMostSignificantBits() + mask2.ExtractMostSignificantBits()
  • kand k1, k2 - mask1 & mask2
  • kandn k1, k2 - ~mask1 & mask2
  • kmov k1, k2 - mask1 = mask2
  • kmov r32, k1 - mask1.ExtractMostSignificantBits()
  • kmov k1, r32 - Vector.Create(...).ExtractMostSignificantBits()
  • knot k1, k2 - ~mask1
  • kor k1, k2 - mask1 | mask2
  • kortest k1, k2; jz - (mask1 | mask2) == Vector.Zero
  • kortest k1, k2; jnz - (mask1 | mask2) != Vector.Zero
  • kortest k1, k2; jc - (mask1 | mask2) == Vector.AllBitsSet
  • kortest k1, k2; jnc - (mask1 | mask2) != Vector.AllBitsSet
  • kshiftl k1, k2, imm8 - mask1.ExtractMostSignificantBits() << amount
  • kshiftr k1, k2, imm8 - mask1.ExtractMostSignificantBits() >> amount
  • ktest k1, k2; jz - (mask1 & mask2) == Vector.Zero
  • ktest k1, k2; jnz - (mask1 & mask2) != Vector.Zero
  • ktest k1, k2; jc - (~mask1 & mask2) == Vector.Zero
  • ktest k1, k2; jnc - (~mask1 & mask2) == Vector.Zero
  • kunpck k1, k2, k3 - UnpackLow(mask1, mask2)
  • kxnor k1, k2 - ~( mask1 ^ mask2)
  • kxor k1, k2 - (mask1 ^ mask2)
  • vpbroadcastm - Vector.Create(mask1)
  • vpmovm2* - mask1.ExtractMostSignificantBits()
  • vpmov*2m - vector1.ExtractMostSignificantBits()

API Proposal

namespace System.Runtime.Intrinsics.X86;

public enum IntComparisonMode : byte
{
    Equals = 0,
    LessThan = 1,
    LessThanOrEqual = 2,
    False = 3,

    NotEquals = 4,
    GreaterThanOrEqual = 5,
    GreaterThan = 6,
    True = 7,

    // Additional names for parity
    //
    // FloatComparisonMode has similar but they are necessary there since
    // `!(x > y)` is not the same as `(x <= y)` due to the existance of NaN
    //
    // The architecture manual formally uses NotLessThan and NotLessThanOrEqual

    NotGreaterThanOrEqual = 1,
    NotGreaterThan = 2,

    NotLessThan = 5,
    NotLessThanOrEqual = 6,
}

public static partial class Avx512F
{
    public static Vector512<double> BlendVariable(Vector512<double> left, Vector512<double> right, Vector512<double> mask);
    public static Vector512<int>    BlendVariable(Vector512<int>    left, Vector512<int>    right, Vector512<int>    mask);
    public static Vector512<long>   BlendVariable(Vector512<long>   left, Vector512<long>   right, Vector512<long>   mask);
    public static Vector512<float>  BlendVariable(Vector512<float>  left, Vector512<float>  right, Vector512<float>  mask);
    public static Vector512<uint>   BlendVariable(Vector512<uint>   left, Vector512<uint>   right, Vector512<uint>   mask);
    public static Vector512<ulong>  BlendVariable(Vector512<ulong>  left, Vector512<ulong>  right, Vector512<ulong>  mask);

    public static Vector512<double> Compare                     (Vector512<double> left, Vector512<double> right, [ConstantExpected(Max = FloatComparisonMode.UnorderedTrueSignaling)] FloatComparisonMode mode);
    public static Vector512<double> CompareEqual                (Vector512<double> left, Vector512<double> right);
    public static Vector512<double> CompareGreaterThan          (Vector512<double> left, Vector512<double> right);
    public static Vector512<double> CompareGreaterThanOrEqual   (Vector512<double> left, Vector512<double> right);
    public static Vector512<double> CompareLessThan             (Vector512<double> left, Vector512<double> right);
    public static Vector512<double> CompareLessThanOrEqual      (Vector512<double> left, Vector512<double> right);
    public static Vector512<double> CompareNotEqual             (Vector512<double> left, Vector512<double> right);
    public static Vector512<double> CompareNotGreaterThan       (Vector512<double> left, Vector512<double> right);
    public static Vector512<double> CompareNotGreaterThanOrEqual(Vector512<double> left, Vector512<double> right);
    public static Vector512<double> CompareNotLessThan          (Vector512<double> left, Vector512<double> right);
    public static Vector512<double> CompareNotLessThanOrEqual   (Vector512<double> left, Vector512<double> right);
    public static Vector512<double> CompareOrdered              (Vector512<double> left, Vector512<double> right);
    public static Vector512<double> CompareUnordered            (Vector512<double> left, Vector512<double> right);

    public static Vector512<float> Compare                     (Vector512<float> left, Vector512<float> right, [ConstantExpected(Max = FloatComparisonMode.UnorderedTrueSignaling)] FloatComparisonMode mode);
    public static Vector512<float> CompareEqual                (Vector512<float> left, Vector512<float> right);
    public static Vector512<float> CompareGreaterThan          (Vector512<float> left, Vector512<float> right);
    public static Vector512<float> CompareGreaterThanOrEqual   (Vector512<float> left, Vector512<float> right);
    public static Vector512<float> CompareLessThan             (Vector512<float> left, Vector512<float> right);
    public static Vector512<float> CompareLessThanOrEqual      (Vector512<float> left, Vector512<float> right);
    public static Vector512<float> CompareNotEqual             (Vector512<float> left, Vector512<float> right);
    public static Vector512<float> CompareNotGreaterThan       (Vector512<float> left, Vector512<float> right);
    public static Vector512<float> CompareNotGreaterThanOrEqual(Vector512<float> left, Vector512<float> right);
    public static Vector512<float> CompareNotLessThan          (Vector512<float> left, Vector512<float> right);
    public static Vector512<float> CompareNotLessThanOrEqual   (Vector512<float> left, Vector512<float> right);
    public static Vector512<float> CompareOrdered              (Vector512<float> left, Vector512<float> right);
    public static Vector512<float> CompareUnordered            (Vector512<float> left, Vector512<float> right);

    public static Vector512<int> Compare                  (Vector512<int> left, Vector512<int> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
    public static Vector512<int> CompareEqual             (Vector512<int> left, Vector512<int> right);
    public static Vector512<int> CompareGreaterThan       (Vector512<int> left, Vector512<int> right);
    public static Vector512<int> CompareGreaterThanOrEqual(Vector512<int> left, Vector512<int> right);
    public static Vector512<int> CompareLessThan          (Vector512<int> left, Vector512<int> right);
    public static Vector512<int> CompareLessThanOrEqual   (Vector512<int> left, Vector512<int> right);
    public static Vector512<int> CompareNotEqual          (Vector512<int> left, Vector512<int> right);

    public static Vector512<long> Compare                  (Vector512<long> left, Vector512<long> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
    public static Vector512<long> CompareEqual             (Vector512<long> left, Vector512<long> right);
    public static Vector512<long> CompareGreaterThan       (Vector512<long> left, Vector512<long> right);
    public static Vector512<long> CompareGreaterThanOrEqual(Vector512<long> left, Vector512<long> right);
    public static Vector512<long> CompareLessThan          (Vector512<long> left, Vector512<long> right);
    public static Vector512<long> CompareLessThanOrEqual   (Vector512<long> left, Vector512<long> right);
    public static Vector512<long> CompareNotEqual          (Vector512<long> left, Vector512<long> right);

    public static Vector512<uint> Compare                  (Vector512<uint> left, Vector512<uint> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
    public static Vector512<uint> CompareEqual             (Vector512<uint> left, Vector512<uint> right);
    public static Vector512<uint> CompareGreaterThan       (Vector512<uint> left, Vector512<uint> right);
    public static Vector512<uint> CompareGreaterThanOrEqual(Vector512<uint> left, Vector512<uint> right);
    public static Vector512<uint> CompareLessThan          (Vector512<uint> left, Vector512<uint> right);
    public static Vector512<uint> CompareLessThanOrEqual   (Vector512<uint> left, Vector512<uint> right);
    public static Vector512<uint> CompareNotEqual          (Vector512<uint> left, Vector512<uint> right);

    public static Vector512<ulong> Compare                  (Vector512<ulong> left, Vector512<ulong> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
    public static Vector512<ulong> CompareEqual             (Vector512<ulong> left, Vector512<ulong> right);
    public static Vector512<ulong> CompareGreaterThan       (Vector512<ulong> left, Vector512<ulong> right);
    public static Vector512<ulong> CompareGreaterThanOrEqual(Vector512<ulong> left, Vector512<ulong> right);
    public static Vector512<ulong> CompareLessThan          (Vector512<ulong> left, Vector512<ulong> right);
    public static Vector512<ulong> CompareLessThanOrEqual   (Vector512<ulong> left, Vector512<ulong> right);
    public static Vector512<ulong> CompareNotEqual          (Vector512<ulong> left, Vector512<ulong> right);

    public static Vector512<double> Compress(Vector512<double> value, Vector512<double> mask);
    public static Vector512<int>    Compress(Vector512<int>    value, Vector512<int>    mask);
    public static Vector512<long>   Compress(Vector512<long>   value, Vector512<long>   mask);
    public static Vector512<float>  Compress(Vector512<float>  value, Vector512<float>  mask);
    public static Vector512<uint>   Compress(Vector512<uint>   value, Vector512<uint>   mask);
    public static Vector512<ulong>  Compress(Vector512<ulong>  value, Vector512<ulong>  mask);

    public static Vector512<double> Expand(Vector512<double> value, Vector512<double> mask);
    public static Vector512<int>    Expand(Vector512<int>    value, Vector512<int>    mask);
    public static Vector512<long>   Expand(Vector512<long>   value, Vector512<long>   mask);
    public static Vector512<float>  Expand(Vector512<float>  value, Vector512<float>  mask);
    public static Vector512<uint>   Expand(Vector512<uint>   value, Vector512<uint>   mask);
    public static Vector512<ulong>  Expand(Vector512<ulong>  value, Vector512<ulong>  mask);

    public static unsafe Vector512<double> GatherMaskVector512(Vector512<double> source, double* baseAddress, Vector512<int> index, Vector512<double> mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<int>    GatherMaskVector512(Vector512<int>    source, int*    baseAddress, Vector512<int> index, Vector512<int>    mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<long>   GatherMaskVector512(Vector512<long>   source, long*   baseAddress, Vector512<int> index, Vector512<long>   mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<float>  GatherMaskVector512(Vector512<float>  source, float*  baseAddress, Vector512<int> index, Vector512<float>  mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<uint>   GatherMaskVector512(Vector512<uint>   source, uint*   baseAddress, Vector512<int> index, Vector512<uint>   mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<ulong>  GatherMaskVector512(Vector512<ulong>  source, ulong*  baseAddress, Vector512<int> index, Vector512<ulong>  mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);

    public static unsafe Vector512<double> GatherMaskVector512(Vector512<double> source, double* baseAddress, Vector512<long> index, Vector512<double> mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<int>    GatherMaskVector512(Vector512<int>    source, int*    baseAddress, Vector512<long> index, Vector512<int>    mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<long>   GatherMaskVector512(Vector512<long>   source, long*   baseAddress, Vector512<long> index, Vector512<long>   mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<uint>   GatherMaskVector512(Vector512<uint>   source, uint*   baseAddress, Vector512<long> index, Vector512<uint>   mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<float>  GatherMaskVector512(Vector512<float>  source, float*  baseAddress, Vector512<long> index, Vector512<float>  mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<ulong>  GatherMaskVector512(Vector512<ulong>  source, ulong*  baseAddress, Vector512<long> index, Vector512<ulong>  mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);

    public static unsafe Vector512<double> GatherVector512(double* baseAddress, Vector512<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<int>    GatherVector512(int*    baseAddress, Vector512<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<long>   GatherVector512(long*   baseAddress, Vector512<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<float>  GatherVector512(float*  baseAddress, Vector512<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<uint>   GatherVector512(uint*   baseAddress, Vector512<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<ulong>  GatherVector512(ulong*  baseAddress, Vector512<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);

    public static unsafe Vector512<double> GatherVector512(double* baseAddress, Vector512<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<int>    GatherVector512(int*    baseAddress, Vector512<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<long>   GatherVector512(long*   baseAddress, Vector512<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<float>  GatherVector512(float*  baseAddress, Vector512<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<uint>   GatherVector512(uint*   baseAddress, Vector512<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe Vector512<ulong>  GatherVector512(ulong*  baseAddress, Vector512<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);

    public static unsafe Vector512<double> MaskLoad(double* address, Vector512<double> mask);
    public static unsafe Vector512<int>    MaskLoad(int*    address, Vector512<int>    mask);
    public static unsafe Vector512<long>   MaskLoad(long*   address, Vector512<long>   mask);
    public static unsafe Vector512<float>  MaskLoad(float*  address, Vector512<float>  mask);
    public static unsafe Vector512<uint>   MaskLoad(uint*   address, Vector512<uint>   mask);
    public static unsafe Vector512<ulong>  MaskLoad(ulong*  address, Vector512<ulong>  mask);

    public static unsafe void MaskStore(double* address, Vector512<double> mask, Vector512<double> source);
    public static unsafe void MaskStore(int*    address, Vector512<int>    mask, Vector512<int>    source);
    public static unsafe void MaskStore(long*   address, Vector512<long>   mask, Vector512<long>   source);
    public static unsafe void MaskStore(float*  address, Vector512<float>  mask, Vector512<float>  source);
    public static unsafe void MaskStore(uint*   address, Vector512<uint>   mask, Vector512<uint>   source);
    public static unsafe void MaskStore(ulong*  address, Vector512<ulong>  mask, Vector512<ulong>  source);

    public static int MoveMask(Vector256<short>  value);
    public static int MoveMask(Vector256<ushort> value);
    public static int MoveMask(Vector512<int>    value);
    public static int MoveMask(Vector512<float>  value);
    public static int MoveMask(Vector512<uint>   value);

    public static unsafe void ScatterMaskVector512(Vector512<double> value, double* baseAddress, Vector512<int> index, Vector512<double> mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterMaskVector512(Vector512<int>    value, int*    baseAddress, Vector512<int> index, Vector512<int>    mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterMaskVector512(Vector512<long>   value, long*   baseAddress, Vector512<int> index, Vector512<long>   mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterMaskVector512(Vector512<float>  value, float*  baseAddress, Vector512<int> index, Vector512<float>  mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterMaskVector512(Vector512<uint>   value, uint*   baseAddress, Vector512<int> index, Vector512<uint>   mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterMaskVector512(Vector512<ulong>  value, ulong*  baseAddress, Vector512<int> index, Vector512<ulong>  mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);

    public static unsafe void ScatterMaskVector512(Vector512<double> value, double* baseAddress, Vector512<long> index, Vector512<double> mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterMaskVector512(Vector512<int>    value, int*    baseAddress, Vector512<long> index, Vector512<int>    mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterMaskVector512(Vector512<long>   value, long*   baseAddress, Vector512<long> index, Vector512<long>   mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterMaskVector512(Vector512<float>  value, uint*   baseAddress, Vector512<long> index, Vector512<uint>   mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterMaskVector512(Vector512<uint>   value, float*  baseAddress, Vector512<long> index, Vector512<float>  mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterMaskVector512(Vector512<ulong>  value, ulong*  baseAddress, Vector512<long> index, Vector512<ulong>  mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);

    public static unsafe void ScatterVector512(Vector512<double> value, double* baseAddress, Vector512<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterVector512(Vector512<int>    value, int*    baseAddress, Vector512<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterVector512(Vector512<long>   value, long*   baseAddress, Vector512<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterVector512(Vector512<float>  value, float*  baseAddress, Vector512<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterVector512(Vector512<uint>   value, uint*   baseAddress, Vector512<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterVector512(Vector512<ulong>  value, ulong*  baseAddress, Vector512<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);

    public static unsafe void ScatterVector512(Vector512<double> value, double* baseAddress, Vector512<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterVector512(Vector512<int>    value, int*    baseAddress, Vector512<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterVector512(Vector512<long>   value, long*   baseAddress, Vector512<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterVector512(Vector512<float>  value, float*  baseAddress, Vector512<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterVector512(Vector512<uint>   value, uint*   baseAddress, Vector512<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    public static unsafe void ScatterVector512(Vector512<ulong>  value, ulong*  baseAddress, Vector512<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);

    public static bool TestC(Vector512<double> left, Vector512<double> right);
    public static bool TestC(Vector512<int>    left, Vector512<int>    right);
    public static bool TestC(Vector512<long>   left, Vector512<long>   right);
    public static bool TestC(Vector512<float>  left, Vector512<float>  right);
    public static bool TestC(Vector512<uint>   left, Vector512<uint>   right);
    public static bool TestC(Vector512<ulong>  left, Vector512<ulong>  right);

    public static bool TestNotZAndNotC(Vector512<double> left, Vector512<double> right);
    public static bool TestNotZAndNotC(Vector512<int>    left, Vector512<int>    right);
    public static bool TestNotZAndNotC(Vector512<long>   left, Vector512<long>   right);
    public static bool TestNotZAndNotC(Vector512<float>  left, Vector512<float>  right);
    public static bool TestNotZAndNotC(Vector512<uint>   left, Vector512<uint>   right);
    public static bool TestNotZAndNotC(Vector512<ulong>  left, Vector512<ulong>  right);

    public static bool TestZ(Vector512<double> left, Vector512<double> right);
    public static bool TestZ(Vector512<int>    left, Vector512<int>    right);
    public static bool TestZ(Vector512<long>   left, Vector512<long>   right);
    public static bool TestZ(Vector512<float>  left, Vector512<float>  right);
    public static bool TestZ(Vector512<uint>   left, Vector512<uint>   right);
    public static bool TestZ(Vector512<ulong>  left, Vector512<ulong>  right);

    public static partial class VL
    {
        public static Vector128<int> Compare                  (Vector128<int> left, Vector128<int> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
        public static Vector128<int> CompareGreaterThanOrEqual(Vector128<int> left, Vector128<int> right);
        public static Vector128<int> CompareLessThan          (Vector128<int> left, Vector128<int> right);
        public static Vector128<int> CompareLessThanOrEqual   (Vector128<int> left, Vector128<int> right);
        public static Vector128<int> CompareNotEqual          (Vector128<int> left, Vector128<int> right);

        public static Vector256<int> Compare                  (Vector256<int> left, Vector256<int> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
        public static Vector256<int> CompareGreaterThanOrEqual(Vector256<int> left, Vector256<int> right);
        public static Vector256<int> CompareLessThan          (Vector256<int> left, Vector256<int> right);
        public static Vector256<int> CompareLessThanOrEqual   (Vector256<int> left, Vector256<int> right);
        public static Vector256<int> CompareNotEqual          (Vector256<int> left, Vector256<int> right);

        public static Vector128<long> Compare                  (Vector128<long> left, Vector128<long> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
        public static Vector128<long> CompareGreaterThanOrEqual(Vector128<long> left, Vector128<long> right);
        public static Vector128<long> CompareLessThan          (Vector128<long> left, Vector128<long> right);
        public static Vector128<long> CompareLessThanOrEqual   (Vector128<long> left, Vector128<long> right);
        public static Vector128<long> CompareNotEqual          (Vector128<long> left, Vector128<long> right);
        public static Vector256<long> Compare                  (Vector256<long> left, Vector256<long> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
        public static Vector256<long> CompareGreaterThanOrEqual(Vector256<long> left, Vector256<long> right);
        public static Vector256<long> CompareLessThan          (Vector256<long> left, Vector256<long> right);
        public static Vector256<long> CompareLessThanOrEqual   (Vector256<long> left, Vector256<long> right);
        public static Vector256<long> CompareNotEqual          (Vector256<long> left, Vector256<long> right);

        public static Vector128<uint> Compare                  (Vector128<uint> left, Vector128<uint> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
        public static Vector128<uint> CompareGreaterThan       (Vector128<uint> left, Vector128<uint> right);
        public static Vector128<uint> CompareGreaterThanOrEqual(Vector128<uint> left, Vector128<uint> right);
        public static Vector128<uint> CompareLessThan          (Vector128<uint> left, Vector128<uint> right);
        public static Vector128<uint> CompareLessThanOrEqual   (Vector128<uint> left, Vector128<uint> right);
        public static Vector128<uint> CompareNotEqual          (Vector128<uint> left, Vector128<uint> right);
        public static Vector256<uint> Compare                  (Vector256<uint> left, Vector256<uint> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
        public static Vector256<uint> CompareGreaterThan       (Vector256<uint> left, Vector256<uint> right);
        public static Vector256<uint> CompareGreaterThanOrEqual(Vector256<uint> left, Vector256<uint> right);
        public static Vector256<uint> CompareLessThan          (Vector256<uint> left, Vector256<uint> right);
        public static Vector256<uint> CompareLessThanOrEqual   (Vector256<uint> left, Vector256<uint> right);
        public static Vector256<uint> CompareNotEqual          (Vector256<uint> left, Vector256<uint> right);

        public static Vector128<ulong> Compare                  (Vector128<ulong> left, Vector128<ulong> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
        public static Vector128<ulong> CompareGreaterThan       (Vector128<ulong> left, Vector128<ulong> right);
        public static Vector128<ulong> CompareGreaterThanOrEqual(Vector128<ulong> left, Vector128<ulong> right);
        public static Vector128<ulong> CompareLessThan          (Vector128<ulong> left, Vector128<ulong> right);
        public static Vector128<ulong> CompareLessThanOrEqual   (Vector128<ulong> left, Vector128<ulong> right);
        public static Vector128<ulong> CompareNotEqual          (Vector128<ulong> left, Vector128<ulong> right);
        public static Vector256<ulong> Compare                  (Vector256<ulong> left, Vector256<ulong> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
        public static Vector256<ulong> CompareGreaterThan       (Vector256<ulong> left, Vector256<ulong> right);
        public static Vector256<ulong> CompareGreaterThanOrEqual(Vector256<ulong> left, Vector256<ulong> right);
        public static Vector256<ulong> CompareLessThan          (Vector256<ulong> left, Vector256<ulong> right);
        public static Vector256<ulong> CompareLessThanOrEqual   (Vector256<ulong> left, Vector256<ulong> right);
        public static Vector256<ulong> CompareNotEqual          (Vector256<ulong> left, Vector256<ulong> right);

        public static Vector128<double> Compress(Vector128<double> value, Vector128<double> mask);
        public static Vector128<int>    Compress(Vector128<int>    value, Vector128<int>    mask);
        public static Vector128<long>   Compress(Vector128<long>   value, Vector128<long>   mask);
        public static Vector128<float>  Compress(Vector128<float>  value, Vector128<float>  mask);
        public static Vector128<uint>   Compress(Vector128<uint>   value, Vector128<uint>   mask);
        public static Vector128<ulong>  Compress(Vector128<ulong>  value, Vector128<ulong>  mask);
        public static Vector256<double> Compress(Vector256<double> value, Vector256<double> mask);
        public static Vector256<int>    Compress(Vector256<int>    value, Vector256<int>    mask);
        public static Vector256<long>   Compress(Vector256<long>   value, Vector256<long>   mask);
        public static Vector256<float>  Compress(Vector256<float>  value, Vector256<float>  mask);
        public static Vector256<uint>   Compress(Vector256<uint>   value, Vector256<uint>   mask);
        public static Vector256<ulong>  Compress(Vector256<ulong>  value, Vector256<ulong>  mask);

        public static Vector128<double> Expand(Vector128<double> value, Vector128<double> mask);
        public static Vector128<int>    Expand(Vector128<int>    value, Vector128<int>    mask);
        public static Vector128<long>   Expand(Vector128<long>   value, Vector128<long>   mask);
        public static Vector128<float>  Expand(Vector128<float>  value, Vector128<float>  mask);
        public static Vector128<uint>   Expand(Vector128<uint>   value, Vector128<uint>   mask);
        public static Vector128<ulong>  Expand(Vector128<ulong>  value, Vector128<ulong>  mask);
        public static Vector256<double> Expand(Vector256<double> value, Vector256<double> mask);
        public static Vector256<int>    Expand(Vector256<int>    value, Vector256<int>    mask);
        public static Vector256<long>   Expand(Vector256<long>   value, Vector256<long>   mask);
        public static Vector256<float>  Expand(Vector256<float>  value, Vector256<float>  mask);
        public static Vector256<uint>   Expand(Vector256<uint>   value, Vector256<uint>   mask);
        public static Vector256<ulong>  Expand(Vector256<ulong>  value, Vector256<ulong>  mask);

        public static unsafe void ScatterMaskVector128(Vector128<double> value, double* baseAddress, Vector128<int> index, Vector128<double> mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector128(Vector128<int>    value, int*    baseAddress, Vector128<int> index, Vector128<int>    mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector128(Vector128<long>   value, long*   baseAddress, Vector128<int> index, Vector128<long>   mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector128(Vector128<float>  value, float*  baseAddress, Vector128<int> index, Vector128<float>  mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector128(Vector128<uint>   value, uint*   baseAddress, Vector128<int> index, Vector128<uint>   mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector128(Vector128<ulong>  value, ulong*  baseAddress, Vector128<int> index, Vector128<ulong>  mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector256(Vector256<double> value, double* baseAddress, Vector256<int> index, Vector256<double> mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector256(Vector256<int>    value, int*    baseAddress, Vector256<int> index, Vector256<int>    mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector256(Vector256<long>   value, long*   baseAddress, Vector256<int> index, Vector256<long>   mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector256(Vector256<float>  value, float*  baseAddress, Vector256<int> index, Vector256<float>  mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector256(Vector256<uint>   value, uint*   baseAddress, Vector256<int> index, Vector256<uint>   mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector256(Vector256<ulong>  value, ulong*  baseAddress, Vector256<int> index, Vector256<ulong>  mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);

        public static unsafe void ScatterMaskVector128(Vector128<double> value, double* baseAddress, Vector128<long> index, Vector128<double> mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector128(Vector128<int>    value, int*    baseAddress, Vector128<long> index, Vector128<int>    mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector128(Vector128<long>   value, long*   baseAddress, Vector128<long> index, Vector128<long>   mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector128(Vector128<float>  value, uint*   baseAddress, Vector128<long> index, Vector128<uint>   mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector128(Vector128<uint>   value, float*  baseAddress, Vector128<long> index, Vector128<float>  mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector128(Vector128<ulong>  value, ulong*  baseAddress, Vector128<long> index, Vector128<ulong>  mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector256(Vector256<double> value, double* baseAddress, Vector256<long> index, Vector256<double> mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector256(Vector256<int>    value, int*    baseAddress, Vector256<long> index, Vector256<int>    mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector256(Vector256<long>   value, long*   baseAddress, Vector256<long> index, Vector256<long>   mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector256(Vector256<float>  value, uint*   baseAddress, Vector256<long> index, Vector256<uint>   mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector256(Vector256<uint>   value, float*  baseAddress, Vector256<long> index, Vector256<float>  mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterMaskVector256(Vector256<ulong>  value, ulong*  baseAddress, Vector256<long> index, Vector256<ulong>  mask, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);

        public static unsafe void ScatterVector128(Vector128<double> value, double* baseAddress, Vector128<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector128(Vector128<int>    value, int*    baseAddress, Vector128<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector128(Vector128<long>   value, long*   baseAddress, Vector128<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector128(Vector128<float>  value, float*  baseAddress, Vector128<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector128(Vector128<uint>   value, uint*   baseAddress, Vector128<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector128(Vector128<ulong>  value, ulong*  baseAddress, Vector128<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector256(Vector256<double> value, double* baseAddress, Vector256<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector256(Vector256<int>    value, int*    baseAddress, Vector256<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector256(Vector256<long>   value, long*   baseAddress, Vector256<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector256(Vector256<float>  value, float*  baseAddress, Vector256<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector256(Vector256<uint>   value, uint*   baseAddress, Vector256<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector256(Vector256<ulong>  value, ulong*  baseAddress, Vector256<int> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);

        public static unsafe void ScatterVector128(Vector128<double> value, double* baseAddress, Vector128<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector128(Vector128<int>    value, int*    baseAddress, Vector128<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector128(Vector128<long>   value, long*   baseAddress, Vector128<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector128(Vector128<float>  value, float*  baseAddress, Vector128<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector128(Vector128<uint>   value, uint*   baseAddress, Vector128<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector128(Vector128<ulong>  value, ulong*  baseAddress, Vector128<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector256(Vector256<double> value, double* baseAddress, Vector256<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector256(Vector256<int>    value, int*    baseAddress, Vector256<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector256(Vector256<long>   value, long*   baseAddress, Vector256<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector256(Vector256<float>  value, float*  baseAddress, Vector256<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector256(Vector256<uint>   value, uint*   baseAddress, Vector256<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
        public static unsafe void ScatterVector256(Vector256<ulong>  value, ulong*  baseAddress, Vector256<long> index, [ConstantExpected(Min = (byte)(1), Max = (byte)(8))] byte scale);
    }
}

public static partial class Avx512BW
{
    public static Vector512<byte>   BlendVariable(Vector512<byte>   left, Vector512<byte>   right, Vector512<byte>   mask);
    public static Vector512<short>  BlendVariable(Vector512<short>  left, Vector512<short>  right, Vector512<short>  mask);
    public static Vector512<sbyte>  BlendVariable(Vector512<sbyte>  left, Vector512<sbyte>  right, Vector512<sbyte>  mask);
    public static Vector512<ushort> BlendVariable(Vector512<ushort> left, Vector512<ushort> right, Vector512<ushort> mask);

    public static Vector512<byte> Compare                  (Vector512<byte> left, Vector512<byte> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
    public static Vector512<byte> CompareEqual             (Vector512<byte> left, Vector512<byte> right);
    public static Vector512<byte> CompareGreaterThan       (Vector512<byte> left, Vector512<byte> right);
    public static Vector512<byte> CompareGreaterThanOrEqual(Vector512<byte> left, Vector512<byte> right);
    public static Vector512<byte> CompareLessThan          (Vector512<byte> left, Vector512<byte> right);
    public static Vector512<byte> CompareLessThanOrEqual   (Vector512<byte> left, Vector512<byte> right);
    public static Vector512<byte> CompareNotEqual          (Vector512<byte> left, Vector512<byte> right);

    public static Vector512<short> Compare                  (Vector512<short> left, Vector512<short> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
    public static Vector512<short> CompareEqual             (Vector512<short> left, Vector512<short> right);
    public static Vector512<short> CompareGreaterThan       (Vector512<short> left, Vector512<short> right);
    public static Vector512<short> CompareGreaterThanOrEqual(Vector512<short> left, Vector512<short> right);
    public static Vector512<short> CompareLessThan          (Vector512<short> left, Vector512<short> right);
    public static Vector512<short> CompareLessThanOrEqual   (Vector512<short> left, Vector512<short> right);
    public static Vector512<short> CompareNotEqual          (Vector512<short> left, Vector512<short> right);

    public static Vector512<sbyte> Compare                  (Vector512<sbyte> left, Vector512<sbyte> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
    public static Vector512<sbyte> CompareEqual             (Vector512<sbyte> left, Vector512<sbyte> right);
    public static Vector512<sbyte> CompareGreaterThan       (Vector512<sbyte> left, Vector512<sbyte> right);
    public static Vector512<sbyte> CompareGreaterThanOrEqual(Vector512<sbyte> left, Vector512<sbyte> right);
    public static Vector512<sbyte> CompareLessThan          (Vector512<sbyte> left, Vector512<sbyte> right);
    public static Vector512<sbyte> CompareLessThanOrEqual   (Vector512<sbyte> left, Vector512<sbyte> right);
    public static Vector512<sbyte> CompareNotEqual          (Vector512<sbyte> left, Vector512<sbyte> right);

    public static Vector512<ushort> Compare                  (Vector512<ushort> left, Vector512<ushort> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
    public static Vector512<ushort> CompareEqual             (Vector512<ushort> left, Vector512<ushort> right);
    public static Vector512<ushort> CompareGreaterThan       (Vector512<ushort> left, Vector512<ushort> right);
    public static Vector512<ushort> CompareGreaterThanOrEqual(Vector512<ushort> left, Vector512<ushort> right);
    public static Vector512<ushort> CompareLessThan          (Vector512<ushort> left, Vector512<ushort> right);
    public static Vector512<ushort> CompareLessThanOrEqual   (Vector512<ushort> left, Vector512<ushort> right);
    public static Vector512<ushort> CompareNotEqual          (Vector512<ushort> left, Vector512<ushort> right);

    public static int MoveMask(Vector512<short>  value);
    public static int MoveMask(Vector512<ushort> value);

    public static long MoveMask(Vector512<byte>  value);
    public static long MoveMask(Vector512<sbyte> value);

    public static bool TestC(Vector512<byte>   left, Vector512<byte>   right);
    public static bool TestC(Vector512<short>  left, Vector512<short>  right);
    public static bool TestC(Vector512<sbyte>  left, Vector512<sbyte>  right);
    public static bool TestC(Vector512<ushort> left, Vector512<ushort> right);

    public static bool TestNotZAndNotC(Vector512<byte>   left, Vector512<byte>   right);
    public static bool TestNotZAndNotC(Vector512<short>  left, Vector512<short>  right);
    public static bool TestNotZAndNotC(Vector512<sbyte>  left, Vector512<sbyte>  right);
    public static bool TestNotZAndNotC(Vector512<ushort> left, Vector512<ushort> right);

    public static bool TestZ(Vector512<byte>   left, Vector512<byte>   right);
    public static bool TestZ(Vector512<short>  left, Vector512<short>  right);
    public static bool TestZ(Vector512<sbyte>  left, Vector512<sbyte>  right);
    public static bool TestZ(Vector512<ushort> left, Vector512<ushort> right);

    public static partial class VL
    {
        public static Vector128<byte> Compare                  (Vector128<byte> left, Vector128<byte> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
        public static Vector128<byte> CompareGreaterThan       (Vector128<byte> left, Vector128<byte> right);
        public static Vector128<byte> CompareGreaterThanOrEqual(Vector128<byte> left, Vector128<byte> right);
        public static Vector128<byte> CompareLessThan          (Vector128<byte> left, Vector128<byte> right);
        public static Vector128<byte> CompareLessThanOrEqual   (Vector128<byte> left, Vector128<byte> right);
        public static Vector128<byte> CompareNotEqual          (Vector128<byte> left, Vector128<byte> right);
        public static Vector256<byte> Compare                  (Vector256<byte> left, Vector256<byte> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
        public static Vector256<byte> CompareGreaterThan       (Vector256<byte> left, Vector256<byte> right);
        public static Vector256<byte> CompareGreaterThanOrEqual(Vector256<byte> left, Vector256<byte> right);
        public static Vector256<byte> CompareLessThan          (Vector256<byte> left, Vector256<byte> right);
        public static Vector256<byte> CompareLessThanOrEqual   (Vector256<byte> left, Vector256<byte> right);
        public static Vector256<byte> CompareNotEqual          (Vector256<byte> left, Vector256<byte> right);

        public static Vector128<short> Compare                  (Vector128<short> left, Vector128<short> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
        public static Vector128<short> CompareGreaterThanOrEqual(Vector128<short> left, Vector128<short> right);
        public static Vector128<short> CompareLessThan          (Vector128<short> left, Vector128<short> right);
        public static Vector128<short> CompareLessThanOrEqual   (Vector128<short> left, Vector128<short> right);
        public static Vector128<short> CompareNotEqual          (Vector128<short> left, Vector128<short> right);
        public static Vector256<short> Compare                  (Vector256<short> left, Vector256<short> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
        public static Vector256<short> CompareGreaterThanOrEqual(Vector256<short> left, Vector256<short> right);
        public static Vector256<short> CompareLessThan          (Vector256<short> left, Vector256<short> right);
        public static Vector256<short> CompareLessThanOrEqual   (Vector256<short> left, Vector256<short> right);
        public static Vector256<short> CompareNotEqual          (Vector256<short> left, Vector256<short> right);

        public static Vector128<sbyte> Compare                  (Vector128<sbyte> left, Vector128<sbyte> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
        public static Vector128<sbyte> CompareGreaterThanOrEqual(Vector128<sbyte> left, Vector128<sbyte> right);
        public static Vector128<sbyte> CompareLessThan          (Vector128<sbyte> left, Vector128<sbyte> right);
        public static Vector128<sbyte> CompareLessThanOrEqual   (Vector128<sbyte> left, Vector128<sbyte> right);
        public static Vector128<sbyte> CompareNotEqual          (Vector128<sbyte> left, Vector128<sbyte> right);
        public static Vector256<sbyte> Compare                  (Vector256<sbyte> left, Vector256<sbyte> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
        public static Vector256<sbyte> CompareGreaterThanOrEqual(Vector256<sbyte> left, Vector256<sbyte> right);
        public static Vector256<sbyte> CompareLessThan          (Vector256<sbyte> left, Vector256<sbyte> right);
        public static Vector256<sbyte> CompareLessThanOrEqual   (Vector256<sbyte> left, Vector256<sbyte> right);
        public static Vector256<sbyte> CompareNotEqual          (Vector256<sbyte> left, Vector256<sbyte> right);

        public static Vector128<ushort> Compare                  (Vector128<ushort> left, Vector128<ushort> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
        public static Vector128<ushort> CompareGreaterThan       (Vector128<ushort> left, Vector128<ushort> right);
        public static Vector128<ushort> CompareGreaterThanOrEqual(Vector128<ushort> left, Vector128<ushort> right);
        public static Vector128<ushort> CompareLessThan          (Vector128<ushort> left, Vector128<ushort> right);
        public static Vector128<ushort> CompareLessThanOrEqual   (Vector128<ushort> left, Vector128<ushort> right);
        public static Vector128<ushort> CompareNotEqual          (Vector128<ushort> left, Vector128<ushort> right);
        public static Vector256<ushort> Compare                  (Vector256<ushort> left, Vector256<ushort> right, [ConstantExpected(Max = IntComparisonMode.True)] IntComparisonMode mode);
        public static Vector256<ushort> CompareGreaterThan       (Vector256<ushort> left, Vector256<ushort> right);
        public static Vector256<ushort> CompareGreaterThanOrEqual(Vector256<ushort> left, Vector256<ushort> right);
        public static Vector256<ushort> CompareLessThan          (Vector256<ushort> left, Vector256<ushort> right);
        public static Vector256<ushort> CompareLessThanOrEqual   (Vector256<ushort> left, Vector256<ushort> right);
        public static Vector256<ushort> CompareNotEqual          (Vector256<ushort> left, Vector256<ushort> right);
    }
}

public static partial class Avx512DQ
{
    public static Vector512<double> Classify(Vector512<double> value, [ConstantExpected] byte control);
    public static Vector512<float>  Classify(Vector512<float>  value, [ConstantExpected] byte control);

    public static Vector128<double> ClassifyScalar(Vector128<double> value, [ConstantExpected] byte control);
    public static Vector128<float>  ClassifyScalar(Vector128<float>  value, [ConstantExpected] byte control);

    public static int MoveMask(Vector128<short>  value);
    public static int MoveMask(Vector128<ushort> value);
    public static int MoveMask(Vector256<int>    value);
    public static int MoveMask(Vector256<uint>   value);
    public static int MoveMask(Vector512<double> value);
    public static int MoveMask(Vector512<long>   value);
    public static int MoveMask(Vector512<ulong>  value);

    public static partial class VL
    {
        public static Vector128<double> Classify(Vector128<double> value, [ConstantExpected] byte control);
        public static Vector128<float>  Classify(Vector128<float>  value, [ConstantExpected] byte control);
        public static Vector256<double> Classify(Vector256<double> value, [ConstantExpected] byte control);
        public static Vector256<float>  Classify(Vector256<float>  value, [ConstantExpected] byte control);
    }
}

public abstract class Avx512Vbmi2 : Avx512BW
{
    public static new bool IsSupported { get; }

    public static Vector512<byte>   Compress(Vector512<byte>   value, Vector512<byte>   mask);
    public static Vector512<short>  Compress(Vector512<short>  value, Vector512<short>  mask);
    public static Vector512<sbyte>  Compress(Vector512<sbyte>  value, Vector512<sbyte>  mask);
    public static Vector512<ushort> Compress(Vector512<ushort> value, Vector512<ushort> mask);

    public static Vector512<byte>   Expand(Vector512<byte>   value, Vector512<byte>   mask);
    public static Vector512<short>  Expand(Vector512<short>  value, Vector512<short>  mask);
    public static Vector512<sbyte>  Expand(Vector512<sbyte>  value, Vector512<sbyte>  mask);
    public static Vector512<ushort> Expand(Vector512<ushort> value, Vector512<ushort> mask);

    public abstract class VL : Avx512BW.VL
    {
        public static new bool IsSupported { get; }

        public static Vector128<byte>   Compress(Vector128<byte>   value, Vector128<byte>   mask);
        public static Vector128<short>  Compress(Vector128<short>  value, Vector128<short>  mask);
        public static Vector128<sbyte>  Compress(Vector128<sbyte>  value, Vector128<sbyte>  mask);
        public static Vector128<ushort> Compress(Vector128<ushort> value, Vector128<ushort> mask);
        public static Vector256<byte>   Compress(Vector256<byte>   value, Vector256<byte>   mask);
        public static Vector256<short>  Compress(Vector256<short>  value, Vector256<short>  mask);
        public static Vector256<sbyte>  Compress(Vector256<sbyte>  value, Vector256<sbyte>  mask);
        public static Vector256<ushort> Compress(Vector256<ushort> value, Vector256<ushort> mask);

        public static Vector128<byte>   Expand(Vector128<byte>   value, Vector128<byte>   mask);
        public static Vector128<short>  Expand(Vector128<short>  value, Vector128<short>  mask);
        public static Vector128<sbyte>  Expand(Vector128<sbyte>  value, Vector128<sbyte>  mask);
        public static Vector128<ushort> Expand(Vector128<ushort> value, Vector128<ushort> mask);
        public static Vector256<byte>   Expand(Vector256<byte>   value, Vector256<byte>   mask);
        public static Vector256<short>  Expand(Vector256<short>  value, Vector256<short>  mask);
        public static Vector256<sbyte>  Expand(Vector256<sbyte>  value, Vector256<sbyte>  mask);
        public static Vector256<ushort> Expand(Vector256<ushort> value, Vector256<ushort> mask);
    }

    public abstract class X64 : Avx512BW.X64
    {
        public static new bool IsSupported { get; }
    }
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions