This is more of a tracking issue right now than a proper API proposal.

In TensorPrimitives, we now have ~40 routines, most of which are vectorized across Vector128, Vector256, Vector512, plus a scalar fallback. Rather than duplicate all of the boilerplate across them, we employ several helper functions that are parameterized via generic method parameters constrained to specific interfaces, with a caller supplying the kernel of the processing via a struct that implements the relevant interfaces. For example, TensorPrimitives.Abs (which computes the absolute value of each input element and stores the result into the corresponding destination slot) is implemented as:

        public static void Abs(ReadOnlySpan<float> x, Span<float> destination) =>
            InvokeSpanIntoSpan<AbsoluteOperator>(x, destination);

and TensorPrimitives.Add (which adds each pair of elements and stores the result for each into the corresponding destination slot) is implemented as:

        public static void Add(ReadOnlySpan<float> x, ReadOnlySpan<float> y, Span<float> destination) =>
            InvokeSpanSpanIntoSpan<AddOperator>(x, y, destination);

There are helpers for various shapes of these element-wise operations, as well as helpers for performing various forms of aggregations. For example, TensorPrimitives.ProductOfSums (which sums each element-wise pair from the two inputs and multiplies together all of those sums) is implemented as:

        public static float ProductOfSums(ReadOnlySpan<float> x, ReadOnlySpan<float> y)
        {
            ...
            return Aggregate<AddOperator, MultiplyOperator>(x, y);
        }

These "Operator"s each implement various interfaces for IUnaryOperator, IBinaryOperator, etc. For example, the AddMultipleOperator (which implements ITernaryOperator) is defined as:

        readonly struct AddMultiplyOperator : ITernaryOperator
        {
            public static float Invoke(float x, float y, float z) => (x + y) * z;
            public static Vector128<float> Invoke(Vector128<float> x, Vector128<float> y, Vector128<float> z) => (x + y) * z;
            public static Vector256<float> Invoke(Vector256<float> x, Vector256<float> y, Vector256<float> z) => (x + y) * z;
            public static Vector512<float> Invoke(Vector512<float> x, Vector512<float> y, Vector512<float> z) => (x + y) * z;
        }

We should consider exposing a set of such helpers and interfaces, to enable developers to express efficient vectorized implementations across various shapes without having to write all the boilerplate themselves. With an IVector as proposed in #76244 and as prototyped internally in #90764, and if we exposed a Scalar<T> that also implemented this interface, this could be consolidated down to just a single method and that would also be more future proof for vector types that might be added in the future, e.g.

        readonly struct AddMultiplyOperator<T> : ITernaryOperator<T>
        {
            public static TVector Invoke<TVector>(TVector x, TVector y, TVector z) where TVector : ISimdVector<TVector, T> =>
                (x + y) * z;
        }