Can you submit this as a pull request?

On Wed, Oct 26, 2016 at 9:48 AM, rwmjones [email protected] wrote:

Patch for RISC-V (64 bit only) support:

http://oirase.annexia.org/tmp/numpy-riscv64.patch

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#8213, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAlOaKBaNTsHz0wrqaPGkyHKeDhVBnjmks5q34RdgaJpZM4KhaEZ
.

Nathaniel J. Smith -- https://vorpus.org http://vorpus.org

@rwmjones ping? :-)

This patch as written won't work because the final version of GCC used different macros. You would need to use something like:

#if defined(__riscv) && __riscv_xlen == 64

Here's the latest patch, written by David Abdurachmanov:

http://oirase.annexia.org/tmp/numpy-add-riscv64.patch

Again, please submit this as a pull request.

Couldn't you just grab the patch and apply it? The github "workflow" for pull requests is incredibly tedious.

So it was compile tested, yes. You can find the logs for that here:

https://fedorapeople.org/groups/risc-v/logs/numpy-1.14.2-1.0.riscv64.fc28.src.rpm/build.log

We have built other packages using this, but we have not run the numpy test suite.

BTW my colleague suggested that you should just use __BYTE_ORDER from <endian.h>. It's unfortunately not fully portable, but it is available on at least glibc. Maybe it could be used as a fallback in case the architecture is not a known one?

If you like you can grab bootable disk images which can be run on x86_64: https://fedorapeople.org/groups/risc-v/disk-images/

Compiling numpy will take a very long time, probably days. No idea about running the test suite but that could equally take days.

Couldn't you just grab the patch and apply it? The github "workflow" for pull requests is incredibly tedious.

All changes to numpy go through pull requests, so you're asking random volunteers to do your "tedious" work for you. It's not usually a great way to make people sympathetic to your change...

If the patch has only been compile-tested, then I think it should not be applied.

I backported the patch to the Debian package, so on version 1.13.3. It built fine on riscv64 and the testsuite is almost fine too:

Python 2.7:

Ran 6601 tests in 1307.277s

FAILED (KNOWNFAIL=12, SKIP=24, failures=2)

Python 3.6:

Ran 6610 tests in 1493.899s

FAILED (KNOWNFAIL=13, SKIP=15, failures=2)

In both cases the failures are the same:

======================================================================
FAIL: test_float (test_numeric.TestBoolCmp)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/<<PKGBUILDDIR>>/debian/tmp/usr/lib/python2.7/dist-packages/numpy/core/tests/test_numeric.py", line 396, in test_float
    assert_array_equal(np.signbit(self.signf[i:]), self.ef[i:])
  File "/<<PKGBUILDDIR>>/debian/tmp/usr/lib/python2.7/dist-packages/numpy/testing/utils.py", line 854, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "/<<PKGBUILDDIR>>/debian/tmp/usr/lib/python2.7/dist-packages/numpy/testing/utils.py", line 778, in assert_array_compare
    raise AssertionError(msg)
AssertionError:
Arrays are not equal

(mismatch 4.296875%)
 x: array([False, False, False, False, False, False, False, False,  True,
       False, False, False, False, False, False, False, False,  True,
       False, False, False, False, False, False,  True,  True, False,...
 y: array([False, False, False, False, False, False, False, False,  True,
       False, False, False, False, False, False, False, False,  True,
       False, False, False, False, False, False,  True,  True, False,...

======================================================================
FAIL: test_scripts.test_f2py
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/<<PKGBUILDDIR>>/debian/tmp/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 147, in skipper_func
    return f(*args, **kwargs)
  File "/<<PKGBUILDDIR>>/debian/tmp/usr/lib/python2.7/dist-packages/numpy/tests/test_scripts.py", line 93, in test_f2py
    assert_(success, msg)
  File "/<<PKGBUILDDIR>>/debian/tmp/usr/lib/python2.7/dist-packages/numpy/testing/utils.py", line 92, in assert_
    raise AssertionError(smsg)
AssertionError: Warning: neither f2py nor f2py2 nor f2py2.7 found in path

----------------------------------------------------------------------

The first failure is specific to RISC-V and probably needs some investigation. The second failure is due to f2py missing during the build, and also happens when building the Debian package on other architectures.

I guess the per-arch flags are correct, otherwise I would have expected many more failures.

I investigated the test_float (test_numeric.TestBoolCmp) failure on RISC-V. It comes from the following part of the RISC-V specifications:

Except when otherwise stated, if the result of a floating-point operation is NaN, it is the canonical
NaN. The canonical NaN has a positive sign and all significand bits clear except the MSB, a.k.a.
the quiet bit. For single-precision floating-point, this corresponds to the pattern 0x7fc00000.

The sign-injection instructions (FSGNJ, FSGNJN, FSGNJX) do not canonicalize NaNs; they
manipulate the underlying bit patterns directly.

The code that doesn't behave on RISC-V is the following one:

        self.signf[3::6][self.ef[3::6]] = -np.nan
        self.signd[3::6][self.ed[3::6]] = -np.nan

Basically -np.nan is a float64 value, that is converted to float32 when assigned to the array. This caused it to be canonicalized and to loose the negative sign. That's also the reason why the test_double part doesn't fail as there is no float64 to float32 conversion.

One (ugly) way to make the test pass is to change the sign after converting the value to float32:

        signf[3::6][ef[3::6]] = -np.array(np.nan, dtype=np.float32)
        signd[3::6][ed[3::6]] = -np.array(np.nan, dtype=np.float32)

Huh.

Indeed, on page 48 of Volume I: RISC-V User-Level ISA V2.2 it says:

We considered propagating NaN payloads, as is recommended by [IEEE 754-2008], but this decision
would have increased hardware cost. Moreover, since this feature is optional in the standard, it
cannot be used in portable code.
Implementors are free to provide a NaN payload propagation scheme as a nonstandard extension enabled by a nonstandard operating mode. However, the canonical NaN scheme described
above must always be supported and should be the default mode.

It might be "optional", but it's provided by all the major general-purpose ISAs, and whole ecosystems like R have grown up assuming that CPUs do NaN payload propagation, since it's required for efficient NA support. Hopefully NumPy will assume this soon too.

Sorta tempted to say that if a weirdo ISA that no-one uses has broken IEEE-754 then it's the compiler's job to work around that, not ours. Of course this would destroy floating point performance, but wtf, you can't just leave out a chunk of IEEE 754. I get that RISC-V is supposed to scale down to special-purpose DSPs and stuff like that where you'd never run general-purpose software like NumPy, so if that kind of chip wants to leave out important functionality like this then whatever, but I sure hope that by the time people start shipping bigger chips they'll have sorted out that "nonstandard operating mode"...

I went ahead and merged the patch as it doesn't affect any other architectures. If risc-v doesn't work after this, I figure it is someone else's problem ;)