Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit c899475

Browse files
rgommersrgommers
authored
Merge pull request #22 from data-apis/scope-stakeholders
Add Scope and Stakeholders sections
2 parents f14aebe + e05864b commit c899475

File tree

7 files changed

+230
-8
lines changed

7 files changed

+230
-8
lines changed

‎spec/_static/images/scope_of_array_API.png

83.5 KB
Loading

‎spec/design_topics/device_support.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
.. _device-support:
2+
3+
# Device support
4+
5+
TODO. See https://github.com/data-apis/array-api/issues/39
6+

‎spec/design_topics/index.rst

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,5 @@ Design topics & constraints
1111
static_typing
1212
array_ducktyping
1313
data_interchange
14-
missing_data
15-
portability
14+
device support
1615
C_API

‎spec/design_topics/missing_data.md

Lines changed: 0 additions & 1 deletion
This file was deleted.

‎spec/design_topics/portability.md

Lines changed: 0 additions & 1 deletion
This file was deleted.

‎spec/purpose_and_scope.md

Lines changed: 217 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,18 +3,221 @@
33
## Introduction
44

55

6+
### This API standard
7+
68

79
## History
810

911

1012

1113
## Scope (includes out-of-scope / non-goals)
1214

15+
This section outlines what is in scope and out of scope for this API standard.
1316

17+
### In scope
1418

15-
## Stakeholders
19+
The scope of the array API standard includes:
20+
21+
- Functionality which needs to be included in an array library for it to adhere
22+
to this standard.
23+
- Names of functions, methods, classes and other objects.
24+
- Function signatures, including type annotations.
25+
- Semantics of functions and methods. I.e. expected outputs including precision
26+
for and dtypes of numerical results.
27+
- Semantics in the presence of `nan`'s, `inf`'s, empty arrays (i.e. arrays
28+
including one or more dimensions of size `0`).
29+
- Casting rules, broadcasting, indexing
30+
- Data interchange. I.e. protocols to convert one type of array into another
31+
type, potentially sharing memory.
32+
- Device support.
33+
34+
Furthermore, meta-topics included in this standard include:
35+
36+
- Use cases for the API standard and assumptions made in it
37+
- API standard adoption
38+
- API standard versioning
39+
- Future API standard evolution
40+
- Array library and API standard versioning
41+
- Verification of API standard conformance
42+
43+
The concrete set of functionality that is in scope for this version of the
44+
standard is shown in this diagram (_TODO: update after deciding on how optional
45+
extensions are dealt with_):
46+
47+
![Scope of array API](_static/images/scope_of_array_API.png)
48+
49+
50+
**Goals** for the API standard include:
51+
52+
- Make it possible for array-consuming libraries to start using multiple types
53+
of arrays as inputs.
54+
- Enable more sharing and reuse of code built on top of the core functionality
55+
in the API standard.
56+
- For authors of new array libraries, provide a concrete API that can be
57+
adopted as is, rather than each author having to decide what to borrow from
58+
where and where to deviate.
59+
- Make the learning curve for users less steep when they switch from one array
60+
library to another one.
61+
62+
63+
### Out of scope
64+
65+
1. Implementations of the standard are out of scope.
66+
67+
_Rationale: the standard will consist of a document and an accompanying test
68+
suite with which the conformance of an implementation can be verified. Actual
69+
implementations will live in array libraries; no reference implementation is
70+
planned._
71+
72+
2. Execution semantics are out of scope. This includes single-threaded vs.
73+
parallel execution, task scheduling and synchronization, eager vs. delayed
74+
evaluation, performance characteristics of a particular implementation of the
75+
standard, and other such topics.
76+
77+
_Rationale: execution is the domain of implementations. Attempting to specify
78+
execution behavior in a standard is likely to require much more fine-grained
79+
coordination between developers of implementations, and hence is likely to
80+
become an obstable to adoption._
81+
82+
3. Non-Python API standardization (e.g., Cython or NumPy C APIs)
83+
84+
_Rationale: this is an important topic for some array-consuming libraries,
85+
but there is no widely shared C/Cython API and hence it doesn't make sense at
86+
this point in time to standardize anything. See
87+
[the C API section](design_topics/C_API.md) for more details._
88+
89+
4. Standardization of these dtypes is out of scope: bfloat16, complex, extended
90+
precision floating point, datetime, string, object and void dtypes.
91+
92+
_Rationale: these dtypes aren't uniformly supported, and their inclusion at
93+
this point in time could put a significant implementation burden on
94+
libraries. It is expected that some of these dtypes - in particular
95+
`bfloat16`, `complex64`, and `complex128` - will be included in a future
96+
version of the standard._
97+
98+
5. The following topics are out of scope: I/O, polynomials, error handling,
99+
testing routines, building and packaging related functionality, methods of
100+
binding compiled code (e.g., `cffi`, `ctypes`), subclassing of an array
101+
class, masked arrays, and missing data.
16102

103+
_Rationale: these topics are not core functionality for an array library,
104+
and/or are too tied to implementation details._
17105

106+
6. NumPy (generalized) universal functions, i.e. ufuncs and gufuncs.
107+
108+
_Rationale: these are NumPy-specific concepts, and are mostly just a
109+
particular way of building regular functions with a few extra
110+
methods/properties._
111+
112+
113+
**Non-goals** for the API standard include:
114+
115+
- Making array libraries identical so they can be merged.
116+
117+
_Each library will keep having its own particular strength, whether it's
118+
offering functionality beyond what's in the standard, performance advantages
119+
for a given use case, specific hardware or software environment support, or
120+
more._
121+
122+
- Implement a backend or runtime switching system to be able to switch from one
123+
array library to another with a single setting or line of code.
124+
125+
_This may be feasible, however it's assumed that when an array-consuming
126+
library switches from one array type to another, some testing and possibly
127+
code adjustment for performance or other reasons may be needed._
128+
129+
130+
### TBD whether or not in scope, or for a later version
131+
132+
- Random number generation, Fourier transforms, and miscellaneous functionality
133+
like a padding function.
134+
135+
_This will be decided later, depending on whether "optional extensions" will
136+
be added to the standard._
137+
138+
139+
### Implications of in/out of scope
140+
141+
If something is out of scope and therefore will not be part of (the current
142+
version of) the API standard, that means that there are no guarantees that that
143+
functionality works the same way, or even exists at all, across the set of
144+
array libraries that conform to the standard. It does _not_ imply that this
145+
functionality is less important or should not be used.
146+
147+
148+
## Stakeholders
149+
150+
Arrays are fundamental to scientific computing, data science, and machine
151+
learning and deep learning. Hence there are many stakeholders for an array API
152+
standard. The _direct_ stakeholders of this standard are **authors/maintainers of
153+
Python array libraries**. There are many more types of _indirect_ stakeholders
154+
though, including:
155+
156+
- maintainers of libraries and other programs which depend on array libraries
157+
(called "array-consuming libraries" in the rest of this document)
158+
- authors of non-Python array libraries
159+
- developers of compilers and runtimes with array-specific functionality
160+
- end users
161+
162+
Libraries that are being actively considered - in terms of current behaviour and
163+
API surface - during the creation of the first version of this standard
164+
include:
165+
166+
- [NumPy](https://numpy.org)
167+
- [TensorFlow](https://www.tensorflow.org/)
168+
- [PyTorch](https://pytorch.org/)
169+
- [MXNet](https://numpy.mxnet.io/)
170+
- [JAX](https://github.com/google/jax)
171+
- [Dask](https://dask.org/)
172+
- [CuPy](https://cupy.chainer.org/)
173+
174+
Other Python array libraries that are currently under active development and
175+
could adopt this API standard include:
176+
177+
- [xarray](https://xarray.pydata.org/)
178+
- [PyData/Sparse](https://sparse.pydata.org)
179+
- [Weld](https://github.com/weld-project/weld)
180+
- [Bohrium](https://bohrium.readthedocs.io/)
181+
- [Arkouda](https://github.com/mhmerrill/arkouda)
182+
- [Legate](https://research.nvidia.com/publication/2019-11_Legate-NumPy%3A-Accelerated)
183+
184+
There are a huge amount of array-consuming libraries; some of the most
185+
prominent ones that are being taken into account - in terms of current array
186+
API usage or impact of design decisions on them - include (this list is likely
187+
to grow it over time):
188+
189+
- [Pandas](https://pandas.pydata.org/)
190+
- [SciPy](https://github.com/scipy/scipy)
191+
- [scikit-learn](https://scikit-learn.org/)
192+
- [Matplotlib](https://matplotlib.org/)
193+
- [scikit-image](https://scikit-image.org/)
194+
- [NetworkX](https://networkx.github.io/)
195+
196+
Array libraries in other languages, some of which may grow a Python API in the
197+
future or have taken inspiration from NumPy or other array libraries, include:
198+
199+
- [Xtensor](https://xtensor.readthedocs.io) (C++, cross-language)
200+
- [XND](https://xnd.io/) (C, cross-language)
201+
- [stdlib](https://stdlib.io/) (JavaScript)
202+
- [rust-ndarray](https://github.com/rust-ndarray/ndarray) (Rust)
203+
- [rray](https://github.com/r-lib/rray) (R)
204+
- [ND4J](https://github.com/deeplearning4j/nd4j) (JVM)
205+
- [NumSharp](https://github.com/SciSharp/NumSharp) (C#)
206+
207+
Compilers, runtimes, and dispatching layers for which this API standard may be
208+
relevant:
209+
210+
- [Cython](https://cython.org/)
211+
- [Numba](http://numba.pydata.org/)
212+
- [Pythran](https://pythran.readthedocs.io/en/latest/)
213+
- [Transonic](https://transonic.readthedocs.io)
214+
- [ONNX](https://onnx.ai/)
215+
- [Apache TVM](https://tvm.apache.org/)
216+
- [MLIR](https://mlir.llvm.org/)
217+
- [TACO](https://github.com/tensor-compiler/taco)
218+
- [unumpy](https://github.com/Quansight-Labs/unumpy)
219+
- [einops](https://github.com/arogozhnikov/einops)
220+
- [Apache Arrow](https://arrow.apache.org/)
18221

19222

20223
## High-level API overview
@@ -34,9 +237,19 @@ For guidance on how to read and understand the type annotations included in this
34237

35238
## Conformance
36239

37-
A conforming implementation of the array API standard must provide and support all the functions, arguments, data types, syntax, and semantics described in this specification.
240+
A conforming implementation of the array API standard must provide and support
241+
all the functions, arguments, data types, syntax, and semantics described in
242+
this specification.
243+
244+
A conforming implementation of the array API standard may provide additional
245+
values, objects, properties, data types, and functions beyond those described
246+
in this specification.
247+
248+
Libraries which aim to provide a conforming implementation but haven't yet
249+
completed such an implementation may, and are encouraged to, provide details on
250+
the level of (non-)conformance. For details on how to do this, see
251+
[Verification - measuring conformance](verification_test_suite.md).
38252

39-
A conforming implementation of the array API standard may provide additional values, objects, properties, data types, and functions beyond those described in this specification.
40253

41254
* * *
42255

@@ -92,4 +305,4 @@ a one-dimensional array.
92305

93306
The following referenced documents are indispensable for the application of this specification.
94307

95-
- __IEEE 754-2019: IEEE Standard for Floating-Point Arithmetic.__ Institute of Electrical and Electronic Engineers, New York (2019).
308+
- __IEEE 754-2019: IEEE Standard for Floating-Point Arithmetic.__ Institute of Electrical and Electronic Engineers, New York (2019).

‎spec/verification_test_suite.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,8 @@
11
# Verification - test suite
22

3+
4+
5+
## Measuring conformance
6+
7+
TODO: explain how to use the test suite to measure and report on the level of
8+
(non-)conformance.

0 commit comments

Comments
 (0)