MAINT (API?): organize npyio.recfromcsv defaults #3830

jarondl · 2013-09-29T07:47:56Z

Organizes the default kwargs in recfromcsv. Changes two undocumented
kwargs behaviors:

previously, if a user set names=None, it was ignored and replaced
with names=True
the dtype kwarg was ignored. If update was given, it was used as
dtype, and if not, None was used. We can retain the update behavior
by using kwargs.setdefault("dtype",kwargs.get('update', None)).
This Closes npyio.py --> recfromcsv seems to have a bug that ignores any dtypes provided #311 .

hpaulj · 2013-11-24T23:25:23Z

You might want to add a test to test_io.py, such as:

    #
    data = TextIO('A,B\n0,1\n2,3')
    dtype = [('a', np.int), ('b', np.float)]
    test = np.recfromcsv(data, missing_values='N/A', dtype=dtype)
    control = np.array([(0, 1), (2, 3)],
                       dtype=dtype)
    self.assertTrue(isinstance(test, np.recarray))
    assert_equal(test, control)

The existing tests all use dtype=None, which works even if it mistakenly tries to get that value from update.

hpaulj · 2013-11-24T23:27:33Z

numpy/lib/npyio.py

@@ -1867,16 +1867,18 @@ def recfromcsv(fname, **kwargs):
    numpy.genfromtxt : generic function to load ASCII data.



Add:
Notes
-----
By default, dtype is None, which means that the data-type of the output
array will be determined from the data.

Hi.

I think we should consider the state of the io documentation as a whole. The four functions ndfromtxt,
mafromtxt, recfromtxt and recfromcsv exist online only as remarks in the user guide : http://docs.scipy.org/doc/numpy/user/basics.io.genfromtxt.html . They are not mentioned in http://docs.scipy.org/doc/numpy/reference/routines.io.html and therefore have no sphinx docs in the reference. Of-course they do appear in the python help() system.

Not only dtype is given a default, but also case_sensitive, names and delimiter,
so they should be addressed together. Any ideas on how to do that?

Thanks,
Yaron

Sounds like it would be a good idea to add them. The appropriate spot looks to be doc/source/reference/routines.io.rst. You can open a new PR for that.

Yeah, I'll open another PR for that.

charris · 2014-02-17T18:29:25Z

Travis error looks bogus, restarted tests.

charris · 2014-02-17T19:01:04Z

LGTM. Can you add a note to doc/release/1.9.0-notes.rst? You should probably mention the change in behavior of the update keyword.

charris · 2014-02-27T20:39:36Z

numpy/lib/npyio.py

-                  case_sensitive=case_sensitive)
-    usemask = kwargs.get("usemask", False)
+    # Set default kwargs as relevant to csv import.
+    # These seem to be undocumented. (case_sensitive is mentioned in basics.io)


These are all keywords in genfromtxt.

Should mention that these are keywords used by genfromtxt.

charris · 2014-02-27T20:42:33Z

Needs a rebase.

Organizes the default kwargs in recfromcsv. Changes two undocumented kwargs behaviors: * previously, if a user set `names=None`, it was ignored and replaced with `names=True` * the `dtype` kwarg was ignored. If `update` was given, it was used as `dtype`, and if not, None was used. We can retain the `update` behavior by using `kwargs.setdefault("dtype",kwargs.get('update', None))`. This Closes numpy#311 .

…@hpaulj

…@hpaulj

@hpaulj

Added a note to recfromcsv about the `dtype` keyword, as suggested by @hpaulj. Also added a note to the release notes, about the change in the `update` keyword, as suggested by @charris.

jarondl · 2014-02-28T07:47:05Z

It needed a rebase because of the release notes. I've rebased.

jarondl · 2014-03-11T16:47:55Z

Hi @charris, any news?

charris · 2014-03-11T21:40:26Z

numpy/lib/npyio.py

+    # Set default kwargs as relevant to csv import.
+    # These seem to be undocumented. (case_sensitive is mentioned in basics.io)
+    kwargs.setdefault("case_sensitive", "lower")
+    # previously, names were set to True if they were not set or if they


These two comment lines can be removed.

charris · 2014-03-11T21:43:59Z

LGTM modulo a few nitpicks.

Removed two irrelevant comments about code history. P.S. my first try with Github's online editor.

jarondl · 2014-03-12T19:52:59Z

The travis error is about fetching from sphinx or something. Anyway, the only changes in the commit are in comments.

jarondl · 2014-03-25T08:40:01Z

Hi @charris or any other dev, could you please try to rebuild the travis run?

MAINT (API?): organize npyio.recfromcsv defaults

charris · 2014-03-29T00:33:45Z

Thanks @jarondl.

hpaulj reviewed Nov 24, 2013
View reviewed changes

jarondl mentioned this pull request Nov 26, 2013

npyio.py --> recfromcsv seems to have a bug that ignores any dtypes provided #311

Closed

charris reviewed Feb 27, 2014
View reviewed changes

Yaron de Leeuw added 3 commits February 28, 2014 09:18

TST: Added regression test for recfromcsv dtype error, as suggested by …

3b99f37

…@hpaulj

DOC: npyio.recfromcsv keyword argument changes

39f1133

Added a note to recfromcsv about the `dtype` keyword, as suggested by @hpaulj. Also added a note to the release notes, about the change in the `update` keyword, as suggested by @charris.

charris reviewed Mar 11, 2014
View reviewed changes

STY: change and delete some comments in recfromcsv

aff43c0

Removed two irrelevant comments about code history. P.S. my first try with Github's online editor.

charris added a commit that referenced this pull request Mar 29, 2014

Merge pull request #3830 from jarondl/recfromcsv

bce5bfb

MAINT (API?): organize npyio.recfromcsv defaults

charris merged commit bce5bfb into numpy:master Mar 29, 2014

jarondl deleted the recfromcsv branch March 30, 2014 05:46

normallytangent mentioned this pull request Apr 12, 2019

DOC: add missing routines to reference guide #13214

Closed

11 tasks

		@@ -1867,16 +1867,18 @@ def recfromcsv(fname, **kwargs):
		numpy.genfromtxt : generic function to load ASCII data.

Uh oh!

MAINT (API?): organize npyio.recfromcsv defaults #3830

MAINT (API?): organize npyio.recfromcsv defaults #3830

Uh oh!

Conversation

jarondl commented Sep 29, 2013

Uh oh!

hpaulj commented Nov 24, 2013

Uh oh!

hpaulj Nov 24, 2013

Choose a reason for hiding this comment

Uh oh!

jarondl Nov 25, 2013

Choose a reason for hiding this comment

Uh oh!

charris Feb 27, 2014

Choose a reason for hiding this comment

Uh oh!

jarondl Feb 28, 2014

Choose a reason for hiding this comment

Uh oh!

charris commented Feb 17, 2014

Uh oh!

charris commented Feb 17, 2014

Uh oh!

charris Feb 27, 2014

Choose a reason for hiding this comment

Uh oh!

charris Mar 11, 2014

Choose a reason for hiding this comment

Uh oh!

charris commented Feb 27, 2014

Uh oh!

jarondl commented Feb 28, 2014

Uh oh!

jarondl commented Mar 11, 2014

Uh oh!

charris Mar 11, 2014

Choose a reason for hiding this comment

Uh oh!

charris commented Mar 11, 2014

Uh oh!

jarondl commented Mar 12, 2014

Uh oh!

jarondl commented Mar 25, 2014

Uh oh!

charris commented Mar 29, 2014

Uh oh!

Uh oh!