Thanks to visit codestin.com
Credit goes to github.com

Skip to content

BUG: Savetxt may ignore newline character on Windows: #3976

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

BUG: Savetxt may ignore newline character on Windows: #3976

wants to merge 2 commits into from

Conversation

816-8055
Copy link

Under some circumstances savetxt ignores the newline character on Python 2.7 running on Windows:
If newline is set to '\n', actually '\n\r' is written, because of the linesep conversion of file.write.

Closes #3975.

@charris
Copy link
Member

charris commented Oct 25, 2013

I'm not clear on the issue here, but generally LGTM. The change needs a test, and the commit message needs fixing, something like:

BUG: Savetxt may ignore newline character on Windows:

Extended explanation of problem...

Closes #3975.

@charris
Copy link
Member

charris commented Oct 25, 2013

Does loadtxt need a complimentary fix?

@816-8055
Copy link
Author

Sorry, I'm new to all this. Is there a quick-start-guide anywhere?
How can I commit a test? And can I edit the commit message to match your example?

loadtxt doesn't have a newline parameter, and therefore there's no need for a fix.

@charris
Copy link
Member

charris commented Oct 31, 2013

There is http://docs.scipy.org/doc/numpy/dev/, but is may not have what you are looking for.

To edit commit messages or squash commits for fixes, the easiest way it git rebase -i HEAD~n, where n is the number of commits you want to fix up. There are instructions that will come up when you do that, but for commit messages replace pick with r. After that you will need to force push git push --force origin whatever.

For tests, look to use existing tests as templates. The place for the test will be somewhere in numpy/lib/tests/test_io.py.

Under some circumstances savetxt ignores the newline character on Python 2.7 running on Windows:
If newline is set to '\n', actually '\n\r' is written, because of the linesep conversion of file.write.

Closes #3975.
@816-8055
Copy link
Author

816-8055 commented Nov 6, 2013

Thanks for your help.
I've just changed the commit message.

After looking at numpy/lib/tests/test_io.py I found the tests in class TestSaveTxt. Almost all of them check for a \n as line-separator but as the data is written to a BytesIO object and not a file, the bug was not detected. I'll now write my own test function.

Tests the correct usage of newline in savetxt.

See #3976
@816-8055
Copy link
Author

816-8055 commented Nov 7, 2013

Running my test with the unpatched version fails, as expected:

FAIL: test_newline (test_io.TestSaveTxt)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "numpy\lib\tests\test_io.py", line 433, in test_newline
    assert_equal(d, newline.join(['1', '2', '3', '']))
  File "numpy\ma\testutils.py", line 100, in assert_equal
    raise AssertionError(msg)
AssertionError: 
Items are not equal:
 ACTUAL: '1\r\n2\r\n3\r\n'
 DESIRED: '1\n2\n3\n'

With the patched version the test is passed, but 7 others fail:
test_1D, test_array, test_complex_arrays, test_delimiter, test_format, test_header_footer, test_record.
These test-function expect newline to be set to \n, that's the reason for those fails.

Possible solutions:

  1. replacing os.linesep with \n in my patch
  2. changing all tests to explicitly set newline to \n
  3. replacing \n with os.linesep in all tests

What do you think?

@charris
Copy link
Member

charris commented Mar 13, 2014

@816-8055 Sorry for the long pause. I think the issues you raise would be best discussed on the mailing list before proceeding.

fh = open(fname, 'wb')
else:
fh = open(fname, 'w')
fh = open(fname, 'wb')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can do this because of the use of os.linesp? Might be worth a comment.

charris pushed a commit to charris/numpy that referenced this pull request Jan 25, 2015
Tests the correct usage of newline in savetxt.

See numpy#3976
@charris
Copy link
Member

charris commented Jan 25, 2015

Cleanup at #5503, closing this. Thanks @816-8055 .

@charris charris closed this Jan 25, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

savetxt ignores newline on Python 2.7
2 participants