Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Pandas to_csv adds extra line terminator by default - winpython 3.8.5.0 #884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ceaza opened this issue Aug 25, 2020 · 7 comments
Open
Labels

Comments

@ceaza
Copy link

ceaza commented Aug 25, 2020

import pandas as pd
import numpy as np
df = pd.DataFrame(np.random.randint(0,15,size=(15, 4)), columns=list('ABCD'))
df.to_csv('d://data//df.csv')

# gives

,A,B,C,D

0,2,3,0,1

1,0,1,2,0

2,0,4,2,0

3,0,0,3,2

Where as with the line_terminator explicitly referenced:


df.to_csv('d://data//test.csv',line_terminator='\n')

# gives the normal output expected
,A,B,C,D
0,2,3,0,1
1,0,1,2,0
2,0,4,2,0
3,0,0,3,2
4,4,4,0,4

Did upgrade to pandas 1.1.1 from 1.1.0 but had no effect.

@stonebig
Copy link
Contributor

With what you look at the output?
Don't you think it's rather a remark to post on stackoverflow, or gitter for pandas ?

@stonebig
Copy link
Contributor

It may be normal or not. Before creating an issue on pandas dev, you may have a confirmation per expert user

@stonebig
Copy link
Contributor

Better a false alarm than no alarm.

@ceaza
Copy link
Author

ceaza commented Aug 25, 2020 via email

@ceaza
Copy link
Author

ceaza commented Sep 1, 2020

With what you look at the output?
Excel mostly
Don't you think it's rather a remark to post on stackoverflow, or gitter for pandas ?

https://stackoverflow.com/questions/63682677/pandas-to-csv-adds-extra-line-terminator-by-default-winpython-3-8-5-0-pandas

@ceaza
Copy link
Author

ceaza commented Sep 3, 2020

Looks like it's an os thing.

from : https://github.com/pandas-dev/pandas/blob/361166f86da1a665a1b5808e0935c28f8cf56d34/pandas/core/generic.py

line_terminator : str, optional
            The newline character or character sequence to use in the output
            file. Defaults to `os.linesep`, which depends on the OS in which
            this method is called ('\n' for linux, '\r\n' for Windows, i.e.).
            .. versionchanged:: 0.24.0

Somehow the implementation in 3.8.5 now thinks '\r\n' is two lines.

import pandas as pd
import numpy as np
import os
os.linesep = '\n'
df = pd.DataFrame(np.random.randint(0,15,size=(15, 4)), columns=list('ABCD'))
df.to_csv('d://data//df.csv')

gives the correct format in windows

,A,B,C,D
0,12,0,4,9
1,1,6,9,8
2,6,5,14,0
3,12,13,10,4
4,7,4,7,10

Is there an easy way to override os.linesep at system level?

@ceaza
Copy link
Author

ceaza commented Nov 16, 2020

With WinPython3.8.6 the problem persists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants