Thanks to visit codestin.com
Credit goes to github.com

Skip to content

perf(python): make 1D numpy to polars conversion zero-copy for numeric data #12403

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
ritchie46 merged 1 commit into from
Nov 13, 2023
Merged

perf(python): make 1D numpy to polars conversion zero-copy for numeric data #12403

ritchie46 merged 1 commit into from
Nov 13, 2023

Conversation

@ritchie46
Copy link
Member

@ritchie46 ritchie46 commented Nov 12, 2023

This was actually very easy to add. We keep the numpy object alive as foreign memory owner and memory map the data.

a = np.ones(int(1e7))

# zero copy series conversion
pl.Series("a", a)

# zero copy DataFrame conversion
pl.DataFrame({
   "a": a,
   "b": a
})

@github-actions github-actions bot added performance Performance issues or improvements python Related to Python Polars labels Nov 12, 2023
@ritchie46 ritchie46 mentioned this pull request Nov 12, 2023
Merged
@ritchie46 ritchie46 added the highlight Highlight this PR in the changelog label Nov 13, 2023
@ritchie46 ritchie46 merged commit 7711af2 into main Nov 13, 2023
@ritchie46 ritchie46 deleted the numpy branch November 13, 2023 07:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@stinodego stinodego Awaiting requested review from stinodego

@alexander-beedie alexander-beedie Awaiting requested review from alexander-beedie alexander-beedie is a code owner

@MarcoGorelli MarcoGorelli Awaiting requested review from MarcoGorelli MarcoGorelli is a code owner

@orlp orlp Awaiting requested review from orlp orlp is a code owner

Assignees

No one assigned

Labels

highlight Highlight this PR in the changelog performance Performance issues or improvements python Related to Python Polars

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ritchie46