Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Improve performance of os.join by replacing map with a direct method call #117648

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
eendebakpt opened this issue Apr 8, 2024 · 2 comments
Closed

Improve performance of os.join by replacing map with a direct method call #117648

eendebakpt opened this issue Apr 8, 2024 · 2 comments
Labels
performance Performance or resource usage stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@eendebakpt
Copy link
Contributor

eendebakpt commented Apr 8, 2024 •
edited by bedevere-app bot
Loading

Feature or enhancement

Proposal:

We can improve performance of os.join by changing

        for b in map(os.fspath, p):

into

        for w in p:
            b=os.fspath(w)

The map generator takes time to create and the application of the method to each element also takes some time. A quick benchmark

main: 385 ns ± 11.5 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)
map replaced: 328 ns ± 9.8 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)

The idea still needs to be tested on other platforms and with longer sequences.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs
@eendebakpt eendebakpt added the type-feature A feature request or enhancement label Apr 8, 2024
@eendebakpt
Copy link
Contributor Author

@nineteendo Is this something you are interested in to implement?

@nineteendo
Copy link
Contributor

Yeah, I'll make another PR. :-) For ntpath.join() we can even remove the os.fspath() call alltogether, as we already have splitroot():

-for p in map(os.fspath, paths):
+for p in paths:
    p_drive, p_root, p_path = splitroot(p)

@bedevere-app bedevere-app bot mentioned this issue Apr 8, 2024
Merged
@erlend-aasland erlend-aasland added performance Performance or resource usage stdlib Python modules in the Lib dir labels Apr 9, 2024
erlend-aasland pushed a commit that referenced this issue Apr 9, 2024
@nineteendo @eendebakpt
gh-117648: Improve performance of os.join (#117654)
99852d9 
Replace map() with a method call in the loop body.

Co-authored-by: Pieter Eendebak <[email protected]>
This was referenced Apr 9, 2024
Closed
Merged
erlend-aasland added a commit that referenced this issue Apr 9, 2024
@erlend-aasland
gh-117648: Amend NEWS entry (#117697)
ca75915 
Make the wording more vague; the performance impact varies a lot
depending on platform and input.
diegorusso pushed a commit to diegorusso/cpython that referenced this issue Apr 17, 2024
@nineteendo @eendebakpt @diegorusso
pythongh-117648: Improve performance of os.join (python#117654)
ccbd15b 
Replace map() with a method call in the loop body.

Co-authored-by: Pieter Eendebak <[email protected]>
diegorusso pushed a commit to diegorusso/cpython that referenced this issue Apr 17, 2024
@erlend-aasland @diegorusso
pythongh-117648: Amend NEWS entry (python#117697)
2eb58d1 
Make the wording more vague; the performance impact varies a lot
depending on platform and input.
@nineteendo nineteendo mentioned this issue Apr 26, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
performance Performance or resource usage stdlib Python modules in the Lib dir type-feature A feature request or enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@eendebakpt @erlend-aasland @nineteendo