Thanks to visit codestin.com
Credit goes to github.com

Skip to content

GH-119169: Speed up os.walk(topdown=False) #119186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
barneygale wants to merge 12 commits into from
Closed

GH-119169: Speed up os.walk(topdown=False) #119186

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
74a495c
GH-119169: Speed up `os.walk()`
barneygale May 19, 2024
eb4b6ff
Merge branch 'main' into os-walk-speedup
barneygale May 23, 2024
46a3ba5
Simplify bottom-up implementation.
barneygale May 23, 2024
2d957a8
Merge branch 'main' into os-walk-speedup
barneygale May 30, 2024
a638202
Apply the same changes to fwalk()
barneygale May 30, 2024
91b5a78
Simplify code slightly
barneygale May 30, 2024
b32c1c0
Simplify exception handling in fwalk()
barneygale Jul 6, 2024
8c7cf04
Merge branch 'main' into os-walk-speedup
barneygale Jul 6, 2024
efd31e2
Undo trailing slash optimisation (moved to another PR)
barneygale Jul 6, 2024
5b89c57
Undo fwalk() changes.
barneygale Jul 6, 2024
525618c
Fix news entry.
barneygale Jul 6, 2024
d3af655
Merge branch 'main' into os-walk-speedup
barneygale Jul 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Simplify exception handling in fwalk()
  • Loading branch information
@barneygale
barneygale committed Jul 6, 2024
commit b32c1c0641f9cec2e6788b0ae49ba61d9b9b3d75
49 changes: 21 additions & 28 deletions Lib/os.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -505,35 +505,28 @@ def _fwalk(stack, isbytes, topdown, onerror, follow_symlinks):
stack.append((_fwalk_yield, (toppath, dirs, nondirs, topfd)))

topprefix = path.join(toppath, toppath[:0]) # Add trailing slash.
try:
for entry in scandir(topfd):
name = entry.name
if isbytes:
name = fsencode(name)

try:
is_dir = entry.is_dir()
except OSError:
# If is_dir() raises an OSError, consider the entry not to
# be a directory, same behaviour as os.path.isdir().
is_dir = False
for entry in scandir(topfd):
name = entry.name
if isbytes:
name = fsencode(name)

if is_dir:
if not topdown:
# Bottom-up: traverse into sub-directory.
stack.append(
(_fwalk_walk, (
False, topfd, topprefix + name, name,
None if follow_symlinks else entry)))
dirs.append(name)
else:
nondirs.append(name)
except:
# Undo additions to stack in bottom-up mode.
if not topdown:
while stack.pop()[0] != _fwalk_yield:
pass
raise
try:
is_dir = entry.is_dir()
except OSError:
# If is_dir() raises an OSError, consider the entry not to
# be a directory, same behaviour as os.path.isdir().
is_dir = False

if is_dir:
if not topdown:
# Bottom-up: traverse into sub-directory.
stack.append(
(_fwalk_walk, (
False, topfd, topprefix + name, name,
None if follow_symlinks else entry)))
dirs.append(name)
else:
nondirs.append(name)

if topdown:
# Yield before sub-directory traversal if going top down
Expand Down