mkdir: Fix stack overflow with deeply nested directories #8947

naoNao89 · 2025-10-18T00:02:05Z

mkdir -p with 200+ nested directories? Segfault. The recursive implementation ate stack frames like candy.

Fix
Swapped recursion for iteration. Collect parent directories into a vector (heap), create them root-to-leaf. Constant stack space, infinite depth.

Proof It Works

# Before: Segmentation fault: 11
$ ulimit -s 128
$ ./target/release/coreutils mkdir -p `python3 -c 'print("a/" * 250)'`
Segmentation fault: 11

AFTER THE FIX:

$ ulimit -s 128  
$ ./target/release/coreutils mkdir -p `python3 -c 'print("a/" * 250)'`
(success - directories created)

$ ulimit -s 128
$ ./target/release/coreutils mkdir -p `python3 -c 'print("a/" * 400)'`  
(success - even deeper nesting works)

Our changes touch code originally by 8ccc45c (2022)

Replaced recursive parent directory creation with iterative approach to prevent stack overflow when creating directories with 200+ nesting levels. The previous recursive implementation consumed one stack frame per directory level, causing crashes on systems with limited stack size. The new implementation collects parent directories into a vector and creates them iteratively, using constant stack space regardless of depth. All existing functionality preserved: verbose output, permissions, SELinux context, and ACLs. Includes regression test for 350-level directory nesting.

github-actions · 2025-10-18T00:24:10Z

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)

codspeed-hq · 2025-10-18T04:19:19Z

CodSpeed Performance Report

Merging #8947 will not alter performance

_{Comparing naoNao89:fix/mkdir-stack-overflow (33a736e) with main (130bbf5)}

Summary

✅ 104 untouched
🗄️ 1 archived benchmark run¹

1 benchmark was run, but is now archived. If it was deleted in another branch, consider rebasing to remove it from the report. Instead if it was added back, click here to restore it. ↩

github-actions · 2025-10-18T04:27:01Z

GNU testsuite comparison:

Skip an intermittent issue tests/timeout/timeout (fails in this run but passes in the 'main' branch)

github-actions · 2025-10-18T04:54:06Z

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)

github-actions · 2025-10-18T06:53:36Z

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)

.vscode/cspell.dictionaries/jargon.wordlist.txt

tests/by-util/test_mkdir.rs

src/uu/mkdir/src/mkdir.rs

github-actions · 2025-10-25T05:27:02Z

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

github-actions · 2025-10-26T02:55:26Z

GNU testsuite comparison:

Skipping an intermittent issue tests/misc/tee (passes in this run but fails in the 'main' branch)

tests/by-util/test_mkdir.rs

asder8215 · 2025-10-26T10:10:38Z

src/uu/mkdir/src/mkdir.rs

+        }
+
+        // Reverse to create from root to leaf
+        dirs_to_create.reverse();


I know that you use a Vec for the dirs_to_create, but is it not possible to use a VecDeque for this? You could use it like a stack to create from root to leaf to avoid the .reverse() call here.

vecDeque caused du regression - its push_front overhead wasn't worth it. Standard Vec + reverse() is faster

Alright, good to note. Also, it's a good idea that you use a reverse iterator instead of reversing the actual Vec to create each directory since reverse iterator don't modify the elements in place, but just traverses in the reverse direction.

src/uu/mkdir/src/mkdir.rs

github-actions · 2025-10-26T18:19:07Z

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

github-actions · 2025-10-26T19:22:08Z

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)

asder8215 · 2025-10-26T22:34:18Z

Hey @naoNao89, I don't think there is an issue with the logic in your code for this, but could you create test cases for the following:

mkdir -pv a/\"\"/b/c

On Linux/Unix machines, double quotes are allowed in filepaths, but on Windows this is not allowed. More specifically, in a test case like above, on Windows, the "a" directory will be created but once it sees that double quote, it should produce an error like this I believe:

mkdir: cannot create directory 'a/""': Invalid argument

(you may want to double check this though because I just remember something like this when I used to have my machine on Windows)

Linux/Unix machines will create this directory tree fine though:

a
├── ""
      ├── b
           ├── c

Do the same thing as well with:

mkdir -p a/\'\'/b/c

I'm pretty certain that both Unix and non-Unix OS's should accept and create the following directory tree:

a
├── ''
     ├── b
            ├── c

tests/by-util/test_mkdir.rs

- Add platform-specific assertions for double quotes in paths - On Unix: Verify mkdir -pv a/""/b/c succeeds and creates directory with quotes - On Windows: Verify mkdir -pv a/""/b/c fails after creating 'a' directory with appropriate error - Fix single quotes test to use a/''/b/c path as requested by reviewer - Ensure tests verify expected behavior rather than just checking if succeeded Addresses reviewer feedback from PR uutils#8947

…_behavior The test was not actually testing concurrent directory creation, but rather the idempotent behavior of mkdir -p when called multiple times on the same directories. This rename accurately reflects what the test is verifying: - mkdir -p can be called multiple times on existing directories without errors - The command properly handles existing directories without failing - Multiple calls to create the same directory structure work correctly This addresses the misleading test name and clarifies the actual behavior being tested. GNU mkdir -p is designed to be concurrency-safe rather than concurrency-atomic, handling race conditions gracefully when they occur.

Removed obvious comments that just state what the code is clearly doing: - '// Set an environment variable' before std::env::set_var - '// Clean up' before std::env::remove_var The code is self-explanatory and these comments add no value.

Fix rustfmt formatting violation in test_mkdir_control_characters. The multi-line assert statement with OR condition was not properly formatted according to the project's rustfmt configuration. This resolves a CI formatting check failure.

Correct inaccurate comment in test_mkdir_control_characters. The test was actually checking space characters in directory names, not backspace characters as the comment incorrectly stated. This addresses reviewer feedback about the confusing comment.

Windows test was failing because the error message for invalid directory names with quotes doesn't contain the expected strings. Different Windows versions and locales use different error messages. The test now focuses on the core behavior: - Unix: Should succeed and create directories with quotes - Windows: Should fail and not create directories with quotes This makes the test more robust across different Windows environments while still verifying the platform-specific behavior.

github-actions · 2025-10-27T14:24:46Z

GNU testsuite comparison:

Skipping an intermittent issue tests/misc/tee (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

asder8215

Hey @naoNao89, I wanted to correct what I mentioned earlier about not worrying about concurrent test cases. Apparently, from this stack exchange post: https://unix.stackexchange.com/questions/523434/is-mkdir-p-safe-to-parallelize

mkdir -p is safe to parallelize. You may want to check if this implementation of mkdir operates correctly in this manner.

Conveniently, there's a test case for create_dir_all() within rust's std lib that checks for this case:

#[test]
fn concurrent_recursive_mkdir() {
    for _ in 0..100 {
        let dir = tmpdir();
        let mut dir = dir.join("a");
        for _ in 0..40 {
            dir = dir.join("a");
        }
        let mut join = vec![];
        for _ in 0..8 {
            let dir = dir.clone();
            join.push(thread::spawn(move || {
                check!(fs::create_dir_all(&dir));
            }))
        }

        // No `Display` on result of `join()`
        join.drain(..).map(|join| join.join().unwrap()).count();
    }
}

You can find this here. Maybe it might be a good idea to take inspiration from that function to make a concurrent test case here?

Aside from that, everything looks good to me here. Thanks for all the work on this PR!

github-actions · 2025-10-28T12:27:12Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

github-actions · 2025-10-28T12:52:48Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

Add concurrent test based on Rust's std::fs::create_dir_all() pattern: - 100 iterations, 8 threads, 40 levels nesting - Tests that mkdir -p is safe to parallelize - Uses Rust's thread joining pattern - Addresses reviewer request for concurrent testing

- Use at.plus() to get full path to test fixtures directory - Use std::fs::create_dir_all directly instead of system mkdir - Remove unused PathBuf import - Fixes CI failure where test couldn't find created directories

github-actions · 2025-10-28T13:32:20Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

tests/by-util/test_mkdir.rs

- Use TestScenario.bin_path to get the actual uutils binary - Execute real mkdir -p command instead of std::fs::create_dir_all - This tests the actual coreutils implementation as requested by reviewer - Reduce iterations to 10 for faster test execution while maintaining coverage - Fixes feedback that test was only testing std lib implementation

- Fix clippy manual_assert warning - Use assert! instead of if !success { panic! } - Format assert! call according to rustfmt requirements - Maintains same error message and behavior - Passes clippy pedantic checks

github-actions · 2025-10-28T15:12:54Z

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

asder8215

Looks good to me! 👍

mkdir: Fix stack overflow with deeply nested directories

sylvestre reviewed Oct 18, 2025

View reviewed changes

.vscode/cspell.dictionaries/jargon.wordlist.txt Outdated Show resolved Hide resolved