Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@sorathpanzer
Copy link

@sorathpanzer sorathpanzer commented Dec 13, 2025

Which issue does this PR resolve?

Currently yazi fails to build in OpenBSD, this PR solves the problem, and I been able
to use yazi in my OpenBSD 7.8-Current AMD64 machine.

Rationale of this PR

It was a simple solution, some target_os specifics, also turn libc::RLIMIT_AS intto libc::RLIMIT_DATA.
Also disable tikv-jemallocator for OpenBSD target.
I had also to change lto to false, that is crucial for a standard OpenBSD, I hope some better idea to enable lto
in a standard OpenBSD.
I am not a developer, just trying to contribute... keep the awesome work.

@@ -1,4 +1,4 @@
#[cfg(all(not(target_os = "macos"), not(target_os = "windows")))]
#[cfg(all(not(target_os = "macos"), not(target_os = "windows"), not(target_os = "openbsd")))]
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you tell me what problem this resolves exactly?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I understand OpenBSD uses its own allocator, third-party allocators (jemalloc, tcmalloc, mimalloc) are intentionally not supported,
OpenBSD has a very strong security focus.
tikv_jemallocator is known to build failure in OpenBSD
- missing symbols
- unsupported platform
- allocator conflicts

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which allocator does OpenBSD use?

We're using jemallocator here because in the past users reported that on Linux the default allocator (glibc) didn't release memory promptly, but I'm not sure if OpenBSD has the same problem.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OpenBSD doesn't use glibc malloc, instead uses a libc malloc optimized for security.

  • Hardened by default
  • Memory randomization
  • Minimal Thread arenas

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you seen a continuous increase in memory usage while running Yazi, especially when switching back and forth between multiple image previews?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yazi image preview, memory speaking, in OpenBSD is not eficient as Linux,
but experimenting my self with large hundreds of photos, it is still good,
went from the initial 20M to (after large hundred of images) about 165M.
But yes, the memory will very slightly increase, that's not a problem of Yazi,
many packages in OpenBSD has these problem, they are very
conservative and always prioritizes security over... anything else. :)

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will the memory drop from 165M to 20M after some time? Or does it stay at 165M until the program exits?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, it doesn't! I don't know if there is a way to solve it in OpenBSD.
And my technical habilities in OpenBSD are also limited, I must confess, well not just in OpenBSD! :)
For me at least is not a problem, maybe is a problem if someone going through
large thousands of images, but yes, the problem although subtle, is there.


#[cfg(target_os = "openbsd")]
fn final_path(path: &Path) -> io::Result<PathBuf> {
std::fs::canonicalize(path)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not right as final_path should not follow symlinks but canonicalize does

Copy link
Author

@sorathpanzer sorathpanzer Dec 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right — canonicalize() follows symlinks and changes semantics.
I used it as a workaround for OpenBSD.
I can replace it with a component-based normalization that preserves symlinks and matches the original behavior.

#[cfg(target_os = "openbsd")]
fn final_path(path: &Path) -> io::Result<PathBuf> {
    use std::path::{Component, PathBuf};

    let mut out = PathBuf::new();

    for comp in path.components() {
        match comp {
            Component::Prefix(_) => out.push(comp),
            Component::RootDir => out.push(comp),
            Component::CurDir => {}
            Component::ParentDir => {
                out.pop();
            }
            Component::Normal(c) => out.push(c),
        }
    }

    Ok(out)
}

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

component-based normalization won't work in this case unfortunately since final_path needs to obtain the actual path of the given path on the filesystem without following symlinks, which means that when passed /root/abc, if the filesystem is case‑insensitive and the real entry is named ABC, it should return /root/ABC instead.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see, maybe this way instead?

#[cfg(target_os = "openbsd")]
fn final_path(path: &Path) -> io::Result<PathBuf> {
    use std::{fs, os::unix::fs::MetadataExt, path::PathBuf};

    let meta = fs::symlink_metadata(path)?; 
    let parent = path.parent().ok_or_else(|| std::io::Error::new(
        std::io::ErrorKind::Other,
        "Cannot get parent directory",
    ))?;

    // scan parent dir
    for entry in fs::read_dir(parent)? {
        let entry = entry?;
        let entry_meta = entry.metadata()?;
        if entry_meta.ino() == meta.ino() && entry_meta.dev() == meta.dev() {
            return Ok(entry.path());
        }
    }

    Err(std::io::Error::new(std::io::ErrorKind::NotFound, "Path not found"))
}

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would need to walk all files in the directory, which is costly for large directories, so I'd like to avoid it if possible. What does the current implementation error on OpenBSD exactly? Perhaps we can find a fix.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I must say, is a bit challenge in OpenBSD.
OpenBSD prioritizes correctness and security over raw speed.
On OpenBSD there is unfortunately no syscall equivalent to F_GETPATH or /proc/self/fd,
that allows recovering the canonical on-disk path without following symlinks.
What do you think about this alternative:

#[cfg(target_os = "openbsd")]
fn final_path(path: &Path) -> io::Result<PathBuf> {
    use std::{fs, os::unix::fs::DirEntryExt, os::unix::fs::MetadataExt};

    let meta = fs::symlink_metadata(path)?;
    let parent = path.parent()
        .ok_or_else(|| io::Error::other("Cannot get parent directory"))?;
    let target = path.file_name();

    let mut fallback = None;

    for entry in fs::read_dir(parent)? {
        let entry = entry?;

        // inode match = same file, no stat() needed
        if entry.ino() != meta.ino() {
            continue;
        }

        let entry_path = entry.path();

        // exact match → done (fast exit)
        if target == entry_path.file_name() {
            return Ok(entry_path);
        }

        // remember first inode match for case-insensitive FS
        if fallback.is_none() {
            fallback = Some(entry_path);
        }
    }

    fallback.ok_or_else(|| io::Error::new(io::ErrorKind::NotFound, "Path not found"))
}

The implementation therefore falls back to scanning the parent directory only when necessary. To minimize cost:

  • It compares inodes using DirEntryExt::ino() to avoid stat() calls.
  • It exits early on an exact filename match.
  • It only walks the parent directory, never recursively.

This appears to be the minimal correct solution given the APIs OpenBSD exposes today.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Traversing parent directories and comparing file inodes is not as "correct" or "safe" as using F_GETPATH or /proc/self/fd, because multiple different filenames can share the same inode.

For example, if abc has inode X and Abc is a hard link to abc, it will also have inode X. This becomes problematic on case-sensitive filesystems because we always perform case-insensitive filename comparisons and that's also why the Linux implementation first tries to get the path via /proc/self/fd which covers most cases without having to fall back to walking parent directories.

Is there a way on OpenBSD to detect whether the filesystem is case-sensitive, and in that case return the path directly to avoid the problems caused by case-insensitive fuzzy matching of inode and filename?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification. On OpenBSD, native FFS filesystems as far as I know, are always case-sensitive, so in most cases the exact filename exists and the fallback scan is safe.

Since there is no equivalent of Linux’s /proc/self/fd or macOS F_GETPATH on OpenBSD, the implementation first tries the fast path (symlink_metadata) and only scans the parent directory if necessary. The scan uses inode comparison to minimize syscalls, and the first inode match is returned as a fallback, which covers the rare case of foreign or case-insensitive filesystems.

So while it isn’t as “correct” as /proc/self/fd, it is the best practical approach for OpenBSD.
I tried, within my technical habilities to make some improvements as you requested:

#[cfg(target_os = "openbsd")]
fn final_path(path: &std::path::Path) -> std::io::Result<std::path::PathBuf> {
    use std::{fs, os::unix::fs::DirEntryExt, os::unix::fs::MetadataExt};

    // 1. Fast path: exact spelling exists
    if fs::symlink_metadata(path).is_ok() {
        return Ok(path.to_path_buf());
    }

    // 2. Fallback: directory scan
    // On OpenBSD, unlike Linux's /proc/self/fd or macOS's F_GETPATH,
    // there is no API to get the exact cased path without scanning.
    // For most cases on FFS (case-sensitive), the first inode match will
    // be the same file, so this is usually safe. On rare case-insensitive
    // or foreign filesystems, this handles the best-effort matching.
    let meta = fs::symlink_metadata(path)?;
    let parent = path.parent()
        .ok_or_else(|| std::io::Error::other("Cannot get parent directory"))?;
    let target = path.file_name();

    let mut fallback = None;

    for entry in fs::read_dir(parent)? {
        let entry = entry?;

        // inode match = same file, no stat() needed
        if entry.ino() != meta.ino() {
            continue;
        }

        let entry_path = entry.path();

        // exact filename match → preferred
        if target == entry_path.file_name() {
            return Ok(entry_path);
        }

        // remember first inode match as fallback for rare case-insensitive FS
        if fallback.is_none() {
            fallback = Some(entry_path);
        }
    }

    fallback.ok_or_else(|| {
        std::io::Error::new(std::io::ErrorKind::NotFound, "Path not found")
    })
}

Let me know if that approach aligns with what you would recommend.
And I hope to have answer your questions clearly.
I am trying to follow, my sincere apologies.

self.inner.pre_exec(move || {
let rlp = libc::rlimit { rlim_cur: max as _, rlim_max: max as _ };
#[cfg(target_os = "openbsd")]
libc::setrlimit(libc::RLIMIT_DATA, &rlp);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason for using RLIMIT_DATA instead of RLIMIT_AS?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RLIMIT_AS on OpenBSD is not supported.
OpenBSD intentionally avoids relying on global address-space limits for security and correctness reasons.
Instead, OpenBSD favors:
- Per-subsystem limits
- Conservative enforcement
- Predictable failure modes

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure whether using RLIMIT_DATA to limit the memory usage of spawned child processes makes sense - is this a common practice on OpenBSD? Or is there an OpenBSD-specific API that should be used for this purpose?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't answer correctly your question, my knowledge is "dumb by default" in some specifics,
but if I try to build with RLIMIT_AS it fails, because undeed it's not supported.
If there is an API I don't know, but I don't think OpenBSD has that, they are
very rigid with the code.

Cargo.toml Outdated
[profile.release]
codegen-units = 1
lto = true
lto = false
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be changed upstream as we (as downstream) need to have lto enabled for better optimized code

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you're right, in OpenBSD lld is intentionally conservative, the best I could do it it was using lto = "thin",
I supposed the alternative is some exception for OpenBSD, I don't know.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you revert this change?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reverted lto to true.
Although, to build it successufully in OpenBSD,
the person would have to change it, at least to "thin".

@sxyazi
Copy link
Owner

sxyazi commented Jan 19, 2026

Hi, I've fixed a build issue on NetBSD in #3506, could you try to see if it also fixes the OpenBSD problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants