github.com/cyphar/filepath-securejoin v0.5.0 -- "Let the past die. Ki…

…ll it if you have to."

========================================================================
NOTE: With this release, some parts of filepath-securejoin are now
      licensed under the Mozilla Public License (version 2). Please see
      COPYING.md as well as the the license header in each file for more
      details.
========================================================================

Breaking:

- The new API introduced in the 0.3.0 release has been moved to a new
  subpackage called "pathrs-lite". This was primarily done to better
  indicate the split between the new and old APIs, as well as indicate
  to users the purpose of this subpackage (it is a less complete version
  of libpathrs).

  We have added some wrappers to the top-level package to ease the
  transition, but those are deprecated and will be removed in the next
  minor release of filepath-securejoin. Users should update their import
  paths.

  This new subpackage has also been relicensed under the Mozilla Public
  License (version 2), please see COPYING.md for more details.

Added:

- Most of the key bits the safe procfs API have now been exported and
  are available in .../filepath-securejoin/pathrs-lite/procfs. At the
  moment this primarily consists of a new procfs.Handle API:

   * OpenProcRoot returns a new handle to /proc, endeavouring to make it
     safe if possible ("subset=pid" to protect against mistaken write
     attacks and leaks, as well as using fsopen(2) to avoid racing mount
     attacks).

     OpenUnsafeProcRoot returns a handle without attempting to create
     one with "subset=pid", which makes it more dangerous to leak. Most
     users should use OpenProcRoot (even if you need to use ProcRoot as
     the base of an operation, as filepath-securejoin will internally
     open a temporary OpenUnsafeProcRoot handle when necessary).

   * The (*procfs.Handle).Open* family of methods lets you get a safe
     O_PATH handle to subpaths within /proc for certain subpaths.

     For OpenThreadSelf, the returned ProcThreadSelfCloser needs to be
     called after you completely finish using the handle (this is
     necessary because Go is multi-threaded and ProcThreadSelf
     references /proc/thread-self which may disappear if we do not
     runtime.LockOSThread -- ProcThreadSelfCloser is currently
     equivalent to runtime.UnlockOSThread).

     Note that you cannot open any procfs symlinks (most notably
     magic-links) using this API. At the moment, filepath-securejoin
     does not support this feature (but libpathrs does).

   * ProcSelfFdReadlink lets you get the in-kernel path representation
     of a file descriptor (think readlink("/proc/self/fd/...")), except
     that we verify that there aren't any tricky overmounts that could
     fool the process.

     Please be aware that the returned string is simply a snapshot at
     that particular moment, and an attacker could move the file being
     pointed to. In addition, complex namespace configurations could
     result in non-sensical or confusing paths to be returned. The value
     received from this function should only be used as secondary
     verification of some security property, not as proof that a
     particular handle has a particular path.

  The procfs handle used internally by the API is the same as the rest
  of filepath-securejoin (for privileged programs this is usually a
  private in-process procfs instance created with fsopen(2)).

  As before, this is intended as a stop-gap before users migrate to
  libpathrs, which provides a far more extensive safe procfs API and is
  generally more robust.

- Previously, the hardened procfs implementation (used internally within
  Reopen and Open(at)InRoot) only protected against overmount attacks on
  systems with openat2(2) (Linux 5.6) or systems with fsopen(2) or
  open_tree(2) (Linux 5.2) and programs with privileges to use them
  (with some caveats about locked mounts that probably affect very few
  users). For other users, an attacker with the ability to create
  malicious mounts (on most systems, a sysadmin) could trick you into
  operating on files you didn't expect. This attack only really makes
  sense in the context of container runtime implementations.

  This was considered a reasonable trade-off, as the long-term intention
  was to get all users to just switch to libpathrs if they wanted to use
  the safe procfs API (which had more extensive protections, and is what
  these new protections in filepath-securejoin are based on). However,
  as the API is now being exported it seems unwise to advertise the API
  as "safe" if we do not protect against known attacks.

  The procfs API is now more protected against attackers on systems
  lacking the aforementioned protections. However, the most
  comprehensive of these protections effectively rely on
  statx(STATX_MNT_ID) (Linux 5.8). On older kernel versions, there is no
  effective protection (there is some minimal protection against
  non-procfs filesystem components but a sufficiently clever attacker
  can work around those). In addition, STATX_MNT_ID is vulnerable to
  mount ID reuse attacks by sufficiently motivated and privileged
  attackers -- this problem is mitigated with STATX_MNT_ID_UNIQUE (Linux
  6.8) but that raises the minimum kernel version for more protection.

  The fact that these protections are quite limited despite needing a
  fair bit of extra code to handle was one of the primary reasons we did
  not initially implement this in filepath-securejoin (libpathrs
  supports all of this, of course).

Fixed:

- RHEL 8 kernels have backports of fsopen(2) but in some testing we've
  found that it has very bad (and very difficult to debug) performance
  issues, and so we will explicitly refuse to use fsopen(2) if the
  running kernel version is pre-5.2 and will instead fallback to
  open("/proc").

Thanks to the following contributors who made this release possible:

 * Aleksa Sarai <[email protected]>
 * Kir Kolyshkin <[email protected]>
 * Stephen Kitt <[email protected]>

Signed-off-by: Aleksa Sarai <[email protected]>

github.com/cyphar/filepath-securejoin v0.4.1

This release fixes a regression introduced in one of the hardening
features added to filepath-securejoin 0.4.0.

- The restrictions added for root paths passed to SecureJoin in 0.4.0
  was found to be too strict and caused some regressions when folks
  tried to update, so this restriction has been relaxed to only return
  an error if the path contains a ".." component. We still recommend
  users use filepath.Clean (and even filepath.EvalSymlinks) on the root
  path they are using, but at least you will no longer be punished for
  "trivial" unclean paths.

Signed-off-by: Aleksa Sarai <[email protected]>

github.com/cyphar/filepath-securejoin v0.4.0

This release primarily includes a few minor breaking changes to make the
MkdirAll and SecureJoin interfaces more robust against accidental
misuse.

- SecureJoin(VFS) will now return an error if the provided root is not a
  filepath.Clean'd path.

  While it is ultimately the responsibility of the caller to ensure the
  root is a safe path to use, passing a path like /symlink/.. as a root
  would result in the SecureJoin'd path being placed in / even though
  /symlink/.. might be a different directory, and so we should more
  strongly discourage such usage.

  All major users of securejoin.SecureJoin already ensure that the paths
  they provide are safe (and this is ultimately a question of user
  error), but removing this foot-gun is probably a good idea. Of course,
  this is necessarily a breaking API change (though we expect no real
  users to be affected by it).

  Thanks to Erik Sjölund, who initially reported this issue as a
  possible security issue.

- MkdirAll and MkdirHandle now take an os.FileMode-style mode argument
  instead of a raw unix.S_*-style mode argument, which may cause
  compile-time type errors depending on how you use filepath-securejoin.
  For most users, there will be no change in behaviour aside from the
  type change (as the bottom 0o777 bits are the same in both formats,
  and most users are probably only using those bits).

  However, if you were using unix.S_ISVTX to set the sticky bit with
  MkdirAll(Handle) you will need to switch to os.ModeSticky otherwise
  you will get a runtime error with this update. In addition, the error
  message you will get from passing unix.S_ISUID and unix.S_ISGID will
  be different as they are treated as invalid bits now (note that
  previously passing said bits was also an error).

Thanks to the following contributors for helping make this release
possible:

 * Aleksa Sarai <[email protected]>
 * Erik Sjölund <[email protected]>

Signed-off-by: Aleksa Sarai <[email protected]>

github.com/cyphar/filepath-securejoin v0.3.6

This release lowers the minimum Go version to Go 1.18 as well as some
library dependencies, in order to make it easier for folks that need to
backport patches using the new filepath-securejoin API onto branches
that are stuck using old Go compilers. For users using Go >= 1.21, this
release contains no functional changes.

* The minimum Go version requirement for filepath-securejoin is now Go 1.18
  (we use generics internally).

  For reference, [email protected] somewhat-arbitrarily bumped the
  Go version requirement to 1.21.

  While we did make some use of Go 1.21 stdlib features (and in principle Go
  versions <= 1.21 are no longer even supported by upstream anymore), some
  downstreams have complained that the version bump has meant that they have to
  do workarounds when backporting fixes that use the new filepath-securejoin
  API onto old branches. This is not an ideal situation, but since using this
  library is probably better for most downstreams than a hand-rolled
  workaround, we now have compatibility shims that allow us to build on older
  Go versions.

* Lower minimum version requirement for golang.org/x/sys to v0.18.0 (we
  need the wrappers for fsconfig(2)), which should also make backporting
  patches to older branches easier.

Signed-off-by: Aleksa Sarai <[email protected]>

github.com/cyphar/filepath-securejoin v0.3.5

This release primarily includes a fix for an issue involving two
programs racing to MkdirAll the same directory, which caused a
regression with BuildKit.

- `MkdirAll` will now no longer return an `EEXIST` error if two racing
  processes are creating the same directory. We will still verify that the path
  is a directory, but this will avoid spurious errors when multiple threads or
  programs are trying to `MkdirAll` the same path. opencontainers/runc#4543

Signed-off-by: Aleksa Sarai <[email protected]>

github.com/cyphar/filepath-securejoin v0.3.4

This release primarily includes a fix that blocked using
filepath-securejoin in Kubernetes.

- Previously, some testing mocks we had resulted in us doing import "testing"
  in non-_test.go code, which made some downstreams like Kubernetes unhappy.
  This has been fixed. (#32)

Thanks to all of the contributors who made this release possible:

 * Aleksa Sarai <[email protected]>
 * Stephen Kitt <[email protected]>

Signed-off-by: Aleksa Sarai <[email protected]>

github.com/cyphar/filepath-securejoin v0.3.3

This release primarily includes fixes for spurious errors we hit when
checking that directories created by MkdirAll "look right". Upon further
consideration, these checks were fundamentally buggy and didn't offer
any practical protection anyway.

- The mode and owner verification logic in `MkdirAll` has been removed. This
  was originally intended to protect against some theoretical attacks but upon
  further consideration these protections don't actually buy us anything and
  they were causing spurious errors with more complicated filesystem setups.
- The "is the created directory empty" logic in `MkdirAll` has also been
  removed. This was not causing us issues yet, but some pseudofilesystems (such
  as `cgroup`) create non-empty directories and so this logic would've been
  wrong for such cases.

Thanks to all of the contributors who made this release possible:

 * Aleksa Sarai <[email protected]>
 * Kir Kolyshkin <[email protected]>

Signed-off-by: Aleksa Sarai <[email protected]>

github.com/cyphar/filepath-securejoin v0.3.2

This release includes a few fixes for MkdirAll when dealing with S_ISUID
and S_ISGID, to solve a regression runc hit when switching to MkdirAll.

* Passing the S_ISUID or S_ISGID modes to MkdirAllInRoot will now return
  an explicit error saying that those bits are ignored by mkdirat(2). In
  the past a different error was returned, but since the silent ignoring
  behaviour is codified in the man pages a more explicit error seems
  apt. While silently ignoring these bits would be the most compatible
  option, it could lead to users thinking their code sets these bits
  when it doesn't. Programs that need to deal with compatibility can
  mask the bits themselves.

* If a directory has S_ISGID set, then all child directories will have
  S_ISGID set when created and a different gid will be used for any
  inode created under the directory. Previously, the "expected owner and
  mode" validation in securejoin.MkdirAll did not correctly handle this.
  We now correctly handle this case.

Signed-off-by: Aleksa Sarai <[email protected]>

github.com/cyphar/filepath-securejoin v0.3.1

* By allowing Open(at)InRoot to opt-out of the extra work done by
  MkdirAll to do the necessary "partial lookups", Open(at)InRoot now
  does less work for both implementations (resulting in a many-fold
  decrease in the number of operations for openat2, and a modest
  improvement for non-openat2) and is far more guaranteed to match the
  correct openat2(RESOLVE_IN_ROOT) behaviour.

* We now use readlinkat(fd, "") where possible. For Open(at)InRoot this
  effectively just means that we no longer risk getting spurious errors
  during rename races. However, for our hardened procfs handler, this in
  theory should prevent mount attacks from tricking us when doing
  magic-link readlinks (even when using the unsafe host /proc handle).
  Unfortunately Reopen is still potentially vulnerable to those kinds of
  somewhat-esoteric attacks.

  Technically this will only work on post-2.6.39 kernels[1] but it seems
  incredibly unlikely anyone is using filepath-securejoin on a pre-2011
  kernel.

* Several improvements were made to the errors returned by
  Open(at)InRoot and MkdirAll when dealing with invalid paths under the
  emulated (ie. non-openat2) implementation. Previously, some paths
  would return the wrong error (ENOENT when the last component was a
  non-directory), and other paths would be returned as though they were
  acceptable (trailing-slash components after a non-directory would be
  ignored by Open(at)InRoot).

  These changes were done to match openat2's behaviour and purely is a
  consistency fix (most users are going to be using openat2 anyway).

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=65cfc6722361570bfe255698d9cd4dccaf47570d

Signed-off-by: Aleksa Sarai <[email protected]>

This release contains no changes to SecureJoin.

However, it does introduce a new *os.File-based API which is much safer
to use for most usecases. These are adapted from libpathrs[1] and are
the bare minimum to be able to operate more safely on an untrusted
rootfs where an attacker has write access (something that SecureJoin
cannot protect against). The new APIs are:

 * OpenInRoot, which resolves a path inside a rootfs and returns an
   *os.File handle to the path. Note that the file handle returned by
   OpenInRoot is an O_PATH handle, which cannot be used for reading or
   writing (as well as some other operations -- see open(2) for more
   details).

 * Reopen, which takes an O_PATH file handle and safely re-opens it to
   "upgrade" it to a regular handle.

 * MkdirAll, which is a safe implementation of os.MkdirAll that can be
   used to create directory trees inside a rootfs.

As these are new APIs, it is possible they may change in the future.
However, they should be safe to start migrating to as we have extensive
tests ensuring they behave correctly and are safe against various races
and other attacks.

[1]: https://github.com/openSUSE/libpathrs

Signed-off-by: Aleksa Sarai <[email protected]>