Bug fixes

• The new Windows code from the v0.14.0 release that implemented lstat()-like functionality to get file status information did not work correctly for reparse points that were not symbolic links (i.e. "real" symlinks or junctions). When dealing with such reparse points, the code would incorrectly determine the allocated size of the file to be zero, since it tried to determine that size from the reparse point and not from the target. This caused the progress percentage to be calculated incorrectly, since much more data was processed than was expected, leading to progress percentages well over 100%. Thankfully, @lunighty not only reported this issue, but also provided significant context by mentioning that this was on a Windows Server NTFS volume with deduplication enabled. Fortunately, this was also reproducible on "regular" Windows using a mounted WIM image. The fix will now correctly handle reparse points by re-opening the handle without FILE_FLAG_OPEN_REPARSE_POINT in order to open the target. This bug only affected Windows builds of mkdwarfs, and only when actually used on filesystems with reparse points that were not symlinks. Even then, the resulting DwarFS image would still be correct, except for a way too small total_allocated_fs_size metadata entry. However, this entry is purely informational. Huge thanks to @lunighty for both reporting and helping to debug this issue! Fixes github #307.

• Related to the above, the metadata_builder now recomputes all total sizes (total_fs_size, total_allocated_fs_size, and total_hardlink_size) as part of the build() function. This not only ensures that the totals are correct even if the allocated size changes between scanning and segmenting (which has been happening at least on ZFS volumes), but it also allows images affected by the above bug to be fixed by rebuilding the metadata.

• Instead of making the FUSE drivers fail hard when seeing the options that were removed in v0.14.0, they now just log a warning and ignore them. The options may still be fully removed in a future release. Fixes github #303.

• The pcmaudio categorizer had two minor issues discovered by @lunighty when compressing a large number of WAV files. One was reporting an unsupported format: 3/0 or unsupported format: 65,534/3 warning, which isn't very useful for the end user. These format codes correspond to IEEE floating point formats, which are indeed unsupported. However, the format appears to be quite common, so the warning has been downgraded to an info message that explicitly mentions the floating point format. The second issue was an unexpected fmt chunk size of 20 bytes, which caused the file to be rejected as a PCM audio file (meaning it was added using a generic compressor instead of FLAC). It turns out that these non-conforming fmt chunks are also quite common in practice, so the code has been changed to accept the non-conforming file, but also logging an info message mentioning the non-conformance. Fixes github #309.

• A bunch of changes were made to make the static (Linux) release binaries work on FreeBSD with the Linux emulation layer. While you can easily build DwarFS on FreeBSD natively, the static binaries might be bundled in Linux application images that you may want to run on FreeBSD as well. This required patching musl to fall back to FUTEX_WAKE if FUTEX_REQUEUE is not supported (which is the case on FreeBSD's Linux emulation); otherwise, the binaries would regularly hang when trying to shut down thread pools. Furthermore, the SFX stub did not work with the emulation layer due to lack of support for fexecve; so it now uses a fallback of temporary file + execve in case create_memfd and/or fexecve are not supported. This fallback is very similar to the one that was already in use for old Linux kernels, only that cleanup of the temporary file is now done using a child "janitor" process as soon as the execve call succeeds. Lastly, all static binaries are now branded as using the Linux ABI rather than the generic "System V" in the ELF header, which makes it unnecessary to explicitly brandelf the binaries on FreeBSD. See VHSgunzo/uruntime#21 for more details.

• The test_helpers library used for unit tests was not explicitly listing xxhash as a dependency, which caused linker errors on macOS. These unfortunately only showed up in the Homebrew CI. The dependency has been added.

• The help text for the mkdwarfs compress level option (-l) was misleading in combination with the manual page as neither mentioned that the table with details was shown only by -H / --long-help. Fixes github #312.

Features

• Added shell completion for dwarfsck and dwarfsextract. (Thanks to Ahmad Khalifa for the contribution.)

• Added sample desktop unmount handlers. (Thanks to Ahmad Khalifa for the contribution.)

Documentation

• Updated diagram in dwarfs-format.md to show new inode metadata fields.

Build

• Bumped libarchive, libressl, and libdwarf versions in the static build pipeline.

Test

• Added test for file_stat on symbolic links containing multibyte characters on Windows.

• Added test for obsolete FUSE driver options being ignored and producing a warning.

• Added test for the FUSE driver's imagesize option.

• Added more tests for the --auto-mountpoint option in the FUSE driver.

Full Changelog: v0.14.0...v0.14.1

8c17434c4fc8c33ba303e06eb62904047ee1aab3ea7bd98d2797b04d86b426eb  dwarfs-0.14.1-Linux-aarch64.tar.xz
98cdca2e9406cbb88093bd2ea03cbd8a62b7a3aa5ba0271d22934fe4e81128ac  dwarfs-0.14.1-Linux-arm.tar.xz
531b2c318a810920d4f11f0efac12cec06627068d91291232000a883108952ab  dwarfs-0.14.1-Linux-i386.tar.xz
93ec3d9d649cdb77eed3d1b5c6533ee05e24589be63cca3b0a9c32c18c55ef1a  dwarfs-0.14.1-Linux-loongarch64.tar.xz
813d021f3025968dc7fa0797553b55bd7a5ed1c85af1f6b2c36b2ad5897294c8  dwarfs-0.14.1-Linux-ppc64le.tar.xz
1437bd811bfcf0f9d7db080075906086625ab1465ed9a068bb881da1c8f68681  dwarfs-0.14.1-Linux-ppc64.tar.xz
0a6e1e66b0b0ce63acdc37a8a68ce173d37b135cfad9ca652ca2827e4789d3ec  dwarfs-0.14.1-Linux-riscv64.tar.xz
9f20a528d9757ae7825341438c7d38eb3c91953c34c8d6bb6ace5cc4fe3d2f52  dwarfs-0.14.1-Linux-s390x.tar.xz
7dba2bc4482defc5987652643ab0d58330b74e73be0f4d2d184879e7815c2476  dwarfs-0.14.1-Linux-x86_64.tar.xz
620cf27f2e142a5f8fc05552a70704c3bf4df23c3279c6026b3f37954d0529c5  dwarfs-0.14.1.tar.xz
66f4d28514eb9cd8a28f634d76f274eb899c461cd54775f74efb6a97239c04ec  dwarfs-0.14.1-Windows-AMD64.7z
24bdd5c6adea556ca3a28f26a3774684da7efe4a5e70a13ed7149edd12b3f738  dwarfs-fuse-extract-0.14.1-Linux-aarch64
9f04547e224aa97c3187474de4d49ee12cfc1a414b9a3864f4338272201278fd  dwarfs-fuse-extract-0.14.1-Linux-aarch64.upx
fa31490ac078cd90be86cb43cf12836ad142f115ff5bf4a01771b6e07b73acab  dwarfs-fuse-extract-0.14.1-Linux-arm
6e39a297c75ae2fbb69b2d6075b32e34b3c4a386d6961e4a85b76ff944ccf30b  dwarfs-fuse-extract-0.14.1-Linux-arm.upx
cd637922614ba20b4614c37e61d0bb5ee703ccfce13ba328f9c07ed2848d9424  dwarfs-fuse-extract-0.14.1-Linux-i386
b6274b71bd695d62de21c9631e6d9f62be8a55512e072f5c6124eaa047e1fdf3  dwarfs-fuse-extract-0.14.1-Linux-i386.upx
6b1b699aba544ef407ebc33003f900fa691685c2be4195ca9690da7d7da4bcef  dwarfs-fuse-extract-0.14.1-Linux-loongarch64
948227dc2f87c37a3c584af61b189cbc04557088530335794f54a8a9a17e113f  dwarfs-fuse-extract-0.14.1-Linux-ppc64
8a60b79fc535afeb3352c24252a69db8b0dae0e176a46cc321c0e4e644eb0e08  dwarfs-fuse-extract-0.14.1-Linux-ppc64le
4cb8a50135ad0d3f93f3750e738fcd0f92751445493665d7316e2384a20ba107  dwarfs-fuse-extract-0.14.1-Linux-riscv64
aebd9c079fdafc11e5669f1a2e49192f19e5c3f78d62f8c138eb28208ed1eb8d  dwarfs-fuse-extract-0.14.1-Linux-s390x
b121f472429bfba72b79f1b5de126e8a70872fedefd4d42aba98fc5fc402ae20  dwarfs-fuse-extract-0.14.1-Linux-x86_64
0ded4f3b5c324cd662babece00bf0434fd98b9c98e6268ea745e160bc96a500b  dwarfs-fuse-extract-0.14.1-Linux-x86_64.upx
3a47eda6d496ba12fb6d9af1b1914d4ba7ab73df8df3172ee3cad66c29822cc1  dwarfs-universal-0.14.1-Linux-aarch64
ad9d6e86c120c2a4e635e4843234d9e046ac186772ad719ba508c6f1051a5f57  dwarfs-universal-0.14.1-Linux-aarch64.upx
958ab2e0ae8b1c65b14090b1f56cf062becc955c8287d31f1f7026adad01ac7a  dwarfs-universal-0.14.1-Linux-arm
33c06c3320e9c9907f0803fba3913e59f39c2b07c2a54722a0c8fbd2ea20f542  dwarfs-universal-0.14.1-Linux-arm.upx
dae5171425af53ed1357d6f63c5fc45d6a61b0c70588c7cd8b5c8eb432080673  dwarfs-universal-0.14.1-Linux-i386
7a073913b6c682816e7dbcddde1bcb8ddca9d9c32f679369df21bb7d54f0291c  dwarfs-universal-0.14.1-Linux-i386.upx
17630a7f41d5974a0747f28052bf0339140b22ce67a66fce443ebcc1bc8594bd  dwarfs-universal-0.14.1-Linux-loongarch64
ca378379612b66093e677d45616f242bd08f5e5e808283f4055530c51048b149  dwarfs-universal-0.14.1-Linux-ppc64
06472149ea534d94eb5799f6a2037edc5ff5f3146546af15b3a9d5b427be9db6  dwarfs-universal-0.14.1-Linux-ppc64le
bf827447162016098ef193c6104264a52aaef9089bab434872631a8bf6ed2ffe  dwarfs-universal-0.14.1-Linux-riscv64
a89e4152096b422bda48cb9ec1a957dd9a4745ec9cd83f2d036d00e5e2dbb2ba  dwarfs-universal-0.14.1-Linux-s390x
f3a117fd6d5b7304944b199af7fdb8086a48c509ea2e9832255d8f9a54c98587  dwarfs-universal-0.14.1-Linux-x86_64
fb4ea4a267acf19a5bf9bf0eb13f3d7d88b5f7687fca2aae7136e21810fde0cd  dwarfs-universal-0.14.1-Linux-x86_64.upx
c59967d05cb97937c642964a258f29e437f68e2f156ff71b7205dd489b0f5c00  dwarfs-universal-0.14.1-Windows-AMD64.exe