diff --git a/contrib/linux-kernel/README.md b/contrib/linux-kernel/README.md index a6283851..16bc2d48 100644 --- a/contrib/linux-kernel/README.md +++ b/contrib/linux-kernel/README.md @@ -1,6 +1,6 @@ # Linux Kernel Patch -There are two pieces, the `zstd_compress` and `zstd_decompress` kernel modules, and the BtrFS patch. +There are three pieces, the `zstd_compress` and `zstd_decompress` kernel modules, the BtrFS patch, and the SquashFS patch. The patches are based off of the linux kernel master branch (version 4.10). ## Zstd Kernel modules @@ -22,14 +22,15 @@ The patches are based off of the linux kernel master branch (version 4.10). * The patch is located in `btrfs.diff`. * Additionally `fs/btrfs/zstd.c` is provided as a source for convenience. -* The patch seems to be working, it doesn't crash the kernel, and compresses at speeds and ratios athat are expected. - It can still use some more testing for fringe features, like printing options. +* The patch seems to be working, it doesn't crash the kernel, and compresses at speeds and ratios that are expected. + It could still use some more testing for fringe features, like printing options. ### Benchmarks Benchmarks run on a Ubuntu 14.04 with 2 cores and 4 GiB of RAM. The VM is running on a Macbook Pro with a 3.1 GHz Intel Core i7 processor, 16 GB of ram, and a SSD. +The kernel running was built from the master branch with the patch (version 4.10). The compression benchmark is copying 10 copies of the unzipped [silesia corpus](http://mattmahoney.net/dc/silesia.html) into a BtrFS @@ -50,3 +51,39 @@ See `btrfs-benchmark.sh` for details. | zstd 9 | 2.92 | 43 MB/s | 406 MB/s | | zstd 12 | 2.93 | 21 MB/s | 408 MB/s | | zstd 15 | 3.01 | 11 MB/s | 354 MB/s | + + +## SquashFS + +* The patch is located in `squashfs.diff` +* Additionally `fs/squashfs/zstd_wrapper.c` is provided as a source for convenience. +* The patch has been tested on a 4.10 kernel. + +### Benchmarks + +Benchmarks run on a Ubuntu 14.04 with 2 cores and 4 GiB of RAM. +The VM is running on a Macbook Pro with a 3.1 GHz Intel Core i7 processor, +16 GB of ram, and a SSD. +The kernel running was built from the master branch with the patch (version 4.10). + +The compression benchmark is the file tree from the SquashFS archive found in the +Ubuntu 16.10 desktop image (ubuntu-16.10-desktop-amd64.iso). +The compression benchmark uses mksquashfs with the default block size (128 KB) +and various compression algorithms/compression levels. +`xz` and `zstd` are also benchmarked with 256 KB blocks. +The decompression benchmark is timing how long it takes to `tar` the file tree +into `/dev/null`. +See `squashfs-benchmark.sh` for details. + +| Algorithm | Compression ratio | Compression speed | Decompression speed | +|----------------|-------------------|-------------------|---------------------| +| gzip | 2.92 | 15 MB/s | 128 MB/s | +| lzo | 2.64 | 9.5 MB/s | 217 MB/s | +| lz4 | 2.12 | 94 MB/s | 218 MB/s | +| xz | 3.43 | 5.5 MB/s | 35 MB/s | +| xz 256 KB | 3.53 | 5.4 MB/s | 40 MB/s | +| zstd 1 | 2.71 | 96 MB/s | 210 MB/s | +| zstd 5 | 2.93 | 69 MB/s | 198 MB/s | +| zstd 10 | 3.01 | 41 MB/s | 225 MB/s | +| zstd 15 | 3.13 | 11.4 MB/s | 224 MB/s | +| zstd 16 256 KB | 3.24 | 8.1 MB/s | 210 MB/s | diff --git a/contrib/linux-kernel/squashfs-benchmark.sh b/contrib/linux-kernel/squashfs-benchmark.sh new file mode 100755 index 00000000..02dfd732 --- /dev/null +++ b/contrib/linux-kernel/squashfs-benchmark.sh @@ -0,0 +1,39 @@ +# !/bin/sh +set -e + +# Benchmarks run on a Ubuntu 14.04 VM with 2 cores and 4 GiB of RAM. +# The VM is running on a Macbook Pro with a 3.1 GHz Intel Core i7 processor and +# 16 GB of RAM and an SSD. + +# $BENCHMARK_DIR is generated with the following commands, from the Ubuntu image +# ubuntu-16.10-desktop-amd64.iso. +# > mkdir mnt +# > sudo mount -o loop ubuntu-16.10-desktop-amd64.iso mnt +# > cp mnt/casper/filesystem.squashfs . +# > sudo unsquashfs filesystem.squashfs + +# $HOME is on a ext4 filesystem +BENCHMARK_DIR="$HOME/squashfs-root/" +BENCHMARK_FS="$HOME/filesystem.squashfs" + +# Normalize the environment +sudo rm -f $BENCHMARK_FS 2> /dev/null > /dev/null || true +sudo umount /mnt/squashfs 2> /dev/null > /dev/null || true + +# Run the benchmark +echo "Compression" +echo "sudo mksquashfs $BENCHMARK_DIR $BENCHMARK_FS $@" +time sudo mksquashfs $BENCHMARK_DIR $BENCHMARK_FS $@ 2> /dev/null > /dev/null + +echo "Approximate compression ratio" +printf "%d / %d\n" \ + $(sudo du -sx --block-size=1 $BENCHMARK_DIR | cut -f1) \ + $(sudo du -sx --block-size=1 $BENCHMARK_FS | cut -f1); + +# Mount the filesystem +sudo mount -t squashfs $BENCHMARK_FS /mnt/squashfs + +echo "Decompression" +time sudo tar -c /mnt/squashfs 2> /dev/null | wc -c > /dev/null + +sudo umount /mnt/squashfs