121 Commits

Author SHA1 Message Date
Nick Terrell
b10085d97d [contrib][linux-kernel] Add standard warnings and -Werror to CI
Test the kernel build with the standard warnings enabled so that we
don't miss issues like fixed in PR #2802.

Stacked on top of PR #2802 so CI passes, so it includes the fix. But, I
won't merge until it is merged, so @solbjorn gets the credit for the
fix.
2021-09-24 15:08:11 -07:00
Nick Terrell
32a8443b5c
Merge pull request #2790 from solbjorn/huf-asm-kernel
[contrib][linux] Fix build after introducing ASM HUF implementation
2021-09-24 12:42:03 -07:00
Nick Terrell
14772d97be
Merge pull request #2796 from terrelln/linux-fixes
[lib] Make lib compatible with `-Wfall-through` excepting legacy
2021-09-23 16:11:53 -07:00
Nick Terrell
189e87bcbe [lib] Make lib compatible with -Wfall-through excepting legacy
Switch to a macro `ZSTD_FALLTHROUGH;` instead of a comment. On supported
compilers this uses an attribute, otherwise it becomes a comment.

This is necessary to be compatible with clang's `-Wfall-through`, and
gcc's `-Wfall-through=2` which don't support comments. Without this the
linux build emits a bunch of warnings.

Also add a test to CI to ensure that we don't regress.
2021-09-23 10:51:18 -07:00
Alexander Lobakin
c45b27abe7 [contrib][linux] Add huf_decompress_amd64.o target to Makefile
Commit a5f2c4552803 ("Huffman ASM") added a new ASM source file,
but it wasn't added to the kernel Makefile despite that it received
support for Huffman ASM according to the internal definitions. This
leads to undefined references, as huf_decompress.o now calls those
ASM functions.
Add it to the list of sources when building inside the kernel tree.
Kbuild can handle .S files just fine, so none additional rules
needed.

Fixes: a5f2c4552803 ("Huffman ASM")
Signed-off-by: Alexander Lobakin <alobakin@pm.me>
2021-09-23 16:39:31 +02:00
Alexander Lobakin
e6d62bbfe1 [contrib][linux] Fix build with CONFIG_WERROR
Linux 5.15 introduces a new Kconfig option, CONFIG_WERROR, which
forces -Werror for the entire kernel.
Current in-kernel ZSTD implementation uses functions deprecated
in 1.5.0, and thus fails on -Wdeprecated-declarations.
Turn this particular error into warning to be able to build the
kernel with CONFIG_WERROR. I'm not disabling them completely to
make sure they'll be visible and [hopefully] fixed sooner or later.

Signed-off-by: Alexander Lobakin <alobakin@pm.me>
2021-09-23 16:39:10 +02:00
senhuang42
1d8143c84f Move block splitter from stack to CCtx 2021-09-23 00:02:31 -04:00
Nick Terrell
162491f601 [contrib][linux] Reduce stack usage by 80 bytes
Instead of calling `ZSTD_compress_advanced()` and
`ZSTD_initCStream_advanced()`, which each take a `ZSTD_parameters` by
value, use the new advanced API.

Stack usage went from 2024 -> 1944.
2021-09-22 18:18:47 -07:00
Nick Terrell
4d347a902b [contrib][linux] Fix up SPDX license identifiers
Correctly identify that we are GPL v2+ or BSD 3 clause, as pointed out
in issue #2663.
2021-09-22 15:09:42 -07:00
Nick Terrell
a5f2c45528 Huffman ASM 2021-09-20 14:46:43 -07:00
Nick Terrell
8bf699aa59 [build] Add support for ASM files in Make + CMake
* Extract out common portion of `lib/Makefile` into `lib/libzstd.mk`.
  Most relevantly, the way we find library files.
* Use `lib/libzstd.mk` in the other Makefiles instead of repeating the
  same code.
* Add a test `tests/test-variants.sh` that checks that the builds of
  `make -C programs allVariants` are correct, and run it in Actions.
* Adds support for ASM files in the CMake build.

The Meson build is not updated because it lists every file in zstd,
and supports ASM off the bat, so the Huffman ASM commit will just add
the ASM file to the list.

The Visual Studios build is not updated because I'm not adding ASM
support to Visual Studios yet.
2021-09-17 14:13:53 -07:00
sen
698f261b35
[1.5.0] Deprecate some functions (#2582)
* Add deprecated macro to zstd.h, mark certain functions as deprecated

* Remove ZSTD_compress.c dependencies on deprecated functions
2021-05-06 17:59:32 -04:00
Nick Terrell
09149beaf8 [1.5.0] Move zstd_errors.h and zdict.h to lib/ root
`zstd_errors.h` and `zdict.h` are public headers, so they deserve to be
in the root `lib/` directory with `zstd.h`, not mixed in with our private
headers.
2021-04-30 15:13:54 -07:00
Nick Terrell
fbb9006e18 [linux-kernel] Replace kernel-style comments
Replace kernel-style comments with regular comments.

E.g.

```
/** Before */

/* After */

/**
 * Before
 */

/*
 * After
 */

/***********************************
 * Before
 ***********************************/

/* *********************************
 * After
 ***********************************/
```
2021-04-29 15:50:23 -07:00
Nick Terrell
d334ad2ff4 [contrib][linux-kernel] Add zstd_min_clevel() and zstd_max_clevel() 2021-03-30 10:30:59 -07:00
Nick Terrell
a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:30:43 -07:00
Nick Terrell
cd1551d261 [lib][tracing] Add ZSTD_NO_TRACE macro
When defined, it disables tracing, and avoids including the header.
2021-03-16 11:47:27 -07:00
Nick Terrell
e4b914e663 [contrib][linux] Expose zstd headers to avoid duplication
Expose the zstd headers in `include/linux` to avoid struct duplication.
This makes the member names not follow Kernel style guidelines, and
exposes the zstd symbols. But, the LMKL reviewers are okay with that.
2021-03-16 11:47:22 -07:00
Nick Terrell
49a9e070f5 [contrib][linux-kernel] Update test include stubs
Update the test include stubs so they are able to run the current zstd
version in the kernel, so I can compare stack usage.
2021-03-16 11:40:24 -07:00
Nick Terrell
d2dd35ae65 [contrib][linux-kernel] Fix unaligned.h
Fix the `unaligned.h` shim in the tests that was causing corruption in
the tests. Note that this is a problem with the test shim, not the
kernel code.
2021-03-16 11:40:24 -07:00
Nick Terrell
54a4998a80 Add basic tracing functionality 2021-02-05 16:28:52 -08:00
Nick Terrell
66e811d782 [license] Update year to 2021 2021-01-04 17:53:52 -05:00
Nick Terrell
10f87d2f43 [contrib][linux] Add SPDX-License-Identifier 2020-12-02 12:00:44 -08:00
Nick Terrell
d0cc2e3868 [contrib][linux] Fix make import-upstream 2020-12-02 11:43:28 -08:00
Nick Terrell
3256c00c6e [contrib][linux] Follow style guide more closely 2020-12-02 11:07:38 -08:00
Nick Terrell
2c570e7f87 [contrib][linux] Add a make target to import upstream 2020-12-02 10:42:33 -08:00
Nick Terrell
87b38156be [contrib][linux-kernel] Fix test warnings 2020-12-02 01:12:42 -08:00
Nick Terrell
f58e9a9830 [contrib][linux] Add wrapper API
Add the kernel wrapper API. This keeps the same API and semantics as the
existing kernel API with name changes to be more kernel style and avoid
symbol collisions with zstd.
2020-12-02 01:08:08 -08:00
Nick Terrell
6a1e526ea7 [lib] Add ZSTD_COMPRESS_HEAPMODE tuning parameter 2020-09-24 19:42:04 -07:00
Nick Terrell
caecd8c211 Allow user to override ASAN/MSAN detection
Rename ADDRESS_SANITIZER -> ZSTD_ADDRESS_SANITIZER and same for
MEMORY_SANITIZER. Also set it to 0/1 instead of checking for defined.
This allows the user to override ASAN/MSAN detection for platforms that
don't support it.
2020-09-24 19:42:04 -07:00
Nick Terrell
683150e59f [linux-kernel] Avoid including <string.h> in the tests 2020-09-24 19:42:04 -07:00
Nick Terrell
9ae0483858 Reorganize zstd_deps.h and mem.h + replace mem.h for the kernel 2020-09-24 19:41:59 -07:00
Nick Terrell
9009b59f09 [linux-kernel] Add missing semicolon in zstd_deps.h 2020-09-23 23:26:10 -07:00
Nick Terrell
a70fa342f0 [CI][linux-kernel] Enable -Wunused-const-variable and -Wunused-but-set-variable 2020-09-23 12:59:57 -07:00
Nick Terrell
978659d34f [linux-kernel] Fix unused variable warnings with malloc, calloc, and free 2020-09-23 12:59:57 -07:00
Nick Terrell
5651fe9b49 [contrib][linux-kernel] Add decompress_sources.h
Add decompress_sources.h, which includes all the decompression .c files.
This is used for kernel decompression.

Also, add a test which checks that including decompress_sources.h works.
2020-09-14 12:49:23 -07:00
Nick Terrell
3420c202af [linux-kernel] Update README with initial instructions on how to update zstd in the kernel 2020-09-09 14:36:22 -07:00
Nick Terrell
aab4bf7b0d [linux-kernel] Add test that checks the ifdef hardwiring 2020-09-09 14:36:19 -07:00
Nick Terrell
29c5de8780 Add linux-kernel freestanding 2020-09-09 14:35:57 -07:00
Bimba Shrestha
43da5bf27e Rearranging assert and allowing 4 extra for FSE_BLOCKBOUND() 2019-09-12 14:43:50 -07:00
Josh Soref
a880ca239b Spelling (#1582)
* spelling: accidentally

* spelling: across

* spelling: additionally

* spelling: addresses

* spelling: appropriate

* spelling: assumed

* spelling: available

* spelling: builder

* spelling: capacity

* spelling: compiler

* spelling: compressibility

* spelling: compressor

* spelling: compression

* spelling: contract

* spelling: convenience

* spelling: decompress

* spelling: description

* spelling: deflate

* spelling: deterministically

* spelling: dictionary

* spelling: display

* spelling: eliminate

* spelling: preemptively

* spelling: exclude

* spelling: failure

* spelling: independence

* spelling: independent

* spelling: intentionally

* spelling: matching

* spelling: maximum

* spelling: meaning

* spelling: mishandled

* spelling: memory

* spelling: occasionally

* spelling: occurrence

* spelling: official

* spelling: offsets

* spelling: original

* spelling: output

* spelling: overflow

* spelling: overridden

* spelling: parameter

* spelling: performance

* spelling: probability

* spelling: receives

* spelling: redundant

* spelling: recompression

* spelling: resources

* spelling: sanity

* spelling: segment

* spelling: series

* spelling: specified

* spelling: specify

* spelling: subtracted

* spelling: successful

* spelling: return

* spelling: translation

* spelling: update

* spelling: unrelated

* spelling: useless

* spelling: variables

* spelling: variety

* spelling: verbatim

* spelling: verification

* spelling: visited

* spelling: warming

* spelling: workers

* spelling: with
2019-04-12 11:18:11 -07:00
Nick Terrell
f9252d8347 [linux-kernel] Update license 2017-08-31 12:48:36 -07:00
Nick Terrell
57e2df6651 [kernel] Update squashfs-tools patch 2017-08-14 22:43:36 -07:00
Nick Terrell
8b6702a00d [linux-kernel] Update patches for v5 2017-08-09 13:03:40 -07:00
Nick Terrell
7393b49fbd [linux-kernel] Update patches for v4 2017-08-04 16:57:03 -07:00
Nick Terrell
7d3ac0710d [linux] Update patches for v3 2017-07-20 13:33:55 -07:00
Igor Vuk
e6e25c9507 Fix typos in README.md 2017-07-06 20:43:14 +02:00
Nick Terrell
ce91b64f00 [linux-kernel] Update patches for v2
* Reduce stack usage of many zstd functions, none use over 388 B anymore.
* Remove an incorrect `const` in `xxhash`.
2017-06-28 22:01:46 -07:00
Nick Terrell
aff86be75d [linux] Write all the patch summaries 2017-06-21 20:23:44 -07:00
Nick Terrell
0a94fead41 [linux] Rename last 2 diffs to patch names 2017-06-21 15:27:07 -07:00