Commit Graph

491 Commits (b772f53952fa167e3c8d5630b26397e8d4fb4c5b)

Author SHA1 Message Date
Dominique Pelle b772f53952 Typo and grammar fixes 2022-03-12 08:58:04 +01:00
Elliot Gorokhovsky db2f4a6532 Move bitwise builtins into bits.h 2022-02-14 11:16:03 -05:00
Yann Collet 2d154e627a renamed HufLog into ZSTD_HUFFDTABLE_CAPACITY_LOG
old name was not descriptive and actually misleading
2022-01-26 14:47:24 -08:00
Nick Terrell 5414dd7978 [bmi2] Add lzcnt and bmi target attributes
* When dynamic dispatching to bmi2 add lzcnt and bmi to the
  TARGET_ATTRIBUTE.
* Centralize the bmi2 TARGET_ATTRIBUTE definition to
  BMI2_TARGET_ATTRIBUTE so we can change it in the future.
* Only enable bmi2 when both bmi1 & bmi2 are supported. There shouldn't
  be any cases where bmi2 is supported but bmi1 isn't. But, since we are
  using the instruction we should check bmi1 as well.
2021-11-30 17:54:56 -08:00
Yann Collet 518f06b281 added minimum for decoder buffer
also : introduced macro BOUNDED()
2021-10-26 08:21:31 -07:00
binhdvo 6a7ede3dfc
Reduce size of dctx by reutilizing dst buffer (#2751)
* Reduce size of dctx by reutilizing dst buffer

Co-authored-by: Binh Vo <binhvo@fb.com>
2021-10-25 10:38:01 -04:00
Nick Terrell 31316cf158 [multiple-ddicts] Fix NULL checks
The bug was reported by Dan Carpenter and found by Smatch static
checker.

https://lore.kernel.org/all/20211008063704.GA5370@kili/
2021-10-08 11:24:58 -07:00
Norbert Lange 6763f40331 zstd_decompress: use a helper function for context create
Multiple ZSTD_createDCtx* functions call other (public)
ZSTD_createDCtx* functions, this makes it harder for humans
and compilers to throw out code that is not used.

This farms out the logic into a static function, if a program
only uses a single ZSTD_createDCtx variant, all others can be easily
dropped and the remaining implementation can be specialized.
2021-09-26 14:41:37 +02:00
Norbert Lange 0d45540695 decompress: conditionally remove bmi2 from context
Use an helper function, which will just return 0 in case
the feature is disabled.
Allows constant propagation and removal of dead code.
2021-09-26 14:41:37 +02:00
Norbert Lange 02296cac82 decompress: conditionally remove legacy members from context
Remove the then unneeded variables from the struct,
and all accesses to them.
2021-09-26 12:12:17 +02:00
Nick Terrell 189e87bcbe [lib] Make lib compatible with `-Wfall-through` excepting legacy
Switch to a macro `ZSTD_FALLTHROUGH;` instead of a comment. On supported
compilers this uses an attribute, otherwise it becomes a comment.

This is necessary to be compatible with clang's `-Wfall-through`, and
gcc's `-Wfall-through=2` which don't support comments. Without this the
linux build emits a bunch of warnings.

Also add a test to CI to ensure that we don't regress.
2021-09-23 10:51:18 -07:00
Binh Vo 9d9f7680f8 Add API for fetching skippable frame content 2021-06-14 16:01:28 -04:00
Nick Terrell 6cee3c2c4f [trace] Remove default definitions of weak symbols
Instead of providing a default no-op implementation, check the symbols
for `NULL` before accessing them. Providing a default implementation
doesn't reliably work with dynamic linking. Depending on link order the
default implementations may not be overridden. By skipping the default
implementation, all link order issues are resolved. If the symbols
aren't provided the weak function will be `NULL`.
2021-04-26 16:05:39 -07:00
Nick Terrell a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:30:43 -07:00
Nick Terrell cd1551d261 [lib][tracing] Add ZSTD_NO_TRACE macro
When defined, it disables tracing, and avoids including the header.
2021-03-16 11:47:27 -07:00
Nick Terrell e59c9459a5 [trace] Keep track of a uint64_t tracing context
The most common information that you want to track between begin() and
end() is the timestamp of the begin function, so you can measure the
duration of the (de)compression call. Allow the tracing library to put
this information inside the `ZSTD_TraceCtx`, so it doesn't need to keep
a global map in this case. If a single uint64_t is not enough, the
tracing library can return a unique identifier (like the context
pointer) instead, and use it as a key in a map.

This keeps the simple case simple.
2021-02-09 11:37:05 -08:00
Nick Terrell 54a4998a80 Add basic tracing functionality 2021-02-05 16:28:52 -08:00
Nick Terrell f9b1e711ba [zstd] Fix NULL pointer addition in ZSTD_checkContinuity()
Don't start a new section when `dstSize == 0` to avoid NULL
pointer addition.
2021-02-05 12:18:06 -08:00
Yann Collet b9748757b0 fixed minor cast warning 2021-02-05 09:55:54 -08:00
senhuang42 9ae0dd9336 Fix Visual and staticanalyze warnings 2021-01-07 17:58:37 -05:00
senhuang42 c2c9b8a7ec Address comments, clean up interface/internals 2021-01-07 12:29:12 -05:00
senhuang42 22b7bff2bc Add unit test, improve documentation 2021-01-07 12:29:12 -05:00
senhuang42 ea52fc3606 Use XXHash for hash function, create a sensible public interface 2021-01-07 12:29:12 -05:00
senhuang42 7c1a79f232 Add debuglog statements 2021-01-07 12:29:11 -05:00
senhuang42 d1a6a9d285 Reference requested dict ID at decompression time 2021-01-07 12:29:11 -05:00
senhuang42 5a6d3eef2b Allocate memory for DDict hash set when parameter is set 2021-01-07 12:29:11 -05:00
senhuang42 fd5b608f1c Add parameter to control multiple DDicts 2021-01-07 12:29:11 -05:00
senhuang42 f933668d3f Implement hashset for dictIDs 2021-01-07 12:29:11 -05:00
Nick Terrell 66e811d782 [license] Update year to 2021 2021-01-04 17:53:52 -05:00
Yann Collet 95e74616d5 fix multiple minor conversion warnings
unrelated to #2386, just cleaning up while I'm updating this file ...
2020-11-06 09:57:05 -08:00
Yann Collet 2769e4d459 fix incorrect assert
fix #2386, reported by @Neumann-A
2020-11-06 09:44:04 -08:00
Nick Terrell e3e0775cc8 [API] Add ZSTD_c_stable{In,Out}Buffer parameters
This commit adds the parameters and sets the value in the CCtxParams
but it does not do anything with the value.
2020-10-30 10:54:39 -07:00
Nick Terrell 2e7d174130 Reset all decompression parameters in ZSTD_DCtx_reset()
* Reset all decompression parameters in `ZSTD_DCtx_reset()` when
  resetting parameters.
* Add a test case.
2020-10-01 14:19:21 -07:00
Nick Terrell 9261476b7d [lib] Wrap customMem xor checks in parens for readability
This clarifies operator precedence, and quiets cppcheck in
the Kernel Test Robot. I think this is a slight bonus to
readability, so I am accepting the suggestion.
2020-09-23 23:26:07 -07:00
Nick Terrell c465f24457 ZSTD_ prefix mem{cpy,move,set},malloc,calloc,free 2020-08-26 12:26:03 -07:00
Nick Terrell a686d306d2 Rename ZSTD_{malloc,calloc,free} to ZSTD_custom{Malloc,Calloc,Free} 2020-08-26 12:25:08 -07:00
Nick Terrell 80f577baa2 Move standard includes to zstd_deps.h 2020-08-26 12:25:08 -07:00
Yann Collet f82d9865b9
Merge pull request #2278 from senhuang42/ignore_checksum_advanced_param
New advanced decompression param to ignore checksums
2020-08-25 12:08:53 -07:00
Nick Terrell 614e446000
Merge pull request #2271 from terrelln/small-blocks
Small block optimizations
2020-08-24 18:54:33 -07:00
senhuang42 a030560d62 Add new DCtx param: validateChecksum and update unit tests 2020-08-24 17:28:00 -04:00
Nick Terrell 1302f8d676 [fix] Always return dstSize_tooSmall when it is the case 2020-08-24 13:38:13 -07:00
senhuang42 44c54a3e31 Addressing comments: more comments, cleanup, remove extra function, checksum logic 2020-08-24 16:14:19 -04:00
senhuang42 ffaa0df76d Document change in CLI for --no-check during decompression in --help menu 2020-08-24 09:49:12 -04:00
senhuang42 e3f5f9658a Added CLI tests for --no-check, fixed ignore checksum logic 2020-08-22 16:05:40 -04:00
senhuang42 20eb095882 Added unit test to fuzzer.c, changed definition param name 2020-08-22 13:26:33 -04:00
senhuang42 47685ac856 Move enum into zstd.h, and fix pesky switch() logic 2020-08-21 18:18:53 -04:00
senhuang42 08d3567ba8 Add function prototype 2020-08-21 16:51:43 -04:00
senhuang42 6a8dbdcd1f Modify decompression loop to gnore checksums if flag is enabled 2020-08-21 16:46:46 -04:00
senhuang42 2f39124342 Rename to ZSTD_d_forceIgnoreChecksum, add to DCtx, add function to set the advanced param 2020-08-21 16:23:39 -04:00
Nick Terrell 612e947c5e wire up bmi2 support 2020-08-17 16:35:28 -07:00