Commit Graph

45 Commits (2e76bd7d1013369f0a9a10d5c01c53ec8f65121f)

Author SHA1 Message Date
Nick Terrell a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:30:43 -07:00
Nick Terrell 66e811d782 [license] Update year to 2021 2021-01-04 17:53:52 -05:00
Like Ma cc907770bd Fix building on AIX 5.1 2020-10-09 18:34:00 +08:00
Nick Terrell 5717bd39ee [lib] Fix NULL pointer dereference
When the output buffer is `NULL` with size 0, but the frame content size
is non-zero, we will write to the NULL pointer because our bounds check
underflowed.

This was exposed by a recent PR that allowed an empty frame into the
single-pass shortcut in streaming mode.

* Fix the bug.
* Fix another NULL dereference in zstd-v1.
* Overflow checks in 32-bit mode.
* Add a dedicated test.
* Expose the bug in the dedicated simple_decompress fuzzer.
* Switch all mallocs in fuzzers to return NULL for size=0.
* Fix a new timeout in a fuzzer.

Neither clang nor gcc show a decompression speed regression on x86-64.
On x86-32 clang is slightly positive and gcc loses 2.5% of speed.

Credit to OSS-Fuzz.
2020-05-06 12:09:02 -07:00
W. Felix Handte 6028827fee Rewrite Include Paths to be Relative
Addresses #1998.
2020-05-04 15:20:26 -04:00
W. Felix Handte c7da66c9cf Purge C++-Style Comments (`// ...`), Make Compilation Succeed Under C90 2020-05-04 10:59:15 -04:00
Nick Terrell ac58c8d720 Fix copyright and license lines
* All copyright lines now have -2020 instead of -present
* All copyright lines include "Facebook, Inc"
* All licenses are now standardized

The copyright in `threading.{h,c}` is not changed because it comes from
zstdmt.

The copyright and license of `divsufsort.{h,c}` is not changed.
2020-03-26 17:02:06 -07:00
Dávid Bolvanský 1f7228c040 Use clz ^ 31 instead of 31 - clz; better codegen for GCC 2019-09-23 21:23:09 +02:00
Nick Terrell 07f22d465d [legacy] Fix buffer overflow in v0.2 and v0.4 raw literals decompression
Extends the fix in PR#1722 to v0.2 and v0.4. These aren't built into
zstd by default, and v0.5 onward are not affected.

I only add the `srcSize > BLOCKSIZE` check to v0.4 because the comments
say that it must hold, but the equivalent comment isn't present in v0.2.

Credit to OSS-Fuzz.
2019-08-20 17:13:04 -07:00
Nick Terrell 0fd322f812 [legacy] Fix ZSTDv0*_decodeSequence()
* Version <= 0.5 could read beyond the end of `dumps`, which points into
  the input buffer.
* Check the validity of `dumps` before using it, if it is out of bounds
  return garbage values. There is no return code for this function.
* Introduce `MEM_readLE24()` for simplicity, since I don't want to trust
  that there is an extra byte after `dumps`.
2019-04-19 11:34:52 -07:00
Josh Soref a880ca239b Spelling (#1582)
* spelling: accidentally

* spelling: across

* spelling: additionally

* spelling: addresses

* spelling: appropriate

* spelling: assumed

* spelling: available

* spelling: builder

* spelling: capacity

* spelling: compiler

* spelling: compressibility

* spelling: compressor

* spelling: compression

* spelling: contract

* spelling: convenience

* spelling: decompress

* spelling: description

* spelling: deflate

* spelling: deterministically

* spelling: dictionary

* spelling: display

* spelling: eliminate

* spelling: preemptively

* spelling: exclude

* spelling: failure

* spelling: independence

* spelling: independent

* spelling: intentionally

* spelling: matching

* spelling: maximum

* spelling: meaning

* spelling: mishandled

* spelling: memory

* spelling: occasionally

* spelling: occurrence

* spelling: official

* spelling: offsets

* spelling: original

* spelling: output

* spelling: overflow

* spelling: overridden

* spelling: parameter

* spelling: performance

* spelling: probability

* spelling: receives

* spelling: redundant

* spelling: recompression

* spelling: resources

* spelling: sanity

* spelling: segment

* spelling: series

* spelling: specified

* spelling: specify

* spelling: subtracted

* spelling: successful

* spelling: return

* spelling: translation

* spelling: update

* spelling: unrelated

* spelling: useless

* spelling: variables

* spelling: variety

* spelling: verbatim

* spelling: verification

* spelling: visited

* spelling: warming

* spelling: workers

* spelling: with
2019-04-12 11:18:11 -07:00
shakeelrao 0a3fa6f909 Add legacy mode in documentation 2019-03-18 20:33:15 -07:00
shakeelrao 20aa1b455c Stylistic changes 2019-03-17 19:35:43 -07:00
shakeelrao 60796e76b0 Add legacy support to decompressBound 2019-03-15 16:10:37 -07:00
Yann Collet 73773c6b6a fixed legacy compilation tests
for some reason, these tests started failing recently on CircleCI
2018-09-27 18:15:14 -07:00
Yann Collet c173dbd6e7 no longer supported starting C++17 2017-12-04 18:00:53 -08:00
Yann Collet 3128e03be6 updated license header
to clarify dual-license meaning as "or"
2017-09-08 00:09:23 -07:00
Yann Collet 32fb407c9d updated a bunch of headers
for the new license
2017-08-18 16:52:05 -07:00
Nick Terrell 5152fb2cb2 Convert all tabs to spaces 2017-03-29 18:51:58 -07:00
Sean Purcell 9050e1925e Change name to to findFrameCompressedSize and add skippable support 2017-02-22 12:12:34 -08:00
Sean Purcell d7bfcac18a Expose frameSrcSize to experimental API 2017-02-10 11:55:44 -08:00
Sean Purcell 4e709712e1 Decompressed size functions now handle multiframes and distinguish cases
- Add ZSTD_findDecompressedSize
    - Traverses multiple frames to find total output size
- Add ZSTD_getFrameContentSize
    - Gets the decompressed size of a single frame by reading header
- Deprecate ZSTD_getDecompressedSize
2017-02-08 14:50:10 -08:00
Yann Collet 35168679bd Merge pull request #478 from terrelln/wildcopy-ub
Fix execSequence wildcopy undefined behavior
2016-12-13 11:33:00 +01:00
Nick Terrell 064a143520 Fix execSequence wildcopy undefined behavior
execSequence relied on pointer overflow to handle cases where
`sequence.matchLength < 8`.  Instead of passing an `size_t` to
wildcopy, pass a `ptrdiff_t`.
2016-12-12 19:01:23 -08:00
Nick Terrell e474aa55b4 Fix decompression buffer overrun
Allows an adversary to write up to 3 bytes beyond the end of the buffer.
Occurs if the match overlaps the `extDict` and `currentPrefix`, and the
match length in the `currentPrefix` is less than `MINMATCH`, and
`op-(16-MINMATCH) >= oMatchEnd > op-16`.
2016-12-12 18:05:30 -08:00
Nick Terrell 4359d21ad7 Merge two memset() calls into one 2016-11-14 17:52:51 -08:00
Nick Terrell 24701de877 Fix uninitialized memory read 2016-11-14 13:57:05 -08:00
Nick Terrell d760529a05 Fix stack buffer overrun when weightTotal == 0
If `weightTotal == 0`, then `BIT_highbit32(weightTotal)` is
undefined behavior in the case that it calls `__builtin_clz()`.
If `tableLog == HUF_TABLELOG_ABSOLUTEMAX` then we will access one
byte beyond the end of the buffer.
2016-10-19 11:39:11 -07:00
Nick Terrell ccfcc643da Check if dict is empty before reading first byte 2016-10-17 11:46:03 -07:00
inikep c13faa1b0f legacy decoders: restored #include <intrin.h> for VC++ 2016-09-05 13:25:07 +02:00
inikep 8161e7321a unified error codes for legacy decoders 2016-09-05 12:29:51 +02:00
Yann Collet 1563bfeabc fixing FORCE_INLINE for older compilers (#330) 2016-09-02 11:44:21 -07:00
Yann Collet 4ded9e591c added boilerplate 2016-08-30 11:06:28 -07:00
Yann Collet 87c18b2ebd fixed multiple minor warnings for XCode 2016-08-26 01:43:47 +02:00
inikep a7bb322a93 removed never referenced functions 2016-08-18 10:30:21 +02:00
inikep 48849f86f0 fixed compilation with Intel Compiler with Windows 2016-08-10 14:26:35 +02:00
Yann Collet d5c5a77990 minor comments clarifications 2016-07-20 13:35:14 +02:00
Yann Collet ddb8ebd5b3 Stricter gnu90 tests 2016-05-05 04:59:53 +02:00
Yann Collet 6bff748e43 Fixed minor warnings in legacy modes 2016-02-09 17:55:01 +01:00
Yann Collet 494c786a3b fixed outstanding strict aliasing warnings 2016-01-06 12:54:02 +01:00
Yann Collet 1fdd82312d Fixed : strict aliasing warnings in legacy decoders 2016-01-06 12:35:42 +01:00
Yann Collet c2ce890ac2 fixed bug in legacy decoder v0.2, reported by Maciej Adamczyk 2015-11-26 14:12:23 +01:00
Yann Collet 5054ee0cc0 rollbuffer refactor 2015-11-23 13:34:21 +01:00
Yann Collet ec43ba4a95 updated visual studio projects 2015-10-30 11:51:26 +01:00
Yann Collet aa074051d6 decoding support for legacy v0.2 format 2015-10-30 11:21:50 +01:00