Commit Graph

33 Commits (b772f53952fa167e3c8d5630b26397e8d4fb4c5b)

Author SHA1 Message Date
Dominique Pelle b772f53952 Typo and grammar fixes 2022-03-12 08:58:04 +01:00
Nick Terrell d7542aacd9 [fuzzer] Add huf_decompress fuzzer
Add a fuzzer for Huffman decompression. Fix several bugs in Huffman
decompression, mostly related to `op == NULL` and pointer underflow.
2021-09-17 15:00:49 -07:00
Nick Terrell d8a0797268 [fuzz] Add Huffman round trip fuzzer
* Add a Huffman round trip fuzzer
* Fix two minor bugs in Huffman that aren't exposed in zstd
  - Incorrect weight comparison (weights are allowed to be equal to
    table log).
  - HUF_compress1X_usingCTable_internal() can return compressed
    size >= source size, so the assert that `cSize <= 65535` isn't
    correct, and it needs to be checked instead.
2021-08-03 08:10:06 -07:00
sen d6be7659b0
Add seekable roundtrip fuzzer (#2617) 2021-05-06 10:08:21 -04:00
Nick Terrell a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:30:43 -07:00
Nick Terrell 91e6480458 [fuzz] Fix compiler detection & update ubsan flags
* Fix compiler version regex, which was broken for multi-digit
  versions.
* Fix compiler detection for gcc.
* Disable `pointer-overflow` instead of `integer-overflow` for gcc
  versions newer than 8.0.0.
2021-02-19 13:19:18 -08:00
Nick Terrell 66e811d782 [license] Update year to 2021 2021-01-04 17:53:52 -05:00
senhuang42 26bc0bfdf6 Add new fuzzer to build targets 2020-11-20 16:58:25 -05:00
Nick Terrell 4193638996 [bug] Fix FSE_readNCount()
* Fix bug introduced in PR #2271
* Fix long-standing bug that is impossible to trigger inside of zstd
* Add a fuzzer that makes sure the normalized count always round trips
  correctly
2020-08-25 15:42:41 -07:00
Nick Terrell 1302f8d676 [fix] Always return dstSize_tooSmall when it is the case 2020-08-24 13:38:13 -07:00
Bimba Shrestha 255e5e3f56
[fuzz] Adding dictionary_stream_round_trip fuzzer (#2140)
* Adding dictionary_stream_round_trip

* fixing memory leak
2020-05-15 13:33:31 -07:00
Nick Terrell 1185dfb8d1 [fuzz] Add raw dictionary content fuzzer 2020-05-11 19:03:33 -07:00
Nick Terrell ac58c8d720 Fix copyright and license lines
* All copyright lines now have -2020 instead of -present
* All copyright lines include "Facebook, Inc"
* All licenses are now standardized

The copyright in `threading.{h,c}` is not changed because it comes from
zstdmt.

The copyright and license of `divsufsort.{h,c}` is not changed.
2020-03-26 17:02:06 -07:00
Nick Terrell 75e7c0d107 [fuzz] Add dictionary_loader fuzzer
* Adds the fuzzer
* Adds an additional `InputType` for the fuzzer

I ran the fuzzer for about 10 minutes and it found 2 bugs:

* Catches the original bug without any help
* Catches an additional bug with 8-byte dictionaries
2019-11-01 15:54:24 -07:00
Nick Terrell 7c4578160e [fuzz] Generate seed data up to 256KB 2019-09-12 15:02:01 -07:00
Dario Pavlovic cd8588077e It's time for all of rng seed code to go. Goodbye 2019-09-12 13:10:34 -07:00
Nick Terrell 3982935aef [fuzz] Improve fuzzer build script and docs
* Remove the `make libFuzzer` target since it is broken and obsoleted
  by `CC=clang CXX=clang++ ./fuzz.py build all --enable-fuzzer`. The
  new `-fsanitize=fuzzer` is much better because it works with MSAN
  by default.
* Improve the `./fuzz.py gen` command by making the input type explicit
  when creating a new target.
* Update the `README` for `--enable-fuzzer`.

Fixes #1727.
2019-08-20 16:44:50 -07:00
Nick Terrell e962f07d19
[fuzz] Add a compression fuzzer with randomly sized output buffer (#1670) 2019-07-02 22:05:07 -07:00
Nick Terrell 09caa4d800 [fuzzer] Add a fuzzer for frame info functions
Add a fuzzer that fuzzes all helper functions that take compressed
input. This fuzzer caught one out of bounds read in
`ZSTD_decompressBound()`.
2019-04-17 11:29:42 -07:00
Josh Soref a880ca239b Spelling (#1582)
* spelling: accidentally

* spelling: across

* spelling: additionally

* spelling: addresses

* spelling: appropriate

* spelling: assumed

* spelling: available

* spelling: builder

* spelling: capacity

* spelling: compiler

* spelling: compressibility

* spelling: compressor

* spelling: compression

* spelling: contract

* spelling: convenience

* spelling: decompress

* spelling: description

* spelling: deflate

* spelling: deterministically

* spelling: dictionary

* spelling: display

* spelling: eliminate

* spelling: preemptively

* spelling: exclude

* spelling: failure

* spelling: independence

* spelling: independent

* spelling: intentionally

* spelling: matching

* spelling: maximum

* spelling: meaning

* spelling: mishandled

* spelling: memory

* spelling: occasionally

* spelling: occurrence

* spelling: official

* spelling: offsets

* spelling: original

* spelling: output

* spelling: overflow

* spelling: overridden

* spelling: parameter

* spelling: performance

* spelling: probability

* spelling: receives

* spelling: redundant

* spelling: recompression

* spelling: resources

* spelling: sanity

* spelling: segment

* spelling: series

* spelling: specified

* spelling: specify

* spelling: subtracted

* spelling: successful

* spelling: return

* spelling: translation

* spelling: update

* spelling: unrelated

* spelling: useless

* spelling: variables

* spelling: variety

* spelling: verbatim

* spelling: verification

* spelling: visited

* spelling: warming

* spelling: workers

* spelling: with
2019-04-12 11:18:11 -07:00
Nick Terrell 7a1fde2957 [fuzzer] Add dictionary fuzzers 2019-04-08 21:07:28 -07:00
Nick Terrell 4b0024a97d [fuzz] Add --enable-fuzzer for clang fuzzing 2019-02-27 17:15:52 -08:00
Nick Terrell 3ff6040848 Publish artifacts with CircleCI
* Updates CircleCI to use workflows.
  We can now specify any number of test jobs to run in parallel.
* Switch the image to `buildpack-deps:trusty` which is only 500 MB
  instead of 7 GB, so that saves 7 minutes to download it if it isn't
  already cached on the host.
* Publish the source tarball and sha256sum as artifacts.
* If the `GITHUB_TOKEN` environment variable is set, we will also
  add the tarball + sha256sum to the tagged release, after manual
  approval.
2018-09-26 13:23:28 -07:00
Yann Collet fa41bcc2c2 grouped debug functions into debug.h
There were 2 competing set of debug functions
within zstd_internal.h and bitstream.h.
They were mostly duplicate, and required care to avoid messing with each other.

There is now a single implementation, shared by both.

Significant change :
The macro variable ZSTD_DEBUG does no longer exist,
it has been replaced by DEBUGLEVEL,
which required modifying several source files.
2018-06-13 15:43:09 -04:00
Nick Terrell fdd4d8510f Improve compiler detection to work on Mac 2018-05-24 14:21:12 -07:00
Nick Terrell ac852abb8b Define BIT_DEBUG for --debug 2018-05-24 14:21:12 -07:00
Nick Terrell e712a3a0a3 Small fixes to fuzz.py 2018-05-24 14:21:12 -07:00
Nick Terrell d9c1e9125f [fuzz] Small changes for oss-fuzz integration 2017-09-27 18:23:06 -07:00
Nick Terrell 917a213254 [fuzz] Determine flags based on compiler version 2017-09-25 15:32:36 -07:00
Nick Terrell 23199b6daf [fuzz] Fix fuzz.py env flags parsing 2017-09-25 13:28:18 -07:00
Nick Terrell 1c23b64049 [fuzz] fuzz.py can minimize and zip corpora
* "minimize" minimizes the corpora into an output directory.
* "zip" zips up the minimized corpora, which are ready to deploy.
2017-09-25 12:04:12 -07:00
Nick Terrell 9712d5ebe6 [fuzzer] Fix bugs in fuzz.py 2017-09-13 19:08:35 -07:00
Nick Terrell 6b8236cf7e [fuzz] Add fuzzing helper script 2017-09-13 17:45:21 -07:00