facebook/zstd
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,162 Commits 2 Branches 0 Tags
Commit Graph

1432 Commits

Author SHA1 Message Date
Nick Terrell
4c2943df23 [regression] Update results for ZSTD_double_fast update 2019-07-15 11:25:22 -07:00
Nick Terrell
75cfe1dc69
[ldm] Fix bug in overflow correction with large job size (#1678) 
* [ldm] Fix bug in overflow correction with large job size

* [zstdmt] Respect ZSTDMT_JOBSIZE_MAX (1G in 64-bit mode)

* [test] Add test that exposes the bug

Sadly the test fails on our CI because it uses too much memory, so
I had to comment it out.
 2019-07-12 18:45:18 -04:00
LeeYoung624
654cb9d439 fix gitignore errors 2019-07-09 21:08:13 +08:00
Nick Terrell
e962f07d19
[fuzz] Add a compression fuzzer with randomly sized output buffer (#1670) 2019-07-02 22:05:07 -07:00
Yann Collet
4d611ca405
Merge pull request #1664 from ephiepark/dev 
decodecorpus
 2019-07-01 14:13:49 -07:00
Ephraim Park
28309520c0 reflect code review comments 2019-07-01 10:17:30 -07:00
Ephraim Park
01e8384000 Add test case for short bistream 2019-06-27 17:37:39 -07:00
Tyler-Tran
c55d2e7ba3 Adding shrinking flag for cover and fastcover (#1656) 
* Changed ERROR(GENERIC) excluding inits

* editing git ignore

* Edited init functions to size_t returns

* moved declarations earlier

* resolved issues with changes to init functions

* fixed style and an error check

* attempting to add tests that might trigger changes

* added && die to cases expecting to fail

* resolved no die on expected failed command

* fixed accel to be incorrect value

* Adding an automated shrinking option

* Fixing build

* finalizing fixes

* fix?

* Removing added comment in cover.h

* Styling fixes

* Merging with fb dev

* removing megic number for default regression

* Requested revisions

* fixing support for fast cover

* fixing casting errors

* parenthesis fix

* fixing some build nits

* resolving travis ci syntax

* might resolve all compilation issues

* removed unused variable

* remodeling the selectDict function

* fixing bad memory access

* fixing error checks

* fixed erroring check in selectDict

* fixing mixed declarations

* modify mixed declaration

* fixing nits and adding test cases

* Adding requested changes + fixed bug for error checking

* switched double comparison from != to <

* fixed declaration typing

* refactoring COVER_best_finish() and changing shrinkDict

* removing the const's

* modifying ZDICT_optimizeTrainFromBuffer_cover functions

* fixing potential bad memcpy

* fixing the error function for dict size
 2019-06-27 16:26:57 -07:00
Ephraim Park
734eff70b8 enable repeat mode on rle 2019-06-26 16:39:00 -07:00
Nick Terrell
8e16b872e5 [tests] Add tests for big dictionaries 2019-06-21 17:58:24 -07:00
Nick Terrell
06d5e5ff5a [regression] Update results for small wlog patch PR#1624 2019-06-17 10:40:33 -07:00
Yann Collet
1166782eb7
Merge pull request #1640 from felixhandte/lstat-macro-guard 
Protect lstat() With Better Macro Guard
 2019-06-14 17:30:03 -07:00
Felix Handte
344f942573
Merge pull request #1620 from michaelforney/test-no-threads 
Skip --adapt and --rsyncable tests when built without thread support
 2019-06-11 15:05:25 -04:00
Nick Terrell
81276c4ee4
Merge pull request #1646 from terrelln/oss-fuzz 
[fuzz] Remove max_len from the options
 2019-06-10 22:13:16 -07:00
Nick Terrell
6810dd6191 [fuzz] Remove max_len from the options 2019-06-10 11:05:45 -07:00
W. Felix Handte
b878bd08dd Clean Up Temp Files Produced By playTests.sh 2019-06-07 15:32:28 -04:00
Nick Terrell
f3800ba073 [regression] Update results.csv 2019-06-05 20:40:52 -07:00
Yann Collet
78056a3e9c consistency : changed constant name to kSampleSizeDefault 
as suggested by @terrelln.

added <assert.h> include
 2019-05-30 10:13:56 -07:00
Yann Collet
ab3625cffd define CONTROL, to ensure check cannot be disabled 
assert() can be disabled with NDEBUG
 2019-05-29 16:18:22 -07:00
Yann Collet
904d4da239 fullbench : minor refactoring, for readability 2019-05-29 16:08:49 -07:00
Yann Collet
ed38b645db fullbench: pass proper parameters in scenario 43 2019-05-29 15:26:06 -07:00
Yann Collet
c63081623f fullbench 43: ensure context is freed after each usage 2019-05-28 17:27:52 -07:00
Yann Collet
eb6b199038 fullbench: added streaming with fresh CCtx scenario 
worst case situation, where context must be recreated every time
but without knowledge of the input size
thus sizing the context for some large input.
 2019-05-28 16:50:49 -07:00
Yann Collet
4c4149452c fullbench can now select sample size with command -B# 2019-05-28 16:37:03 -07:00
Yann Collet
4baecdf72a added comments to better understand enforceMaxDist() 2019-05-28 13:15:48 -07:00
Michael Forney
1bc77ee174 Skip --adapt and --rsyncable tests when built without thread support 
These options require ZSTD_MULTITHREAD.
 2019-05-26 03:20:37 -07:00
W. Felix Handte
6a0638048a Add Test 2019-05-24 17:21:44 -04:00
Tyler-Tran
cb47871a0a [dictBuilder] Be more specific than ERROR(generic) (#1616) 
* Specify errors at a finer granularity than `ERROR(generic)`.
* Add tests for bad parameters in the dictionary builder.
 2019-05-22 18:57:50 -07:00
Nick Terrell
bfbef51afb
Merge pull request #1610 from terrelln/pool-test 
[test] Fix poolTests deadlock and flakiness
 2019-05-02 15:13:50 -07:00
Nick Terrell
81f1f76479 [tests] Only sleep 10ms in poolTests 2019-05-02 14:35:25 -07:00
Nick Terrell
2953d45a0f [test] Fix poolTests deadlock and flakiness 2019-05-02 13:23:41 -07:00
lzutao
1d70bc3ba9 meson: Enable testing in CI (#1609) 
* tests: Fix shellcheck warnings in playTests.sh

* tests: Do not use ../programs which is relative to tests dirs

This commit fixes error when running playTests.sh in Meson.
Mesonbuild runs out of tree, so ./datagen not in `zstd/tests` dir,
it lies in <mesonbuilddir>/tests. This leads to ../programs invalid.

* tests: Replace relative paths for zstd/tests dir

* playTests: Set shell options explicitly, not in shebang

* playTests: Replace echo -e with printf

* meson: Fix test-zstd

Use std=gnu99 to build and test just like `make test`.

* meson: Fix legacy test

* meson: Enable testing in CI

Run build under release mode for faster test time.

* meson: Increase timeout time for test-zstream
 2019-05-02 12:35:37 -07:00
cev
3e1e49d775 add test for zstdcat and zcat on symlink 2019-04-29 18:47:47 -04:00
Nick Terrell
785331acbd [paramgrill] Fix mingw build errors 2019-04-18 15:06:56 -07:00
Nick Terrell
610a81ecf9 [fuzzer] Compile with legacy support 2019-04-18 12:44:55 -07:00
Nick Terrell
cc669006dc [fuzzer] Size the decompression output buffer randomly 2019-04-18 12:44:21 -07:00
Nick Terrell
58bcc328a4 [fuzz] Add a seedcorpora target for oss-fuzz 2019-04-17 12:13:06 -07:00
Nick Terrell
09caa4d800 [fuzzer] Add a fuzzer for frame info functions 
Add a fuzzer that fuzzes all helper functions that take compressed
input. This fuzzer caught one out of bounds read in
`ZSTD_decompressBound()`.
 2019-04-17 11:29:42 -07:00
Nick Terrell
a48e32ca50 [fullbench] Fix measurements 2019-04-15 21:01:48 -07:00
Nick Terrell
6996804d05 [playTets] Don't pass status=none to dd 
The OS X test machine on TravisCI doesn't recognize the parameter
https://travis-ci.org/facebook/zstd/jobs/520517933.
 2019-04-15 16:21:46 -07:00
Josh Soref
a880ca239b Spelling (#1582) 
* spelling: accidentally

* spelling: across

* spelling: additionally

* spelling: addresses

* spelling: appropriate

* spelling: assumed

* spelling: available

* spelling: builder

* spelling: capacity

* spelling: compiler

* spelling: compressibility

* spelling: compressor

* spelling: compression

* spelling: contract

* spelling: convenience

* spelling: decompress

* spelling: description

* spelling: deflate

* spelling: deterministically

* spelling: dictionary

* spelling: display

* spelling: eliminate

* spelling: preemptively

* spelling: exclude

* spelling: failure

* spelling: independence

* spelling: independent

* spelling: intentionally

* spelling: matching

* spelling: maximum

* spelling: meaning

* spelling: mishandled

* spelling: memory

* spelling: occasionally

* spelling: occurrence

* spelling: official

* spelling: offsets

* spelling: original

* spelling: output

* spelling: overflow

* spelling: overridden

* spelling: parameter

* spelling: performance

* spelling: probability

* spelling: receives

* spelling: redundant

* spelling: recompression

* spelling: resources

* spelling: sanity

* spelling: segment

* spelling: series

* spelling: specified

* spelling: specify

* spelling: subtracted

* spelling: successful

* spelling: return

* spelling: translation

* spelling: update

* spelling: unrelated

* spelling: useless

* spelling: variables

* spelling: variety

* spelling: verbatim

* spelling: verification

* spelling: visited

* spelling: warming

* spelling: workers

* spelling: with
 2019-04-12 11:18:11 -07:00
Lzu Tao
fb6901b2c0 tests: Add missing header timefn.h 2019-04-12 21:28:17 +07:00
Yann Collet
8ac2831f3d
Merge pull request #1581 from facebook/benchfn 
benchfn's reduced dependencies
 2019-04-11 14:23:04 -07:00
Yann Collet
058da605cb fixed minor conversion warning 2019-04-11 12:25:27 -07:00
Yann Collet
f8e9bec73a fixed poolTests on Windows 
must use ZSTD_ prefix in front of pthread types
so that they get properly translated for Windows.
 2019-04-11 12:03:42 -07:00
Yann Collet
8f56fa2f58 fixed poolTests 
added poolTests to all
 2019-04-11 09:50:39 -07:00
Yann Collet
9703a59121 fixed minor conversion warning 2019-04-10 15:54:55 -07:00
Nick Terrell
50b9c41196 [libzstd] Fix decompression dictionary bugs and clean up initialization 
Bugs:

* `ZSTD_DCtx_refPrefix()` didn't clear the dictionary after the first
  use. Fix and add a test case.
* `ZSTD_DCtx_reset()` always cleared the dictionary. Fix and add a test
  case.
* After calling `ZSTD_resetDStream()` you could no longer load a
  dictionary, since the stage was set to `zdss_loadHeader`. Fix and add
  a test case.

Cleanup:

* Make `ZSTD_initDStream*()` and `ZSTD_resetDStream()` wrap the new
 advanced API, and add test cases.
* Document the equivalent of these functions in the advanced API and
  document the unstable functions as deprecated.
 2019-04-10 12:59:02 -07:00
Yann Collet
59a7116cc2 benchfn dependencies reduced to only timefn 
benchfn used to rely on mem.h, and util,
which in turn relied on platform.h.
Using benchfn outside of zstd required to bring all these dependencies.

Now, dependency is reduced to timefn only.
This required to create a separate timefn from util,
and rewrite benchfn and timefn to no longer need mem.h.

Separating timefn from util has a wide effect accross the code base,
as usage of time functions is widespread.
A lot of build scripts had to be updated to also include timefn.
 2019-04-10 12:37:03 -07:00
Yann Collet
094c000904 Merge branch 'dev' into benchfn 2019-04-10 11:57:05 -07:00