Commit Graph

1185 Commits (e70912c72bcafdf4f8b43a25017eb579a85b97f6)

Author SHA1 Message Date
Yann Collet e70912c72b Changed : input divided into roughly equal parts.
Debug : can measure time waiting for mutexes to unlock.
2016-12-29 01:24:01 +01:00
Yann Collet 6c0ed9483a compression threads use ZSTD_compressCCtx() 2016-12-28 17:08:28 +01:00
Yann Collet ce9e1452fd protect buffer pool with a mutex 2016-12-28 15:31:19 +01:00
Yann Collet 3d93f2fce7 first zstdmt sketch 2016-12-27 07:19:36 +01:00
Yann Collet 39c105c605 Merge branch 'dev' of github.com:facebook/zstd into dev 2016-12-23 22:25:31 +01:00
Yann Collet aca113f4f5 fixed ZSTD_sizeof_?Dict() 2016-12-23 22:25:03 +01:00
Yann Collet c07d2e3a31 Merge pull request #499 from inikep/dev11
improved *BSD and Solaris compatibility
2016-12-23 21:32:03 +01:00
Przemyslaw Skibinski 63b0014b96 BSD: improved "make install" 2016-12-23 10:05:49 +01:00
Nick Terrell 1b5d4a7d53 ZDICT_finalizeDictionary() flipped comparison 2016-12-22 18:14:57 -08:00
Nick Terrell bcbe77e994 ZDICT_finalizeDictionary() flipped comparison
`ZDICT_finalizeDictionary()` had a flipped comparison.
I also allowed `dictBufferCapacity == dictContentSize`.
It might be the case that the user wants to fill the dictionary
completely up, and then let zstd take exactly the space it needs
for the entropy tables.
2016-12-22 18:01:14 -08:00
Nick Terrell 78a0072d5a Fix failing test due to deprecation warning 2016-12-22 17:36:16 -08:00
Yann Collet d76d1a9ef0 added ZDICT_finalizeDictionary() 2016-12-22 20:18:43 +01:00
Przemyslaw Skibinski b999170311 Solaris: working "make -C lib install" 2016-12-22 20:14:37 +01:00
Yann Collet ba75e9d8c3 fix : zlib wrapper compile in gnu90 mode 2016-12-21 19:57:18 +01:00
Yann Collet 0819abe3c1 added ZSTD_createDDict_byReference() body 2016-12-21 19:25:15 +01:00
Yann Collet 4e5eea61a8 added ZSTD_createDDict_byReference() 2016-12-21 16:44:35 +01:00
Yann Collet 8333106b8a Merge branch 'dev' of github.com:facebook/zstd into dev 2016-12-21 16:44:24 +01:00
Yann Collet 0d7e84899f Merge pull request #489 from inikep/v112
improved detection of POSIX
2016-12-21 16:42:46 +01:00
Yann Collet 1f57c2ed32 added : ZSTD_createCDict_byReference() 2016-12-21 16:20:11 +01:00
Przemyslaw Skibinski 2f6ccee6af platform.h: removed Compiler Options 2016-12-21 13:23:34 +01:00
Przemyslaw Skibinski 5736db219e fix basic types redefinition 2016-12-21 09:26:00 +01:00
Nick Terrell 8157a4c3cc Fix dictionary loading bug causing an MSAN failure
Offset rep codes must be in the range `[1, dictSize)`.
Fix dictionary loading to reject `0` as a offset rep code.
2016-12-20 10:47:52 -08:00
Przemyslaw Skibinski f8046b8e72 Merge remote-tracking branch 'refs/remotes/facebook/dev' into v112
# Conflicts:
#	appveyor.yml
2016-12-19 08:20:26 +01:00
Yann Collet d564faa3c6 fix : ZSTD_initCStream_srcSize() correctly set srcSize in frame header 2016-12-18 21:39:15 +01:00
Yann Collet 1496c3dc47 Fix : size estimation when some samples are very large 2016-12-18 11:58:23 +01:00
Yann Collet d46ecb58a5 added dll compilation tests 2016-12-17 16:28:12 +01:00
Nick Terrell 8de46ab51a Export all API functions 2016-12-16 13:27:30 -08:00
Przemyslaw Skibinski 0a1caeef6d VS: fixed 32-bit DLL compilation 2016-12-15 12:12:46 +01:00
Przemyslaw Skibinski 4e10bd339d appveyor.yml: added tests of fullbench-dll fullbench-lib 2016-12-15 12:09:23 +01:00
Przemyslaw Skibinski 60f10aab6c introduced ZSTDLIB_VISIBILITY 2016-12-15 11:32:31 +01:00
Yann Collet 2b36b238d3 changed variable name to estimatedSrcSize, to emphasize it does not need to be exact 2016-12-13 17:59:55 +01:00
Yann Collet e795c8a5f6 Added ZSTD_initCStream_srcSize().
Added relevant test cases in zstreamtest
2016-12-13 17:00:14 +01:00
Yann Collet 5397a66b19 minor BMI version check 2016-12-13 15:21:06 +01:00
Yann Collet 35168679bd Merge pull request #478 from terrelln/wildcopy-ub
Fix execSequence wildcopy undefined behavior
2016-12-13 11:33:00 +01:00
Nick Terrell 064a143520 Fix execSequence wildcopy undefined behavior
execSequence relied on pointer overflow to handle cases where
`sequence.matchLength < 8`.  Instead of passing an `size_t` to
wildcopy, pass a `ptrdiff_t`.
2016-12-12 19:01:23 -08:00
Nick Terrell e474aa55b4 Fix decompression buffer overrun
Allows an adversary to write up to 3 bytes beyond the end of the buffer.
Occurs if the match overlaps the `extDict` and `currentPrefix`, and the
match length in the `currentPrefix` is less than `MINMATCH`, and
`op-(16-MINMATCH) >= oMatchEnd > op-16`.
2016-12-12 18:05:30 -08:00
Yann Collet c3a5c4bef8 introduced cycleLog 2016-12-12 00:47:30 +01:00
Yann Collet c261f71f6a minor variation of rescale fix 2016-12-12 00:25:07 +01:00
Nick Terrell 3826207a70 Simplify segfault fix
Take advantage of the fact that `chainLog <= windowLog`.
2016-12-10 18:46:55 -08:00
Nick Terrell 0012332ce0 Fix compression segfault
When the overflow protection kicks in, it makes sure that ip - ctx->base
isn't too large.  However, it didn't ensure that saved offsets are
still valid.  This change ensures that any valid offsets (<= windowLog)
are still representable after the update.

The bug would shop up on line 1056, when `offset_1 > current + 1`, which
causes an underflow.  This in turn, would cause a segfault on line 1063.

The input must necessarily be longer than 1 GB for this issue to occur.
Even then, it only occurs if one of the last 3 matches is larger than
the chain size and block size.
2016-12-09 17:15:33 -08:00
Yann Collet 383b8088a3 minor lib build refactoring 2016-12-08 18:42:27 -08:00
Yann Collet 6e754fe76a fixed lib soname.
example : simple_compression : size overflow check
2016-12-08 18:26:56 -08:00
Przemyslaw Skibinski 7687913178 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2016-12-08 10:42:42 +01:00
Yann Collet 426a9d4b71 changed : dll : only approved ZSTD symbols are now exposed. All other symbols remain internal. 2016-12-07 16:39:34 -08:00
Przemyslaw Skibinski 4da53219a0 zstd Manual updated to 1.1.2 2016-12-07 11:18:40 +01:00
Yann Collet 379908be3d fixed zstd.h for manual 2016-12-06 10:36:15 -08:00
Yann Collet 9dfd0af164 ZBUFF_ as a wrapper to ZSTD streaming API. 2016-12-06 17:16:41 +01:00
Yann Collet 825dffbc43 moved zbuff source files into lib/deprecated 2016-12-05 19:28:19 -08:00
Yann Collet 8f8e2b0b4a fixed initialization warning 2016-12-05 18:00:50 -08:00
Yann Collet e7a41a5955 added : dictID retrieval functions.
added : unit tests for dictID retrieval functions
2016-12-05 16:21:06 -08:00