Commit Graph

1191 Commits (18b3ce606435c665d442ef7150812fce30c66817)

Author SHA1 Message Date
Przemyslaw Skibinski 8a0bc30a2d Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2017-01-17 13:02:29 +01:00
Przemyslaw Skibinski d72f4b6b7a added "Makefile is validated" 2017-01-17 12:40:06 +01:00
Gregory Szorc 7d6f478d15 Set dictionary ID in ZSTD_initCStream_usingCDict()
When porting python-zstandard to use ZSTD_initCStream_usingCDict()
so compression dictionaries could be reused, an automated test
failed due to compressed content changing.

I tracked this down to ZSTD_initCStream_usingCDict() not
setting the dictID field of the ZSTD_CCtx attached to the
ZSTD_CStream instance.

I'm not 100% convinced this is the correct or full solution,
as I'm still seeing one automated test failing with this change.
2017-01-14 17:44:54 -08:00
Sean Purcell 834ab50fa3 Fixed decompress_usingDict not propagating corrupted dictionary error 2017-01-11 17:31:34 -08:00
Nick Terrell 8d984699db Document memory requirements for COVER algorithm 2017-01-09 18:20:10 -08:00
Nick Terrell 555e281637 Handle large input size in 32-bit mode correctly 2017-01-09 18:20:06 -08:00
Nick Terrell 3a1fefcf00 Simplify COVER parameters 2017-01-02 17:51:38 -08:00
Nick Terrell 96b39f65fa Add COVER dictionary builder 2017-01-02 13:22:51 -08:00
Yann Collet 8d7432914f Merge pull request #503 from inikep/dev11
Dev11
2016-12-28 16:50:39 +01:00
Przemyslaw Skibinski 75f3a3a335 changed default PREFIX and MANDIR 2016-12-28 12:32:41 +01:00
Yann Collet 39c105c605 Merge branch 'dev' of github.com:facebook/zstd into dev 2016-12-23 22:25:31 +01:00
Yann Collet aca113f4f5 fixed ZSTD_sizeof_?Dict() 2016-12-23 22:25:03 +01:00
Yann Collet c07d2e3a31 Merge pull request #499 from inikep/dev11
improved *BSD and Solaris compatibility
2016-12-23 21:32:03 +01:00
Przemyslaw Skibinski 63b0014b96 BSD: improved "make install" 2016-12-23 10:05:49 +01:00
Nick Terrell 1b5d4a7d53 ZDICT_finalizeDictionary() flipped comparison 2016-12-22 18:14:57 -08:00
Nick Terrell bcbe77e994 ZDICT_finalizeDictionary() flipped comparison
`ZDICT_finalizeDictionary()` had a flipped comparison.
I also allowed `dictBufferCapacity == dictContentSize`.
It might be the case that the user wants to fill the dictionary
completely up, and then let zstd take exactly the space it needs
for the entropy tables.
2016-12-22 18:01:14 -08:00
Nick Terrell 78a0072d5a Fix failing test due to deprecation warning 2016-12-22 17:36:16 -08:00
Yann Collet d76d1a9ef0 added ZDICT_finalizeDictionary() 2016-12-22 20:18:43 +01:00
Przemyslaw Skibinski b999170311 Solaris: working "make -C lib install" 2016-12-22 20:14:37 +01:00
Yann Collet ba75e9d8c3 fix : zlib wrapper compile in gnu90 mode 2016-12-21 19:57:18 +01:00
Yann Collet 0819abe3c1 added ZSTD_createDDict_byReference() body 2016-12-21 19:25:15 +01:00
Yann Collet 4e5eea61a8 added ZSTD_createDDict_byReference() 2016-12-21 16:44:35 +01:00
Yann Collet 8333106b8a Merge branch 'dev' of github.com:facebook/zstd into dev 2016-12-21 16:44:24 +01:00
Yann Collet 0d7e84899f Merge pull request #489 from inikep/v112
improved detection of POSIX
2016-12-21 16:42:46 +01:00
Yann Collet 1f57c2ed32 added : ZSTD_createCDict_byReference() 2016-12-21 16:20:11 +01:00
Przemyslaw Skibinski 2f6ccee6af platform.h: removed Compiler Options 2016-12-21 13:23:34 +01:00
Przemyslaw Skibinski 5736db219e fix basic types redefinition 2016-12-21 09:26:00 +01:00
Nick Terrell 8157a4c3cc Fix dictionary loading bug causing an MSAN failure
Offset rep codes must be in the range `[1, dictSize)`.
Fix dictionary loading to reject `0` as a offset rep code.
2016-12-20 10:47:52 -08:00
Przemyslaw Skibinski f8046b8e72 Merge remote-tracking branch 'refs/remotes/facebook/dev' into v112
# Conflicts:
#	appveyor.yml
2016-12-19 08:20:26 +01:00
Yann Collet d564faa3c6 fix : ZSTD_initCStream_srcSize() correctly set srcSize in frame header 2016-12-18 21:39:15 +01:00
Yann Collet 1496c3dc47 Fix : size estimation when some samples are very large 2016-12-18 11:58:23 +01:00
Yann Collet d46ecb58a5 added dll compilation tests 2016-12-17 16:28:12 +01:00
Nick Terrell 8de46ab51a Export all API functions 2016-12-16 13:27:30 -08:00
Przemyslaw Skibinski 0a1caeef6d VS: fixed 32-bit DLL compilation 2016-12-15 12:12:46 +01:00
Przemyslaw Skibinski 4e10bd339d appveyor.yml: added tests of fullbench-dll fullbench-lib 2016-12-15 12:09:23 +01:00
Przemyslaw Skibinski 60f10aab6c introduced ZSTDLIB_VISIBILITY 2016-12-15 11:32:31 +01:00
Yann Collet 2b36b238d3 changed variable name to estimatedSrcSize, to emphasize it does not need to be exact 2016-12-13 17:59:55 +01:00
Yann Collet e795c8a5f6 Added ZSTD_initCStream_srcSize().
Added relevant test cases in zstreamtest
2016-12-13 17:00:14 +01:00
Yann Collet 5397a66b19 minor BMI version check 2016-12-13 15:21:06 +01:00
Yann Collet 35168679bd Merge pull request #478 from terrelln/wildcopy-ub
Fix execSequence wildcopy undefined behavior
2016-12-13 11:33:00 +01:00
Nick Terrell 064a143520 Fix execSequence wildcopy undefined behavior
execSequence relied on pointer overflow to handle cases where
`sequence.matchLength < 8`.  Instead of passing an `size_t` to
wildcopy, pass a `ptrdiff_t`.
2016-12-12 19:01:23 -08:00
Nick Terrell e474aa55b4 Fix decompression buffer overrun
Allows an adversary to write up to 3 bytes beyond the end of the buffer.
Occurs if the match overlaps the `extDict` and `currentPrefix`, and the
match length in the `currentPrefix` is less than `MINMATCH`, and
`op-(16-MINMATCH) >= oMatchEnd > op-16`.
2016-12-12 18:05:30 -08:00
Yann Collet c3a5c4bef8 introduced cycleLog 2016-12-12 00:47:30 +01:00
Yann Collet c261f71f6a minor variation of rescale fix 2016-12-12 00:25:07 +01:00
Nick Terrell 3826207a70 Simplify segfault fix
Take advantage of the fact that `chainLog <= windowLog`.
2016-12-10 18:46:55 -08:00
Nick Terrell 0012332ce0 Fix compression segfault
When the overflow protection kicks in, it makes sure that ip - ctx->base
isn't too large.  However, it didn't ensure that saved offsets are
still valid.  This change ensures that any valid offsets (<= windowLog)
are still representable after the update.

The bug would shop up on line 1056, when `offset_1 > current + 1`, which
causes an underflow.  This in turn, would cause a segfault on line 1063.

The input must necessarily be longer than 1 GB for this issue to occur.
Even then, it only occurs if one of the last 3 matches is larger than
the chain size and block size.
2016-12-09 17:15:33 -08:00
Yann Collet 383b8088a3 minor lib build refactoring 2016-12-08 18:42:27 -08:00
Yann Collet 6e754fe76a fixed lib soname.
example : simple_compression : size overflow check
2016-12-08 18:26:56 -08:00
Przemyslaw Skibinski 7687913178 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2016-12-08 10:42:42 +01:00
Yann Collet 426a9d4b71 changed : dll : only approved ZSTD symbols are now exposed. All other symbols remain internal. 2016-12-07 16:39:34 -08:00