facebook/zstd
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,801 Commits 2 Branches 0 Tags
Commit Graph

555 Commits

Author SHA1 Message Date
Yann Collet
77d67fb167 Merge pull request #766 from terrelln/real-block-split 
[libzstd] Pull optimal parser state out of seqStore_t
 2017-07-18 08:26:24 -07:00
Yann Collet
14c83b05c7 Merge pull request #765 from terrelln/real-block-split 
[libzstd] Remove ZSTD_CCtx* argument of ZSTD_compressSequences()
 2017-07-17 19:25:55 -07:00
Nick Terrell
7a28b9e4a3 [libzstd] Pull optimal parser state out of seqStore_t 2017-07-17 15:29:11 -07:00
Yann Collet
3381bf4b84 Merge pull request #764 from terrelln/real-block-split 
[libzstd] Refactor ZSTD_compressSequences()
 2017-07-17 14:46:01 -07:00
Nick Terrell
e198230645 [libzstd] Remove ZSTD_CCtx* argument of ZSTD_compressSequences() 2017-07-17 12:27:24 -07:00
Nick Terrell
634f012420 [libzstd] Refactor ZSTD_compressSequences() 2017-07-17 11:36:11 -07:00
Paul Cruz
50ce4eaeb6 added error detection for pthread initialization, added compression completion measurement, fixed const values 2017-07-17 10:12:44 -07:00
Yann Collet
2bd6440be0 pinned down error code enum values 
Note : all error codes are changed by this new version,
but it's expected to be the last change for existing codes.

Codes are now grouped by category, and receive a manually attributed value.
The objective is to guarantee that
error code values will not change in the future
when introducing new codes.
Intentionnal empty spaces and ranges are defined
in order to keep room for potential new codes.
 2017-07-13 17:12:16 -07:00
Nick Terrell
830ef4152a [libzstd] Increase granularity of FSECTable repeat mode 2017-07-13 12:45:39 -07:00
Yann Collet
d985319337 Merge pull request #759 from terrelln/real-block-split 
[libzstd] Pull CTables into sub-structure
 2017-07-13 10:24:19 -07:00
Nick Terrell
de0414b736 [libzstd] Pull CTables into sub-structure 2017-07-12 19:49:19 -07:00
Yann Collet
88da8f1816 fix : propagate custom allocator to ZSTDMT though ZSTD_CCtx_setParameter() 
also : compile fuzzer with MT enabled
 2017-07-10 14:02:33 -07:00
Yann Collet
2cb9774f5e more precise estimation of amount to flush at end of stream (single thread mode) 
also : can use DEBUGLEVEL variable in /tests
 2017-07-04 12:39:26 -07:00
Yann Collet
2084b041f4 fixed comments 2017-07-03 15:52:19 -07:00
Yann Collet
5a77361595 fixed wrong function name in comment 2017-07-03 15:21:24 -07:00
Yann Collet
d5c046c609 implemented shortcut for zstd_compress_generic() in MT mode 
added ZSTDMT_compress_advanced() API
 2017-06-30 14:51:01 -07:00
Yann Collet
a3d9926c40 compression optimization opportunity 
switch to single-pass mode directly into output buffer
when outputSize >= ZSTD_compressBound(inputSize).
Speed gains observed with fullbench (~+15% on level 1)
 2017-06-29 14:44:49 -07:00
Yann Collet
037466245f refactor ZSTD_check_compressionLevel_monotonicIncrease_memoryBudget() 
use less macro statements
the initial version was meant to work with STATIC_ASSERT
but since it doesn't work and needs assert()
it's possible to rewrite it using normally compiled code
which is better for compiler.

Downside : the error message is less precise.
There is a DEBUGLOG(3,) to compensate.
 2017-06-28 20:24:08 -07:00
Yann Collet
2bf428df45 Merge branch 'advancedAPI2' into refPrefix 2017-06-28 16:35:49 -07:00
Yann Collet
1ca76039af fixed -Wdeclaration-after-statement 2017-06-28 15:40:21 -07:00
Yann Collet
813535105b added function to control monotonic memory budget increase of ZSTD_defaultCParameters[0] 
It's a runtime test, based on assert(),
played once, on first ZSTD_getCParams() usage,
when ZSTD_DEBUG is enabled.
 2017-06-28 15:34:56 -07:00
Yann Collet
adbe74a8ac adjusted compression levels to guarantee a monotonically increasing memory budget 2017-06-28 13:22:37 -07:00
Yann Collet
33a6639039 fixed ZSTD_refPrefix with Multithread-enabled CCtx 2017-06-28 11:09:43 -07:00
Yann Collet
2e4274262d controlled dictMode 2017-06-27 17:09:12 -07:00
Yann Collet
b7372933b8 implemented ZSTD_refPrefix() 2017-06-27 15:49:12 -07:00
Yann Collet
7d3816183f exposed ZSTD_MAGIC_DICTIONARY in zstd.h 
makes it easier to explain ZSTD_dictMode
 2017-06-27 13:50:34 -07:00
Yann Collet
fecc721fd9 added parameter ZSTD_p_refDictContent 2017-06-27 11:46:39 -07:00
Yann Collet
dde10b23fe refactored ZSTD_estimateDStreamSize() 
now uses windowSize as argument.
Also : created ZSTD_estimateDStreamSize_fromFrame()
 2017-06-26 17:44:26 -07:00
Yann Collet
09ae03a570 ZSTD_estimateCDictSize_advanced() 
ZSTD_estimateCDictSize() now uses same arguments as ZSTD_createCDict()
ZSTD_estimateCDictSize_advanced() uses same arguments as ZSTD_createCDict_advanced()
 2017-06-26 16:47:32 -07:00
Yann Collet
0c9a915a28 ZSTD_estimateCStreamSize_advanced() 2017-06-26 16:02:25 -07:00
Yann Collet
31af8290d1 ZSTD_estimateCCtx_advanced() 
ZSTD_estimateCCtx() is now a "simple" function,
taking int compressionLevel as single argument.

ZSTD_estimateCCtx_advanced() takes a CParams argument,
which is both more complete and more complex to generate.
 2017-06-26 15:52:39 -07:00
Yann Collet
ef269c1b68 Merge pull request #725 from facebook/advancedAPI2 
New Advanced API
 2017-06-23 09:50:47 -07:00
Yann Collet
ecb0f46866 add controls over streaming buffers 2017-06-21 17:25:01 -07:00
Yann Collet
204b6b7ef6 fixed streaming buffered allocation with CDict compression 2017-06-21 15:13:00 -07:00
Yann Collet
7bd1a2900e added ZSTD_dictMode_e to control dictionary loading mode 2017-06-21 11:50:33 -07:00
Yann Collet
e51d51bdf7 fixed memcpy() overlap 2017-06-20 17:44:55 -07:00
Yann Collet
466f92eaa6 removed one useless streaming compression stage, detected by @terrelln 2017-06-20 16:25:29 -07:00
Yann Collet
c3bce24ef4 fixed potential dangling pointer, detected by @terrelln 2017-06-20 16:09:11 -07:00
Yann Collet
b44ab82f7a ensure new ZSTD_strategy starts at value 1 2017-06-20 14:11:49 -07:00
Yann Collet
c08e649e95 first implementation of bench.c with new API ZSTD_compress_generic() 
Doesn't speed optimize this buffer-to-buffer scenario yet.
Still internally defers to streaming implementation.

Also : fixed a long standing bug in ZSTDMT streaming API.
 2017-06-19 18:25:35 -07:00
Nick Terrell
55f9cd4942 [libzstd] Fix UBSAN failure 2017-06-19 15:12:28 -07:00
Yann Collet
bf99150be3 update new api presentation in zstd.h and manual 2017-06-19 12:56:25 -07:00
Yann Collet
d7a3bffba9 new api : setting compression parameters is refused if a dictionary is already loaded 2017-06-19 11:53:01 -07:00
Yann Collet
381e66cfbd added ZSTD_clampCParams() 
now ZSTD_adjustCParams() is always successful,
it always produces a valid CParams
 2017-06-16 17:34:54 -07:00
Yann Collet
aee916e37c fixed +/-1 error for pledgedSrcSizePlusOne 2017-06-16 17:02:35 -07:00
Yann Collet
d3de3d51a3 fix attempt 2 : Visual sign conversion warning 2017-06-16 16:51:33 -07:00
Yann Collet
944be54774 fixed attempt : minor Visual sign conversion warning 2017-06-16 14:05:01 -07:00
Yann Collet
b26728c9c8 added ZSTD_startNewCompression() 2017-06-16 14:00:46 -07:00
Yann Collet
a0ba849fe6 changed frameContentSize field to pledgedSrcSizePlusOne 
pledgedSrcSize is proper : it's a promise, not yet fulfilled.
It will be controlled at the end.

PlusOne is meant to have 0 (default) == unknown
 2017-06-16 13:29:17 -07:00
Yann Collet
2cf7755da7 fix : pledgedSrcSize correctly reset to unknown in "continue" mode 2017-06-16 12:34:41 -07:00