ee5b725823
ZSTD_initCStream() optimization : do not allocate a CDict when no dictionary used
2016-10-27 14:20:55 -07:00
335ad5d4d4
added ZSTD_initDStream_usingDDict() .
...
slightly optimized ZSTD_initDStream() when no dictionary .
fixed ZSTD_sizeof_CStream() .
2016-10-25 17:47:02 -07:00
9516234e67
first sketch for ZSTD_initCStream_usingCDict()
2016-10-25 16:19:52 -07:00
62d9a7ddfd
Merge pull request #429 from inikep/btopt2
...
Btopt2
2016-10-25 14:48:43 -07:00
5c5f01f3da
added ZSTD_btopt2 strategy
2016-10-25 12:25:07 +02:00
b2c39a22b0
Fix compiler narrowing warning
2016-10-24 14:50:13 -07:00
f698ad6deb
Merge remote-tracking branch 'upstream/dev' into fixes
...
* upstream/dev:
added doc\zstd_manual.html
added contrib\gen_html
zstd_compression_format.md moved to doc/
Fix small bug in ZSTD_execSequence()
improved ZSTD_compressBlock_opt_extDict_generic
protect ZSTD_decodeFrameHeader() from invalid usage, as suggested by @spaskob
zstd_opt.h: small improvement in compression ratio
improved dicitonary segment merge
use implicit rules to compile zstd_decompress.c
detect early impossible decompression scenario in legacy decoder v0.5
no repeat mode in legacy v0.5
fixed invalid invocation of dictionary in legacy decoder v0.5
fix edge case
fix command line interpretation
fixed minor corner case
zstd.h: added the Introduction section
fixed clang 3.5 warnings
zstd.h: updated comments
2016-10-24 13:10:13 -07:00
f9c9af3c2e
Reject dictionaries with incomplete entropy tables
...
If a dictionary specifies that a symbol has probability zero in its
`matchLength`, `literalLength`, or `offset` FSE table, but the symbol
appears when compressing input, the compressor fails.
Ensure that dictionaries support all `matchLength`, and `literalLength`
codes. They must also support all of the `offset` codes required to
represent every possible offset that can appear in the first block.
2016-10-24 10:42:44 -07:00
3ee94a7600
zstd_compression_format.md moved to doc/
2016-10-24 15:58:07 +02:00
4732074a71
improved ZSTD_compressBlock_opt_extDict_generic
2016-10-21 11:19:00 +02:00
d365ae3497
zstd_opt.h: small improvement in compression ratio
2016-10-20 11:49:02 +02:00
197a55ee7b
fix edge case
2016-10-18 11:27:52 -07:00
fd98087047
Fix stack buffer overflow in HUF_readCTable()
...
If `w ==0` on line 153, then `CTable[n].nbBits == tableLog + 1`.
Then `nbPerRank[CTable[n].nbBits]` and `valPerRank[CTable[n].nbBits]`
are stack buffer overflows.
2016-10-17 18:16:59 -07:00
bfd943ace5
Fix buffer overrun in ZSTD_loadDictEntropyStats()
...
The table log set by `FSE_readNCount()` was not checked in
`ZSTD_loadDictEntropyStats()`. This caused `FSE_buildCTable()`
to stack/heap overflow in a few places.
The benchmarks look good, there is no obvious compression performance regression:
> ./zstds/zstd.opt.0 -i10 -b1 -e10 ~/bench/silesia.tar
1#silesia.tar : 211988480 -> 73656930 (2.878), 271.6 MB/s , 716.8 MB/s
2#silesia.tar : 211988480 -> 70162842 (3.021), 204.8 MB/s , 671.1 MB/s
3#silesia.tar : 211988480 -> 66997986 (3.164), 156.8 MB/s , 658.6 MB/s
4#silesia.tar : 211988480 -> 66002591 (3.212), 136.4 MB/s , 665.3 MB/s
5#silesia.tar : 211988480 -> 65008480 (3.261), 98.9 MB/s , 647.0 MB/s
6#silesia.tar : 211988480 -> 62979643 (3.366), 65.2 MB/s , 670.4 MB/s
7#silesia.tar : 211988480 -> 61974560 (3.421), 44.9 MB/s , 688.2 MB/s
8#silesia.tar : 211988480 -> 61028308 (3.474), 32.4 MB/s , 711.9 MB/s
9#silesia.tar : 211988480 -> 60416751 (3.509), 21.1 MB/s , 718.1 MB/s
10#silesia.tar : 211988480 -> 60174239 (3.523), 22.2 MB/s , 721.8 MB/s
> ./compress_zstds/zstd.opt.1 -i10 -b1 -e10 ~/bench/silesia.tar
1#silesia.tar : 211988480 -> 73656930 (2.878), 273.8 MB/s , 722.0 MB/s
2#silesia.tar : 211988480 -> 70162842 (3.021), 203.2 MB/s , 666.6 MB/s
3#silesia.tar : 211988480 -> 66997986 (3.164), 157.4 MB/s , 666.5 MB/s
4#silesia.tar : 211988480 -> 66002591 (3.212), 132.1 MB/s , 661.9 MB/s
5#silesia.tar : 211988480 -> 65008480 (3.261), 96.8 MB/s , 641.6 MB/s
6#silesia.tar : 211988480 -> 62979643 (3.366), 63.1 MB/s , 677.0 MB/s
7#silesia.tar : 211988480 -> 61974560 (3.421), 44.3 MB/s , 678.2 MB/s
8#silesia.tar : 211988480 -> 61028308 (3.474), 33.1 MB/s , 708.9 MB/s
9#silesia.tar : 211988480 -> 60416751 (3.509), 21.5 MB/s , 710.1 MB/s
10#silesia.tar : 211988480 -> 60174239 (3.523), 21.9 MB/s , 723.9 MB/s
2016-10-17 16:55:52 -07:00
2b361cf2f1
minor opt
2016-10-14 16:09:07 -07:00
3b9cdf9220
Fix ubsan failures (pass NULL to memcpy)
2016-10-12 20:54:42 -07:00
cf409a7e2a
fixed : init*_advanced() followed by reset() with different pledgedSrcSiz
2016-09-26 16:41:05 +02:00
97b378a6f8
Streaming : dictionary compression on multiple files / segments can correctly provide srcSize into header (when provided) using pledgedSrcSize.
2016-09-21 17:20:19 +02:00
993060e0f2
cli : better adaptation to small files
2016-09-21 16:46:08 +02:00
a6bdf55759
fixed memory leak
2016-09-15 17:02:06 +02:00
644a8da88a
fixed minor conversion warning
2016-09-15 16:16:21 +02:00
4cb212938c
introduced ZSTD_resetCStream()
2016-09-15 14:54:07 +02:00
fa0c09760c
variable renaming
2016-09-15 14:11:01 +02:00
d7c6589df8
support ZSTD_sizeof_*() on NULL
...
added ZSTD_sizeof_CDict()
2016-09-15 02:57:27 +02:00
64deef3bee
Fixed srcSize=1
2016-09-14 00:16:07 +02:00
ac8bace6b1
support large skippable frames
2016-09-07 14:54:23 +02:00
95d07d7447
introduced CHECK_E
2016-09-06 16:38:51 +02:00
3e21ec5b01
introduced CHECK_F
2016-09-06 15:36:19 +02:00
5c956d593c
FORCE_INLINE common definition
2016-09-06 15:05:19 +02:00
edbcd9f5b2
fixed zbufftest
2016-09-06 14:30:57 +02:00
b624922b14
fixed checksum
2016-09-06 11:16:57 +02:00
a7737f6a60
improved compression on small files when using same parameters
2016-09-06 09:44:59 +02:00
7ae67bb18a
small compression speed gains with using_CDict
2016-09-06 06:28:05 +02:00
855766d73d
clarified dictionary in format description
2016-09-02 17:04:49 -07:00
1563bfeabc
fixing FORCE_INLINE for older compilers ( #330 )
2016-09-02 11:44:21 -07:00
e10f7f3dcb
merge
2016-08-30 12:03:36 -07:00
4ded9e591c
added boilerplate
2016-08-30 11:06:28 -07:00
14200a20f0
Fixed issue #304 , reported by @borzunov
2016-08-30 06:51:00 -07:00
da9d3b7057
Cleanup some errors in typedef comments and remove duplicated HOWTO from zbuff_decompress.c
2016-08-29 17:31:51 -07:00
23b6e05d8e
ZSTD_malloc() and ZSTD_free(), to simplify customMem
2016-08-28 21:05:43 -07:00
e19a9ef05d
update compression level table
2016-08-26 20:02:49 +02:00
87c18b2ebd
fixed multiple minor warnings for XCode
2016-08-26 01:43:47 +02:00
0d59a6f73a
removed debug strings
2016-08-25 22:42:46 +02:00
5a02b69215
reinforced fix for huge files
2016-08-25 22:24:59 +02:00
96bdd87de4
fixed : compression bug on very large files
2016-08-25 14:34:42 +02:00
a3a47ec4d0
Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into Other
2016-08-24 21:25:49 +02:00
a2cdffe556
fixed wrong parameter issue
2016-08-24 19:42:15 +02:00
e416e30019
remove unnecessary comments
2016-08-24 17:32:09 +02:00
4e90f6c1e0
removed ZSTD_LOG_ENCODE and ZSTD_LOG_BLOCK
2016-08-24 17:24:11 +02:00
83388e109f
removed ZSTD_LOG_PARSER
2016-08-24 17:22:20 +02:00
Yann Collet
Yann Collet
Przemyslaw Skibinski
Nick Terrell
Yann Collet
David Lam