7b06ad7a05
Backport fix from commit 125d817
...
This fixes a read of unitialized memory.
Full commit hash: 125d81774f
2016-10-18 14:52:34 -07:00
f45b157d95
Backport fix from commit 9e8b09a
...
Fixes uninitialized memory reads.
Full commit hash: 9e8b09a7bd
2016-10-18 14:22:49 -07:00
f7906d5955
detect early impossible decompression scenario in legacy decoder v0.5
2016-10-18 13:48:32 -07:00
9313c8d953
no repeat mode in legacy v0.5
2016-10-18 13:36:15 -07:00
83d7bdee4b
fixed invalid invocation of dictionary in legacy decoder v0.5
2016-10-18 12:25:43 -07:00
197a55ee7b
fix edge case
2016-10-18 11:27:52 -07:00
fd98087047
Fix stack buffer overflow in HUF_readCTable()
...
If `w ==0` on line 153, then `CTable[n].nbBits == tableLog + 1`.
Then `nbPerRank[CTable[n].nbBits]` and `valPerRank[CTable[n].nbBits]`
are stack buffer overflows.
2016-10-17 18:16:59 -07:00
06573e17be
fixed minor corner case
2016-10-17 17:28:28 -07:00
bfd943ace5
Fix buffer overrun in ZSTD_loadDictEntropyStats()
...
The table log set by `FSE_readNCount()` was not checked in
`ZSTD_loadDictEntropyStats()`. This caused `FSE_buildCTable()`
to stack/heap overflow in a few places.
The benchmarks look good, there is no obvious compression performance regression:
> ./zstds/zstd.opt.0 -i10 -b1 -e10 ~/bench/silesia.tar
1#silesia.tar : 211988480 -> 73656930 (2.878), 271.6 MB/s , 716.8 MB/s
2#silesia.tar : 211988480 -> 70162842 (3.021), 204.8 MB/s , 671.1 MB/s
3#silesia.tar : 211988480 -> 66997986 (3.164), 156.8 MB/s , 658.6 MB/s
4#silesia.tar : 211988480 -> 66002591 (3.212), 136.4 MB/s , 665.3 MB/s
5#silesia.tar : 211988480 -> 65008480 (3.261), 98.9 MB/s , 647.0 MB/s
6#silesia.tar : 211988480 -> 62979643 (3.366), 65.2 MB/s , 670.4 MB/s
7#silesia.tar : 211988480 -> 61974560 (3.421), 44.9 MB/s , 688.2 MB/s
8#silesia.tar : 211988480 -> 61028308 (3.474), 32.4 MB/s , 711.9 MB/s
9#silesia.tar : 211988480 -> 60416751 (3.509), 21.1 MB/s , 718.1 MB/s
10#silesia.tar : 211988480 -> 60174239 (3.523), 22.2 MB/s , 721.8 MB/s
> ./compress_zstds/zstd.opt.1 -i10 -b1 -e10 ~/bench/silesia.tar
1#silesia.tar : 211988480 -> 73656930 (2.878), 273.8 MB/s , 722.0 MB/s
2#silesia.tar : 211988480 -> 70162842 (3.021), 203.2 MB/s , 666.6 MB/s
3#silesia.tar : 211988480 -> 66997986 (3.164), 157.4 MB/s , 666.5 MB/s
4#silesia.tar : 211988480 -> 66002591 (3.212), 132.1 MB/s , 661.9 MB/s
5#silesia.tar : 211988480 -> 65008480 (3.261), 96.8 MB/s , 641.6 MB/s
6#silesia.tar : 211988480 -> 62979643 (3.366), 63.1 MB/s , 677.0 MB/s
7#silesia.tar : 211988480 -> 61974560 (3.421), 44.3 MB/s , 678.2 MB/s
8#silesia.tar : 211988480 -> 61028308 (3.474), 33.1 MB/s , 708.9 MB/s
9#silesia.tar : 211988480 -> 60416751 (3.509), 21.5 MB/s , 710.1 MB/s
10#silesia.tar : 211988480 -> 60174239 (3.523), 21.9 MB/s , 723.9 MB/s
2016-10-17 16:55:52 -07:00
4db751668f
Fix buffer overrun in ZSTD_loadEntropy()
...
The table log set by `FSE_readNCount()` was not checked in
`ZSTD_loadEntropy()`. This caused `FSE_buildDTable(dctx->MLTable, ...)`
to overwrite the beginning of `dctx->hufTable`.
The benchmarks look good, there is no obvious performance regression:
> ./zstds/zstd.opt.0 -i10 -b1 -e5 ~/bench/silesia.tar
1#silesia.tar : 211988480 -> 73656930 (2.878), 268.2 MB/s , 701.0 MB/s
2#silesia.tar : 211988480 -> 70162842 (3.021), 199.5 MB/s , 666.9 MB/s
3#silesia.tar : 211988480 -> 66997986 (3.164), 154.9 MB/s , 655.6 MB/s
4#silesia.tar : 211988480 -> 66002591 (3.212), 128.9 MB/s , 648.4 MB/s
5#silesia.tar : 211988480 -> 65008480 (3.261), 98.4 MB/s , 633.4 MB/s
> ./zstds/zstd.opt.2 -i10 -b1 -e5 ~/bench/silesia.tar
1#silesia.tar : 211988480 -> 73656930 (2.878), 266.1 MB/s , 703.7 MB/s
2#silesia.tar : 211988480 -> 70162842 (3.021), 199.0 MB/s , 666.6 MB/s
3#silesia.tar : 211988480 -> 66997986 (3.164), 156.2 MB/s , 656.2 MB/s
4#silesia.tar : 211988480 -> 66002591 (3.212), 133.2 MB/s , 647.4 MB/s
5#silesia.tar : 211988480 -> 65008480 (3.261), 96.3 MB/s , 633.3 MB/s
2016-10-17 15:51:15 -07:00
ccfcc643da
Check if dict is empty before reading first byte
2016-10-17 11:46:03 -07:00
2b361cf2f1
minor opt
2016-10-14 16:09:07 -07:00
7933434fdf
Merge branch 'dev' of github.com:facebook/zstd into dev
2016-10-14 13:32:35 -07:00
d4cda27b63
new command -M#, to limit memory usage during decompression ( #403 )
2016-10-14 13:32:20 -07:00
3b9cdf9220
Fix ubsan failures (pass NULL to memcpy)
2016-10-12 20:54:42 -07:00
5d919e7ac3
added ZSTD_error_frameParameter_windowTooLarge ( #403 )
2016-10-12 17:29:24 -07:00
e19111c42f
make creates libzstd binaries ( #415 )
2016-10-12 11:09:36 -07:00
8b70d012f0
fix cmake
2016-10-12 10:23:53 -07:00
38fb0dc4cf
Merge pull request #416 from terrelln/exec-sequence
...
Fix ZSTD_execSequence() edge case
2016-10-12 10:17:53 -07:00
7158584399
Fix ZSTD_execSequence() edge case
2016-10-12 10:05:26 -07:00
f52cd03e73
bumped version number
2016-10-11 17:29:27 -07:00
ef2357d0d3
created error_private.c, so that a single list of error strings get included
2016-10-11 17:24:50 -07:00
14efab827b
added zstd_errors.h to include installation
2016-10-11 16:51:29 -07:00
a17fd7312a
changed error_public.h into zstd_errors.h
2016-10-11 16:41:09 -07:00
18b51b99c0
sync fse
2016-10-11 08:21:09 -07:00
2d2613399a
zstd.h: added the Introduction section
2016-10-06 16:28:21 +02:00
ba1db376ac
fixed clang 3.5 warnings
2016-10-06 14:22:48 +02:00
82057aa7ec
zstd.h: updated comments
2016-10-06 13:23:52 +02:00
df6797447f
update dictionary builder warning comments
2016-09-27 15:14:32 +02:00
47094ea66b
added comment on filePos
2016-09-26 18:03:33 +02:00
cf409a7e2a
fixed : init*_advanced() followed by reset() with different pledgedSrcSiz
2016-09-26 16:41:05 +02:00
2f2639438a
zstreamtest can fuzztest pledgedSrcSize
2016-09-26 14:06:08 +02:00
dc245e91cb
Changed to use ZSTDLIBv06_API and ZSTDLIBv07_API for DLL exports to fix warning
...
- changed name to prevent collision with ZSTDLIB_API used by non-legacy dll exports
2016-09-23 17:09:36 +02:00
21412bb3f6
Merge branch 'dev' of github.com:Cyan4973/zstd into dev
2016-09-22 15:57:56 +02:00
51f4d566c2
small decompression speed boost for very small data
2016-09-22 15:57:28 +02:00
97b378a6f8
Streaming : dictionary compression on multiple files / segments can correctly provide srcSize into header (when provided) using pledgedSrcSize.
2016-09-21 17:20:19 +02:00
993060e0f2
cli : better adaptation to small files
2016-09-21 16:46:08 +02:00
1eb2fdc74f
bumped version number
2016-09-18 12:21:47 +02:00
a6bdf55759
fixed memory leak
2016-09-15 17:02:06 +02:00
644a8da88a
fixed minor conversion warning
2016-09-15 16:16:21 +02:00
4cb212938c
introduced ZSTD_resetCStream()
2016-09-15 14:54:07 +02:00
fa0c09760c
variable renaming
2016-09-15 14:11:01 +02:00
d7c6589df8
support ZSTD_sizeof_*() on NULL
...
added ZSTD_sizeof_CDict()
2016-09-15 02:57:27 +02:00
e91c4b4cef
introduced ZSTD_resetDStream() .
...
added : ZSTD_sizeof_DDict()
2016-09-14 16:55:44 +02:00
d092d77cfc
minor variable renaming
2016-09-14 16:14:57 +02:00
64deef3bee
Fixed srcSize=1
2016-09-14 00:16:07 +02:00
26ec254066
new strategy for faster DDict decompression
2016-09-13 16:52:16 +02:00
ac175d46d4
updated comments
2016-09-13 00:51:47 +02:00
a3481d6de0
make uninstall
2016-09-12 05:04:26 +02:00
b3060f7a9e
changed streaming decoder behavior : now, when all compressed frame is consumed, it means decompression is completed, with regenerated data fully flushed.
2016-09-09 16:44:16 +02:00
01c199226a
updated decompression streaming example
2016-09-08 19:29:04 +02:00
5c6d244973
Merge branch 'dev' of github.com:facebook/zstd into dev
2016-09-07 14:54:54 +02:00
ac8bace6b1
support large skippable frames
2016-09-07 14:54:23 +02:00
0e07bf3f60
added comments on searchLength min / max ( #337 )
2016-09-07 06:33:02 +02:00
95d07d7447
introduced CHECK_E
2016-09-06 16:38:51 +02:00
3e21ec5b01
introduced CHECK_F
2016-09-06 15:36:19 +02:00
5c956d593c
FORCE_INLINE common definition
2016-09-06 15:05:19 +02:00
edbcd9f5b2
fixed zbufftest
2016-09-06 14:30:57 +02:00
b624922b14
fixed checksum
2016-09-06 11:16:57 +02:00
a7737f6a60
improved compression on small files when using same parameters
2016-09-06 09:44:59 +02:00
7ae67bb18a
small compression speed gains with using_CDict
2016-09-06 06:28:05 +02:00
1d4208c029
clarified streaming decompression inlined doc
2016-09-06 05:16:40 +02:00
7c83dfd5c2
ZSTD_frameHeaderSize_prefix ( #340 ), as result of ZSTD_initStream
2016-09-05 19:47:43 +02:00
fa72f6bdce
clarified inline doc for streaming
2016-09-05 17:39:56 +02:00
c73a8109bb
Merge pull request #344 from inikep/dev10
...
unified error codes for legacy decoders
2016-09-05 07:46:33 -07:00
45db83f98d
ZSTD_decodeLiteralsBlock renamed to ZSTDv01_decodeLiteralsBlock
2016-09-05 14:46:24 +02:00
476964f6a1
ZSTD_decodeSeqHeaders renamed to ZSTDv01_decodeSeqHeaders
2016-09-05 13:34:57 +02:00
c13faa1b0f
legacy decoders: restored #include <intrin.h> for VC++
2016-09-05 13:25:07 +02:00
8161e7321a
unified error codes for legacy decoders
2016-09-05 12:29:51 +02:00
b85cdabd50
Enable install targets for NetBSD.
2016-09-04 14:37:57 +02:00
33a0465a51
fixed a few links
2016-09-02 22:11:49 -07:00
d56dbc02d3
removed g_displayLevel
2016-09-02 17:28:41 -07:00
855766d73d
clarified dictionary in format description
2016-09-02 17:04:49 -07:00
d725427a3c
g_time => local displayTime
2016-09-02 15:32:39 -07:00
1563bfeabc
fixing FORCE_INLINE for older compilers ( #330 )
2016-09-02 11:44:21 -07:00
7304eb7c09
bumped version number
2016-09-01 15:49:26 -07:00
901e85fe26
version bump
2016-08-31 07:51:25 -07:00
1c59c20903
removed redundant files
2016-08-31 07:15:44 -07:00
599c69d917
minor Makefile updates
2016-08-30 13:33:20 -07:00
e10f7f3dcb
merge
2016-08-30 12:03:36 -07:00
4ded9e591c
added boilerplate
2016-08-30 11:06:28 -07:00
3b15f1f10f
minor refactor
2016-08-30 09:58:50 -07:00
240795bef7
Merge branch 'dev' of github.com:Cyan4973/zstd into dev
2016-08-30 06:51:55 -07:00
14200a20f0
Fixed issue #304 , reported by @borzunov
2016-08-30 06:51:00 -07:00
da9d3b7057
Cleanup some errors in typedef comments and remove duplicated HOWTO from zbuff_decompress.c
2016-08-29 17:31:51 -07:00
09c3c8e885
Merge pull request #307 from inikep/dev08
...
updated README.md
2016-08-29 16:32:33 -07:00
6416b0d705
updated README.md
2016-08-29 13:04:26 +02:00
23b6e05d8e
ZSTD_malloc() and ZSTD_free(), to simplify customMem
2016-08-28 21:05:43 -07:00
5f53b0335e
fixed continuation context
2016-08-28 10:00:49 -07:00
767d8f66fa
legacy contexts can be re-used
2016-08-28 08:19:47 -07:00
4bf317dd00
first version supporting legacy streams (transparent decoding)
2016-08-28 07:43:34 -07:00
e19a9ef05d
update compression level table
2016-08-26 20:02:49 +02:00
9a021c1aae
fixed some minor clang warnings
2016-08-26 09:05:06 +02:00
cb5a320705
made -Wdocumentation a clang only flag
2016-08-26 08:06:36 +02:00
87c18b2ebd
fixed multiple minor warnings for XCode
2016-08-26 01:43:47 +02:00
0d59a6f73a
removed debug strings
2016-08-25 22:42:46 +02:00
5a02b69215
reinforced fix for huge files
2016-08-25 22:24:59 +02:00
96bdd87de4
fixed : compression bug on very large files
2016-08-25 14:34:42 +02:00
a3a47ec4d0
Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into Other
2016-08-24 21:25:49 +02:00
a2cdffe556
fixed wrong parameter issue
2016-08-24 19:42:15 +02:00
e416e30019
remove unnecessary comments
2016-08-24 17:32:09 +02:00
4e90f6c1e0
removed ZSTD_LOG_ENCODE and ZSTD_LOG_BLOCK
2016-08-24 17:24:11 +02:00
83388e109f
removed ZSTD_LOG_PARSER
2016-08-24 17:22:20 +02:00
8a36f8527c
removed stats in debug mode
2016-08-24 17:19:12 +02:00
57ef4b1a0d
zstd_v07.c: removed unused macros
2016-08-24 17:16:56 +02:00
24b68a5b23
update cLevel table for 256KB
2016-08-24 14:22:26 +02:00
c54692faeb
improved level 3
2016-08-24 01:45:46 +02:00
17e482efdd
added ZSTD_setDStreamParameter()
2016-08-23 16:58:10 +02:00
3071c3e303
STREAM_WINDOW_MAX : protect streaming from unreasonable memory requirements
2016-08-23 01:34:34 +02:00
70e3b31306
fixed playtests on os-x
2016-08-23 01:18:06 +02:00
cb3276329a
added sizeof CStream and DStream
2016-08-23 00:31:59 +02:00
d1733f7417
fixed crc bug in rare timing conditions within bench.c
2016-08-21 01:04:46 +02:00
8baf78a291
minor coding style
2016-08-20 13:04:20 +02:00
1bee2d5e08
slight decompression speed improvement
2016-08-20 02:59:04 +02:00
0cfe2ec2fd
sync fse version
2016-08-20 00:26:26 +02:00
af1960396b
sync huff0
2016-08-19 19:38:19 +02:00
7be46bf8f2
promoted streaming API to stable
...
(except _advanced() variants)
2016-08-19 18:39:36 +02:00
fdba57d513
update version number
2016-08-19 18:32:30 +02:00
da3fbcb302
Added ZDICT_getDictID()
2016-08-19 14:23:58 +02:00
a5dbf9f629
Merge pull request #297 from borzunov/dev
...
Export functions related to dictionary compression from DLL
2016-08-18 15:05:01 +02:00
49d105cfcf
better warning and error messages in case of dictionary training failure ( #292 )
2016-08-18 15:02:11 +02:00
0f6f17a14f
Rename ZSTDLIB_API to ZDICTLIB_API in zdict.h
2016-08-18 16:47:06 +05:00
1f48382b1a
Export functions related to dictionary compression from DLL
2016-08-18 16:12:49 +05:00
e80d15304a
Merge pull request #296 from inikep/Other
...
Other
2016-08-18 11:48:48 +02:00
a7bb322a93
removed never referenced functions
2016-08-18 10:30:21 +02:00
18442c1482
minor refactoring
2016-08-18 01:40:32 +02:00
c411902230
fixed g++ conversion warning
2016-08-17 01:50:54 +02:00
53e17fbd5e
updated streaming API
2016-08-17 01:39:22 +02:00
655393cc72
updated doc for streaming API
2016-08-16 15:11:28 +02:00
104e5b072d
added : streaming decompression API
2016-08-16 15:11:28 +02:00
5a0c8e2439
new streaming API (compression)
2016-08-16 15:11:27 +02:00
ba92046031
Merge branch 'dev' of github.com:Cyan4973/zstd into dev
2016-08-11 22:10:04 +02:00
e9b414d825
fixed msan warning ( #281 )
2016-08-11 22:09:09 +02:00
5f49eba512
added usage of rep[0]-1 for the optimal parser
2016-08-10 15:01:53 +02:00
98e08cbe34
fixed: tree not updated after finding very long rep matches
2016-08-10 15:00:30 +02:00
038d1497c9
fixed compilation with Visual Studio 2005
2016-08-10 14:30:10 +02:00
48849f86f0
fixed compilation with Intel Compiler with Windows
2016-08-10 14:26:35 +02:00
1ea5622a32
updated xxhash
2016-08-10 09:40:08 +02:00
666398e7ed
added : xxhash namespace enforced from xxhash.h.
...
added : xxhash namespace test.
removed : -DXXH_NAMESPACE
2016-08-10 08:16:51 +02:00
8ded0b84aa
update xxhash to v0.6.2
2016-08-10 07:40:40 +02:00
280f9a8754
minor comment
2016-08-08 00:44:00 +02:00
e0b4a2d40f
fixed dictionary generation, reported by Bartosz Taudul
2016-08-03 03:36:03 +02:00
ae40b18d55
bumped library number
2016-08-03 01:59:21 +02:00
0763905f44
ZSTD_compress_usingCDict() correctly provides original size by default in frame header
...
Fixed dictionary examples
2016-08-03 01:57:57 +02:00
bf2bc112bb
bench : controlled display update when loading lot of files
2016-08-02 23:48:13 +02:00
346efccc35
fixed doc typo
2016-08-02 14:26:00 +02:00
f116e87f59
fixed analyzer warning
2016-08-01 19:15:18 +02:00
9ba929f1d4
Merge branch 'dev' of github.com:Cyan4973/zstd into dev
2016-08-01 02:26:59 +02:00
3ca750372d
updated doc ( #269 )
2016-08-01 02:26:20 +02:00
c55eb18c11
Merge pull request #267 from inikep/dev08
...
fixed ZSTD_compressBlock_opt_extDict_generic
2016-07-31 22:00:16 +02:00
Nick Terrell
Yann Collet
Yann Collet
inikep
Yann Collet
Christophe Chevalier
Thomas Klausner
David Lam
Alexander Borzunov