e474aa55b4
Fix decompression buffer overrun
...
Allows an adversary to write up to 3 bytes beyond the end of the buffer.
Occurs if the match overlaps the `extDict` and `currentPrefix`, and the
match length in the `currentPrefix` is less than `MINMATCH`, and
`op-(16-MINMATCH) >= oMatchEnd > op-16`.
2016-12-12 18:05:30 -08:00
c3a5c4bef8
introduced cycleLog
2016-12-12 00:47:30 +01:00
c261f71f6a
minor variation of rescale fix
2016-12-12 00:25:07 +01:00
3826207a70
Simplify segfault fix
...
Take advantage of the fact that `chainLog <= windowLog`.
2016-12-10 18:46:55 -08:00
0012332ce0
Fix compression segfault
...
When the overflow protection kicks in, it makes sure that ip - ctx->base
isn't too large. However, it didn't ensure that saved offsets are
still valid. This change ensures that any valid offsets (<= windowLog)
are still representable after the update.
The bug would shop up on line 1056, when `offset_1 > current + 1`, which
causes an underflow. This in turn, would cause a segfault on line 1063.
The input must necessarily be longer than 1 GB for this issue to occur.
Even then, it only occurs if one of the last 3 matches is larger than
the chain size and block size.
2016-12-09 17:15:33 -08:00
383b8088a3
minor lib build refactoring
2016-12-08 18:42:27 -08:00
6e754fe76a
fixed lib soname.
...
example : simple_compression : size overflow check
2016-12-08 18:26:56 -08:00
7687913178
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
2016-12-08 10:42:42 +01:00
426a9d4b71
changed : dll : only approved ZSTD symbols are now exposed. All other symbols remain internal.
2016-12-07 16:39:34 -08:00
4da53219a0
zstd Manual updated to 1.1.2
2016-12-07 11:18:40 +01:00
379908be3d
fixed zstd.h for manual
2016-12-06 10:36:15 -08:00
9dfd0af164
ZBUFF_ as a wrapper to ZSTD streaming API.
2016-12-06 17:16:41 +01:00
825dffbc43
moved zbuff source files into lib/deprecated
2016-12-05 19:28:19 -08:00
8f8e2b0b4a
fixed initialization warning
2016-12-05 18:00:50 -08:00
e7a41a5955
added : dictID retrieval functions.
...
added : unit tests for dictID retrieval functions
2016-12-05 16:21:06 -08:00
9ffbeea875
API : changed : streaming decompression : implicit reset on starting new frames
2016-12-02 18:37:38 -08:00
2238312c2f
fix dict loading
2016-12-02 11:36:11 -08:00
821bf1febc
fixed Doxygen trailing comment
2016-12-02 16:13:41 +01:00
b89af20353
reduced table sizes for HUF_readDTableX4
2016-12-01 18:24:59 -08:00
a0d742b1e4
introduced HUF_buildCTable_wksp(), to reduce stack memory usage
2016-12-01 17:47:30 -08:00
643d9a234b
replaced usage of FSE_buildCTable by FSE_buildCTable_wksp, using less stack space in the process
2016-12-01 16:24:04 -08:00
e928f7e16d
introduced ext_wksp variants of count to reduce stack memory usage
2016-12-01 16:13:35 -08:00
979cab412b
fixed some minor visual silent cast warnings.
...
introduced FSE_count_parallel_wksp().
2016-11-30 18:10:38 -08:00
5e00b848a8
FSE_compress_wksp() uses less stack space
2016-11-30 16:46:13 -08:00
d79a9a00d9
Introduced FSE_compress_wksp() and FSE_buildCTable_wksp() to reduce stack memory usage
2016-11-30 15:52:20 -08:00
766431909f
introduced FSE_decompress_wksp(), to use less stack space
2016-11-30 12:36:45 -08:00
95eb43be09
updated pkg config file
2016-11-30 11:06:58 -08:00
42247705a3
Merge pull request #461 from obache/neatsrc/pkgconfig
...
libzstd.pc.in: Change to use variables for libdir and includedir
2016-11-30 20:03:42 +01:00
ff504de391
minor decompression speed improvement
2016-11-29 17:42:46 -08:00
25f46dcc0f
minor const
2016-11-29 16:59:27 -08:00
a56ac2815c
restored normal decoder speed
2016-11-29 15:30:23 -08:00
37870d7a66
fixed minor visual warning
2016-11-29 14:31:57 -08:00
013a2b58ae
Merge pull request #464 from terrelln/guards
...
Fix ZSTD_STATIC_LINKING_ONLY with double include
2016-11-29 23:06:10 +01:00
167c494748
Merge branch 'dev' of github.com:facebook/zstd into dev
2016-11-29 14:05:15 -08:00
4f5350f610
long matches support overflow
2016-11-29 13:12:24 -08:00
05c00f2ff7
Fix ZSTD_STATIC_LINKING_ONLY with double include
2016-11-29 11:54:39 -08:00
c53eea7c1a
libzstd.pc.in: Change to use variables for libdir and includedir
2016-11-29 16:47:53 +09:00
52e136ed3d
long decoder compatible with round and separate buffers
2016-11-28 19:59:11 -08:00
ce3527ca0c
combined normal and long decoder
2016-11-28 18:38:52 -08:00
8993bee997
restored normal mode
2016-11-28 16:11:30 -08:00
764e70a4f3
added decodeSequencesLong
2016-11-28 15:50:16 -08:00
73f88a66f1
added prefetch
2016-11-23 15:43:30 -08:00
50524bf0da
delayed decompression
2016-11-23 15:11:07 -08:00
fc4193bda5
fixed g++ warnings
2016-11-23 18:17:18 +01:00
9ca65af810
zstd_opt.h: improved price function
2016-11-23 17:22:54 +01:00
ad3e94512c
fixed warnings from static analyzer in zstd_opt.h
2016-11-21 20:22:12 +01:00
eec700a3b7
exclude zbuff_compress.c and zbuff_decompress.c
2016-11-21 14:34:03 +01:00
62d19a6f39
lib\README.md: added Using MinGW+MSYS to create DLL
2016-11-21 14:22:08 +01:00
b85f767743
files to generate ZSTD Windows binary package
2016-11-21 14:10:55 +01:00
8bb86e330b
create DLL with Windows
2016-11-21 12:51:01 +01:00
93a09eedf1
added libzstd.def
2016-11-21 12:33:27 +01:00
5a17223691
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
2016-11-18 11:47:01 +01:00
3d18088b38
updated windres
2016-11-17 18:04:41 +01:00
0d761dbe95
Merge pull request #453 from inikep/dev11
...
fullbench-dll
2016-11-16 15:45:30 -08:00
52afb3993e
zbuff API now generates deprecation warnings
2016-11-16 08:50:54 -08:00
179555c1d1
working fullbench-dll
2016-11-15 18:05:46 +01:00
4359d21ad7
Merge two memset() calls into one
2016-11-14 17:52:51 -08:00
24701de877
Fix uninitialized memory read
2016-11-14 13:57:05 -08:00
8e4901eccd
removed zbuff.h from include installation
2016-11-08 15:45:39 -08:00
fd3be6bc97
bump version number to 1.1.2
2016-11-07 14:35:41 -08:00
dc904ad17b
Fix bug in zstd v0.{5, 6} dictionary decompression
...
Introduced by bb68062c59
2016-11-04 16:18:59 -07:00
38b590ad69
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
...
# Conflicts:
# lib/Makefile
2016-11-04 10:10:54 +01:00
407a11f63e
fixed Visual compatibility
2016-11-03 15:52:01 -07:00
11812260d1
Merge pull request #439 from terrelln/dev
...
ZSTD_compress_usingDict() specify dict == NULL
2016-11-03 14:15:36 -07:00
c8a9ac312b
Fix dynamic libzstd symlinks
2016-11-03 12:32:48 -07:00
3a415594b1
fixed MinGW compilation
2016-11-03 12:59:20 +01:00
7347869fb6
fixed make install
2016-11-02 22:28:37 -07:00
d82efd8a70
ZSTD_compress_usingDict() when dict gets loaded
...
Specify that when `dict == NULL || dictSize < 8` no dictionary
gets loaded.
Also add some periods.
2016-11-02 18:07:16 -07:00
179b19776f
fileio.c does no longer need ZSTD_LEGACY_SUPPORT, and does no longer depend on zstd_legacy.h
...
Added : ZSTD_isFrame() in experimental section
2016-11-02 17:30:49 -07:00
f3f13211ae
Fix #419 : no warning when setting custom LDFLAGS
2016-11-02 17:02:45 -07:00
0a5a5fb7fd
Fix #418 : printing selected segments in zdict debug mode can segfault with certain pathological patterns
2016-11-02 13:57:55 -07:00
31e660e7aa
more accurate default maximum window size
2016-10-29 03:56:45 -07:00
2115724c22
Merge pull request #430 from terrelln/exec-sequences
...
ZSTD_execSequence() accepts match in last 7 bytes
2016-10-28 10:45:05 -07:00
10bfd0c0d5
Fix ZSTD_execSequence() performance regression
...
Commit ae1cb3b3d0
2016-10-27 16:19:57 -07:00
ee5b725823
ZSTD_initCStream() optimization : do not allocate a CDict when no dictionary used
2016-10-27 14:20:55 -07:00
eb7873a048
ZSTD_execSequence() accepts match in last 7 bytes
...
The zstd reference compressor will not emit a match in the last 7
bytes of a block. The decompressor will also not accept a match
in the last 7 bytes. This patch makes the decompressor accept a
match in the last 7 bytes.
2016-10-25 21:24:15 -07:00
335ad5d4d4
added ZSTD_initDStream_usingDDict() .
...
slightly optimized ZSTD_initDStream() when no dictionary .
fixed ZSTD_sizeof_CStream() .
2016-10-25 17:47:02 -07:00
9516234e67
first sketch for ZSTD_initCStream_usingCDict()
2016-10-25 16:19:52 -07:00
62d9a7ddfd
Merge pull request #429 from inikep/btopt2
...
Btopt2
2016-10-25 14:48:43 -07:00
5c5f01f3da
added ZSTD_btopt2 strategy
2016-10-25 12:25:07 +02:00
7b5948cca7
Merge pull request #426 from terrelln/fixes
...
Fix various {A, M}SAN bugs
2016-10-24 23:42:26 -07:00
37d130031d
updated comments on context re-use
2016-10-24 17:22:12 -07:00
b2c39a22b0
Fix compiler narrowing warning
2016-10-24 14:50:13 -07:00
f698ad6deb
Merge remote-tracking branch 'upstream/dev' into fixes
...
* upstream/dev:
added doc\zstd_manual.html
added contrib\gen_html
zstd_compression_format.md moved to doc/
Fix small bug in ZSTD_execSequence()
improved ZSTD_compressBlock_opt_extDict_generic
protect ZSTD_decodeFrameHeader() from invalid usage, as suggested by @spaskob
zstd_opt.h: small improvement in compression ratio
improved dicitonary segment merge
use implicit rules to compile zstd_decompress.c
detect early impossible decompression scenario in legacy decoder v0.5
no repeat mode in legacy v0.5
fixed invalid invocation of dictionary in legacy decoder v0.5
fix edge case
fix command line interpretation
fixed minor corner case
zstd.h: added the Introduction section
fixed clang 3.5 warnings
zstd.h: updated comments
2016-10-24 13:10:13 -07:00
4239a207dd
Merge pull request #425 from inikep/dev11
...
Doc
2016-10-24 11:11:40 -07:00
f9c9af3c2e
Reject dictionaries with incomplete entropy tables
...
If a dictionary specifies that a symbol has probability zero in its
`matchLength`, `literalLength`, or `offset` FSE table, but the symbol
appears when compressing input, the compressor fails.
Ensure that dictionaries support all `matchLength`, and `literalLength`
codes. They must also support all of the `offset` codes required to
represent every possible offset that can appear in the first block.
2016-10-24 10:42:44 -07:00
984b66cd72
added contrib\gen_html
2016-10-24 15:59:51 +02:00
3ee94a7600
zstd_compression_format.md moved to doc/
2016-10-24 15:58:07 +02:00
97611611a3
Merge pull request #423 from terrelln/exec-seq-patch
...
Fix small bug in ZSTD_execSequence()
2016-10-21 17:02:06 -07:00
ae1cb3b3d0
Fix small bug in ZSTD_execSequence()
...
`memmove(op, match, sequence.matchLength)` is not the desired behavior.
Overlap is allowed, and handled as if we did `*op++ = *match++`, which
is not how `memmove()` handles overlap.
Only triggered if both of the following conditions are met:
* The match spans extDict & currentPrefixSegment
* `oLitEnd <= oend_w < oLitEnd + length1 < oMatchEnd <= oend`.
These two conditions imply that the block is less than 15 bytes long.
This bug isn't triggered by the streaming API, because it allocates
enough space for the window size + the block size, so there cannot be
a match that is within 8 bytes of the end and overlaps with itself.
It cannot be triggered by the block decompression API because all of
the decompressed data is in the currentPrefixSegment.
Introduced by commit 7158584399
2016-10-21 12:13:44 -07:00
4732074a71
improved ZSTD_compressBlock_opt_extDict_generic
2016-10-21 11:19:00 +02:00
da3bd8b6de
protect ZSTD_decodeFrameHeader() from invalid usage, as suggested by @spaskob
2016-10-20 20:11:00 -07:00
d365ae3497
zstd_opt.h: small improvement in compression ratio
2016-10-20 11:49:02 +02:00
575ab00db7
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
2016-10-20 11:01:52 +02:00
d760529a05
Fix stack buffer overrun when weightTotal == 0
...
If `weightTotal == 0`, then `BIT_highbit32(weightTotal)` is
undefined behavior in the case that it calls `__builtin_clz()`.
If `tableLog == HUF_TABLELOG_ABSOLUTEMAX` then we will access one
byte beyond the end of the buffer.
2016-10-19 11:39:11 -07:00
bb68062c59
Unitialized memory read in ZSTD_decodeSeqHeaders()
...
Caused by two things:
1. Not checking that `ip` is in range except for the first byte.
2. `ZSTDv0{5,6}_decodeLiteralsBlock()` could return a value larger than `srcSize`.
2016-10-18 16:41:33 -07:00
52c1bf93fe
improved dicitonary segment merge
2016-10-18 16:34:58 -07:00
7b06ad7a05
Backport fix from commit 125d817
...
This fixes a read of unitialized memory.
Full commit hash: 125d81774f
2016-10-18 14:52:34 -07:00
f45b157d95
Backport fix from commit 9e8b09a
...
Fixes uninitialized memory reads.
Full commit hash: 9e8b09a7bd
2016-10-18 14:22:49 -07:00
f7906d5955
detect early impossible decompression scenario in legacy decoder v0.5
2016-10-18 13:48:32 -07:00
9313c8d953
no repeat mode in legacy v0.5
2016-10-18 13:36:15 -07:00
83d7bdee4b
fixed invalid invocation of dictionary in legacy decoder v0.5
2016-10-18 12:25:43 -07:00
197a55ee7b
fix edge case
2016-10-18 11:27:52 -07:00
fd98087047
Fix stack buffer overflow in HUF_readCTable()
...
If `w ==0` on line 153, then `CTable[n].nbBits == tableLog + 1`.
Then `nbPerRank[CTable[n].nbBits]` and `valPerRank[CTable[n].nbBits]`
are stack buffer overflows.
2016-10-17 18:16:59 -07:00
06573e17be
fixed minor corner case
2016-10-17 17:28:28 -07:00
bfd943ace5
Fix buffer overrun in ZSTD_loadDictEntropyStats()
...
The table log set by `FSE_readNCount()` was not checked in
`ZSTD_loadDictEntropyStats()`. This caused `FSE_buildCTable()`
to stack/heap overflow in a few places.
The benchmarks look good, there is no obvious compression performance regression:
> ./zstds/zstd.opt.0 -i10 -b1 -e10 ~/bench/silesia.tar
1#silesia.tar : 211988480 -> 73656930 (2.878), 271.6 MB/s , 716.8 MB/s
2#silesia.tar : 211988480 -> 70162842 (3.021), 204.8 MB/s , 671.1 MB/s
3#silesia.tar : 211988480 -> 66997986 (3.164), 156.8 MB/s , 658.6 MB/s
4#silesia.tar : 211988480 -> 66002591 (3.212), 136.4 MB/s , 665.3 MB/s
5#silesia.tar : 211988480 -> 65008480 (3.261), 98.9 MB/s , 647.0 MB/s
6#silesia.tar : 211988480 -> 62979643 (3.366), 65.2 MB/s , 670.4 MB/s
7#silesia.tar : 211988480 -> 61974560 (3.421), 44.9 MB/s , 688.2 MB/s
8#silesia.tar : 211988480 -> 61028308 (3.474), 32.4 MB/s , 711.9 MB/s
9#silesia.tar : 211988480 -> 60416751 (3.509), 21.1 MB/s , 718.1 MB/s
10#silesia.tar : 211988480 -> 60174239 (3.523), 22.2 MB/s , 721.8 MB/s
> ./compress_zstds/zstd.opt.1 -i10 -b1 -e10 ~/bench/silesia.tar
1#silesia.tar : 211988480 -> 73656930 (2.878), 273.8 MB/s , 722.0 MB/s
2#silesia.tar : 211988480 -> 70162842 (3.021), 203.2 MB/s , 666.6 MB/s
3#silesia.tar : 211988480 -> 66997986 (3.164), 157.4 MB/s , 666.5 MB/s
4#silesia.tar : 211988480 -> 66002591 (3.212), 132.1 MB/s , 661.9 MB/s
5#silesia.tar : 211988480 -> 65008480 (3.261), 96.8 MB/s , 641.6 MB/s
6#silesia.tar : 211988480 -> 62979643 (3.366), 63.1 MB/s , 677.0 MB/s
7#silesia.tar : 211988480 -> 61974560 (3.421), 44.3 MB/s , 678.2 MB/s
8#silesia.tar : 211988480 -> 61028308 (3.474), 33.1 MB/s , 708.9 MB/s
9#silesia.tar : 211988480 -> 60416751 (3.509), 21.5 MB/s , 710.1 MB/s
10#silesia.tar : 211988480 -> 60174239 (3.523), 21.9 MB/s , 723.9 MB/s
2016-10-17 16:55:52 -07:00
4db751668f
Fix buffer overrun in ZSTD_loadEntropy()
...
The table log set by `FSE_readNCount()` was not checked in
`ZSTD_loadEntropy()`. This caused `FSE_buildDTable(dctx->MLTable, ...)`
to overwrite the beginning of `dctx->hufTable`.
The benchmarks look good, there is no obvious performance regression:
> ./zstds/zstd.opt.0 -i10 -b1 -e5 ~/bench/silesia.tar
1#silesia.tar : 211988480 -> 73656930 (2.878), 268.2 MB/s , 701.0 MB/s
2#silesia.tar : 211988480 -> 70162842 (3.021), 199.5 MB/s , 666.9 MB/s
3#silesia.tar : 211988480 -> 66997986 (3.164), 154.9 MB/s , 655.6 MB/s
4#silesia.tar : 211988480 -> 66002591 (3.212), 128.9 MB/s , 648.4 MB/s
5#silesia.tar : 211988480 -> 65008480 (3.261), 98.4 MB/s , 633.4 MB/s
> ./zstds/zstd.opt.2 -i10 -b1 -e5 ~/bench/silesia.tar
1#silesia.tar : 211988480 -> 73656930 (2.878), 266.1 MB/s , 703.7 MB/s
2#silesia.tar : 211988480 -> 70162842 (3.021), 199.0 MB/s , 666.6 MB/s
3#silesia.tar : 211988480 -> 66997986 (3.164), 156.2 MB/s , 656.2 MB/s
4#silesia.tar : 211988480 -> 66002591 (3.212), 133.2 MB/s , 647.4 MB/s
5#silesia.tar : 211988480 -> 65008480 (3.261), 96.3 MB/s , 633.3 MB/s
2016-10-17 15:51:15 -07:00
ccfcc643da
Check if dict is empty before reading first byte
2016-10-17 11:46:03 -07:00
2b361cf2f1
minor opt
2016-10-14 16:09:07 -07:00
7933434fdf
Merge branch 'dev' of github.com:facebook/zstd into dev
2016-10-14 13:32:35 -07:00
d4cda27b63
new command -M#, to limit memory usage during decompression ( #403 )
2016-10-14 13:32:20 -07:00
3b9cdf9220
Fix ubsan failures (pass NULL to memcpy)
2016-10-12 20:54:42 -07:00
5d919e7ac3
added ZSTD_error_frameParameter_windowTooLarge ( #403 )
2016-10-12 17:29:24 -07:00
e19111c42f
make creates libzstd binaries ( #415 )
2016-10-12 11:09:36 -07:00
8b70d012f0
fix cmake
2016-10-12 10:23:53 -07:00
38fb0dc4cf
Merge pull request #416 from terrelln/exec-sequence
...
Fix ZSTD_execSequence() edge case
2016-10-12 10:17:53 -07:00
7158584399
Fix ZSTD_execSequence() edge case
2016-10-12 10:05:26 -07:00
f52cd03e73
bumped version number
2016-10-11 17:29:27 -07:00
ef2357d0d3
created error_private.c, so that a single list of error strings get included
2016-10-11 17:24:50 -07:00
14efab827b
added zstd_errors.h to include installation
2016-10-11 16:51:29 -07:00
a17fd7312a
changed error_public.h into zstd_errors.h
2016-10-11 16:41:09 -07:00
18b51b99c0
sync fse
2016-10-11 08:21:09 -07:00
2d2613399a
zstd.h: added the Introduction section
2016-10-06 16:28:21 +02:00
ba1db376ac
fixed clang 3.5 warnings
2016-10-06 14:22:48 +02:00
82057aa7ec
zstd.h: updated comments
2016-10-06 13:23:52 +02:00
df6797447f
update dictionary builder warning comments
2016-09-27 15:14:32 +02:00
47094ea66b
added comment on filePos
2016-09-26 18:03:33 +02:00
cf409a7e2a
fixed : init*_advanced() followed by reset() with different pledgedSrcSiz
2016-09-26 16:41:05 +02:00
2f2639438a
zstreamtest can fuzztest pledgedSrcSize
2016-09-26 14:06:08 +02:00
dc245e91cb
Changed to use ZSTDLIBv06_API and ZSTDLIBv07_API for DLL exports to fix warning
...
- changed name to prevent collision with ZSTDLIB_API used by non-legacy dll exports
2016-09-23 17:09:36 +02:00
21412bb3f6
Merge branch 'dev' of github.com:Cyan4973/zstd into dev
2016-09-22 15:57:56 +02:00
51f4d566c2
small decompression speed boost for very small data
2016-09-22 15:57:28 +02:00
97b378a6f8
Streaming : dictionary compression on multiple files / segments can correctly provide srcSize into header (when provided) using pledgedSrcSize.
2016-09-21 17:20:19 +02:00
993060e0f2
cli : better adaptation to small files
2016-09-21 16:46:08 +02:00
1eb2fdc74f
bumped version number
2016-09-18 12:21:47 +02:00
a6bdf55759
fixed memory leak
2016-09-15 17:02:06 +02:00
644a8da88a
fixed minor conversion warning
2016-09-15 16:16:21 +02:00
4cb212938c
introduced ZSTD_resetCStream()
2016-09-15 14:54:07 +02:00
fa0c09760c
variable renaming
2016-09-15 14:11:01 +02:00
d7c6589df8
support ZSTD_sizeof_*() on NULL
...
added ZSTD_sizeof_CDict()
2016-09-15 02:57:27 +02:00
e91c4b4cef
introduced ZSTD_resetDStream() .
...
added : ZSTD_sizeof_DDict()
2016-09-14 16:55:44 +02:00
d092d77cfc
minor variable renaming
2016-09-14 16:14:57 +02:00
64deef3bee
Fixed srcSize=1
2016-09-14 00:16:07 +02:00
26ec254066
new strategy for faster DDict decompression
2016-09-13 16:52:16 +02:00
ac175d46d4
updated comments
2016-09-13 00:51:47 +02:00
a3481d6de0
make uninstall
2016-09-12 05:04:26 +02:00
b3060f7a9e
changed streaming decoder behavior : now, when all compressed frame is consumed, it means decompression is completed, with regenerated data fully flushed.
2016-09-09 16:44:16 +02:00
01c199226a
updated decompression streaming example
2016-09-08 19:29:04 +02:00
5c6d244973
Merge branch 'dev' of github.com:facebook/zstd into dev
2016-09-07 14:54:54 +02:00
ac8bace6b1
support large skippable frames
2016-09-07 14:54:23 +02:00
0e07bf3f60
added comments on searchLength min / max ( #337 )
2016-09-07 06:33:02 +02:00
95d07d7447
introduced CHECK_E
2016-09-06 16:38:51 +02:00
3e21ec5b01
introduced CHECK_F
2016-09-06 15:36:19 +02:00
5c956d593c
FORCE_INLINE common definition
2016-09-06 15:05:19 +02:00
edbcd9f5b2
fixed zbufftest
2016-09-06 14:30:57 +02:00
b624922b14
fixed checksum
2016-09-06 11:16:57 +02:00
a7737f6a60
improved compression on small files when using same parameters
2016-09-06 09:44:59 +02:00
7ae67bb18a
small compression speed gains with using_CDict
2016-09-06 06:28:05 +02:00
1d4208c029
clarified streaming decompression inlined doc
2016-09-06 05:16:40 +02:00
7c83dfd5c2
ZSTD_frameHeaderSize_prefix ( #340 ), as result of ZSTD_initStream
2016-09-05 19:47:43 +02:00
fa72f6bdce
clarified inline doc for streaming
2016-09-05 17:39:56 +02:00
c73a8109bb
Merge pull request #344 from inikep/dev10
...
unified error codes for legacy decoders
2016-09-05 07:46:33 -07:00
45db83f98d
ZSTD_decodeLiteralsBlock renamed to ZSTDv01_decodeLiteralsBlock
2016-09-05 14:46:24 +02:00
476964f6a1
ZSTD_decodeSeqHeaders renamed to ZSTDv01_decodeSeqHeaders
2016-09-05 13:34:57 +02:00
c13faa1b0f
legacy decoders: restored #include <intrin.h> for VC++
2016-09-05 13:25:07 +02:00
8161e7321a
unified error codes for legacy decoders
2016-09-05 12:29:51 +02:00
b85cdabd50
Enable install targets for NetBSD.
2016-09-04 14:37:57 +02:00
33a0465a51
fixed a few links
2016-09-02 22:11:49 -07:00
d56dbc02d3
removed g_displayLevel
2016-09-02 17:28:41 -07:00
855766d73d
clarified dictionary in format description
2016-09-02 17:04:49 -07:00
d725427a3c
g_time => local displayTime
2016-09-02 15:32:39 -07:00
1563bfeabc
fixing FORCE_INLINE for older compilers ( #330 )
2016-09-02 11:44:21 -07:00
7304eb7c09
bumped version number
2016-09-01 15:49:26 -07:00
901e85fe26
version bump
2016-08-31 07:51:25 -07:00
1c59c20903
removed redundant files
2016-08-31 07:15:44 -07:00
599c69d917
minor Makefile updates
2016-08-30 13:33:20 -07:00
e10f7f3dcb
merge
2016-08-30 12:03:36 -07:00
4ded9e591c
added boilerplate
2016-08-30 11:06:28 -07:00
3b15f1f10f
minor refactor
2016-08-30 09:58:50 -07:00
240795bef7
Merge branch 'dev' of github.com:Cyan4973/zstd into dev
2016-08-30 06:51:55 -07:00
14200a20f0
Fixed issue #304 , reported by @borzunov
2016-08-30 06:51:00 -07:00
da9d3b7057
Cleanup some errors in typedef comments and remove duplicated HOWTO from zbuff_decompress.c
2016-08-29 17:31:51 -07:00
09c3c8e885
Merge pull request #307 from inikep/dev08
...
updated README.md
2016-08-29 16:32:33 -07:00
6416b0d705
updated README.md
2016-08-29 13:04:26 +02:00
23b6e05d8e
ZSTD_malloc() and ZSTD_free(), to simplify customMem
2016-08-28 21:05:43 -07:00
5f53b0335e
fixed continuation context
2016-08-28 10:00:49 -07:00
767d8f66fa
legacy contexts can be re-used
2016-08-28 08:19:47 -07:00
4bf317dd00
first version supporting legacy streams (transparent decoding)
2016-08-28 07:43:34 -07:00
e19a9ef05d
update compression level table
2016-08-26 20:02:49 +02:00
9a021c1aae
fixed some minor clang warnings
2016-08-26 09:05:06 +02:00
cb5a320705
made -Wdocumentation a clang only flag
2016-08-26 08:06:36 +02:00
87c18b2ebd
fixed multiple minor warnings for XCode
2016-08-26 01:43:47 +02:00
0d59a6f73a
removed debug strings
2016-08-25 22:42:46 +02:00
5a02b69215
reinforced fix for huge files
2016-08-25 22:24:59 +02:00
96bdd87de4
fixed : compression bug on very large files
2016-08-25 14:34:42 +02:00
a3a47ec4d0
Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into Other
2016-08-24 21:25:49 +02:00
a2cdffe556
fixed wrong parameter issue
2016-08-24 19:42:15 +02:00
e416e30019
remove unnecessary comments
2016-08-24 17:32:09 +02:00
4e90f6c1e0
removed ZSTD_LOG_ENCODE and ZSTD_LOG_BLOCK
2016-08-24 17:24:11 +02:00
83388e109f
removed ZSTD_LOG_PARSER
2016-08-24 17:22:20 +02:00
8a36f8527c
removed stats in debug mode
2016-08-24 17:19:12 +02:00
57ef4b1a0d
zstd_v07.c: removed unused macros
2016-08-24 17:16:56 +02:00
24b68a5b23
update cLevel table for 256KB
2016-08-24 14:22:26 +02:00
c54692faeb
improved level 3
2016-08-24 01:45:46 +02:00
17e482efdd
added ZSTD_setDStreamParameter()
2016-08-23 16:58:10 +02:00
3071c3e303
STREAM_WINDOW_MAX : protect streaming from unreasonable memory requirements
2016-08-23 01:34:34 +02:00
70e3b31306
fixed playtests on os-x
2016-08-23 01:18:06 +02:00
cb3276329a
added sizeof CStream and DStream
2016-08-23 00:31:59 +02:00
d1733f7417
fixed crc bug in rare timing conditions within bench.c
2016-08-21 01:04:46 +02:00
8baf78a291
minor coding style
2016-08-20 13:04:20 +02:00
1bee2d5e08
slight decompression speed improvement
2016-08-20 02:59:04 +02:00
0cfe2ec2fd
sync fse version
2016-08-20 00:26:26 +02:00
af1960396b
sync huff0
2016-08-19 19:38:19 +02:00
7be46bf8f2
promoted streaming API to stable
...
(except _advanced() variants)
2016-08-19 18:39:36 +02:00
fdba57d513
update version number
2016-08-19 18:32:30 +02:00
da3fbcb302
Added ZDICT_getDictID()
2016-08-19 14:23:58 +02:00
a5dbf9f629
Merge pull request #297 from borzunov/dev
...
Export functions related to dictionary compression from DLL
2016-08-18 15:05:01 +02:00
49d105cfcf
better warning and error messages in case of dictionary training failure ( #292 )
2016-08-18 15:02:11 +02:00
0f6f17a14f
Rename ZSTDLIB_API to ZDICTLIB_API in zdict.h
2016-08-18 16:47:06 +05:00
1f48382b1a
Export functions related to dictionary compression from DLL
2016-08-18 16:12:49 +05:00
e80d15304a
Merge pull request #296 from inikep/Other
...
Other
2016-08-18 11:48:48 +02:00
a7bb322a93
removed never referenced functions
2016-08-18 10:30:21 +02:00
18442c1482
minor refactoring
2016-08-18 01:40:32 +02:00
c411902230
fixed g++ conversion warning
2016-08-17 01:50:54 +02:00
53e17fbd5e
updated streaming API
2016-08-17 01:39:22 +02:00
655393cc72
updated doc for streaming API
2016-08-16 15:11:28 +02:00
104e5b072d
added : streaming decompression API
2016-08-16 15:11:28 +02:00
5a0c8e2439
new streaming API (compression)
2016-08-16 15:11:27 +02:00
ba92046031
Merge branch 'dev' of github.com:Cyan4973/zstd into dev
2016-08-11 22:10:04 +02:00
e9b414d825
fixed msan warning ( #281 )
2016-08-11 22:09:09 +02:00
5f49eba512
added usage of rep[0]-1 for the optimal parser
2016-08-10 15:01:53 +02:00
98e08cbe34
fixed: tree not updated after finding very long rep matches
2016-08-10 15:00:30 +02:00
038d1497c9
fixed compilation with Visual Studio 2005
2016-08-10 14:30:10 +02:00
48849f86f0
fixed compilation with Intel Compiler with Windows
2016-08-10 14:26:35 +02:00
1ea5622a32
updated xxhash
2016-08-10 09:40:08 +02:00
666398e7ed
added : xxhash namespace enforced from xxhash.h.
...
added : xxhash namespace test.
removed : -DXXH_NAMESPACE
2016-08-10 08:16:51 +02:00
8ded0b84aa
update xxhash to v0.6.2
2016-08-10 07:40:40 +02:00
280f9a8754
minor comment
2016-08-08 00:44:00 +02:00
e0b4a2d40f
fixed dictionary generation, reported by Bartosz Taudul
2016-08-03 03:36:03 +02:00
ae40b18d55
bumped library number
2016-08-03 01:59:21 +02:00
0763905f44
ZSTD_compress_usingCDict() correctly provides original size by default in frame header
...
Fixed dictionary examples
2016-08-03 01:57:57 +02:00
bf2bc112bb
bench : controlled display update when loading lot of files
2016-08-02 23:48:13 +02:00
346efccc35
fixed doc typo
2016-08-02 14:26:00 +02:00
f116e87f59
fixed analyzer warning
2016-08-01 19:15:18 +02:00
9ba929f1d4
Merge branch 'dev' of github.com:Cyan4973/zstd into dev
2016-08-01 02:26:59 +02:00
3ca750372d
updated doc ( #269 )
2016-08-01 02:26:20 +02:00
c55eb18c11
Merge pull request #267 from inikep/dev08
...
fixed ZSTD_compressBlock_opt_extDict_generic
2016-07-31 22:00:16 +02:00
056df510aa
fixed ZSTD_compressBlock_opt_extDict_generic
2016-07-31 20:08:53 +02:00
917fe188f1
Implemented repOffset "minus 1" on ll==0
2016-07-31 04:01:57 +02:00
2a2ba3691c
Merge pull request #266 from jrmarino/master
...
Enable build on FreeBSD ports (includes DragonFly BSD) [dev branch]
2016-07-31 02:21:26 +02:00
0d07ec0c0c
Enable build on FreeBSD ports (includes DragonFly BSD)
...
Zstd has been introduced to FreeBSD ports
(http://www.freshports.org/archivers/zstd/ ) which DragonFly BSD also
uses. FreeBSD and DragonFly use the install targets (albeit modified in
some cases) so they must be added to the associated Makefile filters.
2016-07-30 19:11:15 -05:00
8cebfd1d26
fix attempt on test-zstd-speed
2016-07-31 01:59:23 +02:00
66f69e58d2
restore decompression speed on fizzle
2016-07-30 15:32:47 +02:00
3b2bd1d11c
zstd_opt uses same tables as zstd_compress
2016-07-30 13:21:41 +02:00
f714f59c16
fixed visual warning
2016-07-30 12:05:28 +02:00
761f8dbbd2
back to normal table cell copy
2016-07-30 11:43:53 +02:00
3c6b808870
minor decompression speed gains
2016-07-30 03:20:47 +02:00
70a9ff4af3
fixed too large selectivity level, reported by Ilona Papava
2016-07-30 01:09:14 +02:00
c0ce4f1211
slightly improved compression speed
2016-07-30 00:55:13 +02:00
ed57d8530a
new seqStore
2016-07-29 21:22:17 +02:00
6b615d32cd
Updated API comments, following suggestions by Bryan O'Sullivan
2016-07-29 19:40:37 +02:00
c00d30fbe4
Merge pull request #264 from inikep/dev08
...
Dev08
2016-07-29 17:42:30 +02:00
6b68ba2079
zstd_opt.h: fixed checking of rep codes (2)
2016-07-29 16:45:39 +02:00
59b86fc141
zstd_opt.h: fixed checking of rep codes
2016-07-29 11:00:33 +02:00
6a82f0f8bf
minor comments
2016-07-29 00:55:45 +02:00
ffa7d0ac1e
clarified comment
2016-07-28 21:01:17 +02:00
4c5bbf64f9
fixed : frame concatenation without checksum
2016-07-28 20:30:25 +02:00
60ba31c570
zbuff uses ZSTD_compressEnd()
2016-07-28 19:55:09 +02:00
16e73033ad
introduced stage zbf_end
2016-07-28 16:32:34 +02:00
62470b4bab
Changed ZSTD_compressEnd()
2016-07-28 15:29:08 +02:00
e7bf9156d1
Clarified API comments, from suggestions by Bryan O'Sullivan
2016-07-28 05:00:57 +02:00
d469a98c01
Clarified API comments, from suggestions by Bryan O'Sullivan
2016-07-28 04:55:03 +02:00
19c1002e46
applied ZSTD_compressContinueThenEnd()
2016-07-28 01:25:46 +02:00
5b56739b63
created ZSTD_compressContinueThenEnd()
2016-07-28 01:17:22 +02:00
c991cc1828
new frame end, 32-bits checksums
2016-07-28 00:55:43 +02:00
d4180cad9c
minor code refactoring
2016-07-27 21:21:36 +02:00
731ef16fc1
minor code style refactoring
2016-07-27 21:05:12 +02:00
4b9ca0a6b5
minor example variation
2016-07-27 19:53:19 +02:00
4110534886
ZSTD_maxCLevel() is promoted to "stable" API ( #254 , by @FrancescAlted)
2016-07-27 15:09:11 +02:00
55a8bea0b5
fixed dictionary generation
2016-07-27 14:48:47 +02:00
c154d9d6a2
better support for large dictionaries (> 128 KB)
2016-07-27 14:37:00 +02:00
07626dfa51
improved dictbuilder notifications on selectivity
2016-07-27 13:28:46 +02:00
f796f7ab45
removed fastscan mode
2016-07-27 12:53:54 +02:00
dd25a27702
added tutorial warning messages for dictBuilder
2016-07-27 12:43:09 +02:00
003c7a8568
optimal parser: removed ZSTD_REP_INIT
2016-07-27 11:07:13 +02:00
04cdd8660d
Merge pull request #262 from ebiggers/misc_updates
...
Miscellaneous updates
2016-07-27 01:25:45 +02:00
0a55e7a0bb
ZSTD_decompressFrame(): use remainingSize instead of iend - ip
...
Same behavior, but no need to have redundant variables.
2016-07-26 13:22:27 -07:00
aa6c70bf60
ZSTD_decompressFrame(): pass up error code from ZSTD_decodeFrameHeader()
2016-07-26 13:22:27 -07:00
e4d0265ea9
Replace remaining references to "direct mode" with "single segment mode"
2016-07-26 13:22:27 -07:00
d50f9db3ea
Improved speed on clang and gcc -O2, thanks to @ebiggers ! ( #263 )
2016-07-26 21:30:35 +02:00
7adc2328a3
fixed --test on zero-length files, reported by @amnilsson
2016-07-26 15:49:24 +02:00
4178f5c289
fixed gcc warning: always_inline function might not be inlinable
2016-07-25 21:17:45 +02:00
fca90f8f60
legacy decoder for v0.7 format
2016-07-25 17:49:08 +02:00
cbc5e9dc19
fixes oob read
2016-07-24 18:02:04 +02:00
38b75ddeb2
removed special case all-1 huffman distribution
2016-07-24 15:35:59 +02:00
7ed5e33b89
minor comment changes
2016-07-24 14:26:11 +02:00
10b9c13d07
fixed doc on cLevel default, reported by Oliver Lange
2016-07-24 01:21:53 +02:00
f8e7b5363f
unified encoding types
2016-07-23 16:31:49 +02:00
571a59034a
changed enccoding type order : raw, rle, compressed, repeat-stats
2016-07-23 15:52:05 +02:00
c2e1a68d81
changed streamNb order to 1-4-4-4
2016-07-22 17:30:52 +02:00
772d912c2f
more complete support for literals repeat mode
2016-07-22 15:04:25 +02:00
9f2d82d4a4
fixed : big-endian decoding
2016-07-22 14:37:10 +02:00
32faf6c8e7
fixed conversion warnings
2016-07-22 14:37:09 +02:00
5e45a5fbb3
force loop-align to 32 for zstd_decompress
2016-07-22 14:37:09 +02:00
5288ac0cb7
changed filed order
2016-07-22 14:37:09 +02:00
198e6aac44
Literals header fields use little endian convention
2016-07-22 14:37:09 +02:00
6fa05a2371
cBlockSize uses little-endian convention
2016-07-22 14:37:09 +02:00
7bf72bbf5e
update header to v0.8
2016-07-22 14:37:09 +02:00
5894ea8d01
updated cLevels
2016-07-22 14:36:46 +02:00
d5c5a77990
minor comments clarifications
2016-07-20 13:35:14 +02:00
572b817be3
Merge pull request #253 from gymdis/heapmode_off_legacy_fix
...
Fix compile issue with ZSTD_LEGACY_SUPPORT=1 and ZSTD_HEAPMODE=0
2016-07-19 13:52:03 +02:00
780a9fa857
Fix compile issue with ZSTD_LEGACY_SUPPORT=1 and ZSTD_HEAPMODE=0
2016-07-19 13:25:38 +02:00
cf05b9d477
ZSTD_getBlockSizeMax()
2016-07-18 16:52:10 +02:00
16aa38b0e0
minor doc clarifications
2016-07-18 03:52:47 +02:00
85f3919960
moved `zstd.h` to `/lib`
2016-07-17 20:42:21 +02:00
9375590462
update version to v0.7.5
2016-07-17 16:44:18 +02:00
e557fd5e92
minor compression level corrections
2016-07-17 16:21:37 +02:00
d54b2d23b4
minor static assert for 32/64 bits system. Suggested by @ebiggers
2016-07-17 15:53:18 +02:00
972e5806ee
fixed : premature frame end on zero-sized raw block - reported by @ebiggers
2016-07-17 15:39:24 +02:00
10f999f856
Add legacy support for the low-level streaming API
2016-07-17 01:03:26 +02:00
6cacd34d44
minor formatting changes
2016-07-15 17:58:13 +02:00
f6ff53cd4e
implemented dictID reserved ranges
2016-07-15 17:03:38 +02:00
98c8884999
added target zstd in root Makefile
2016-07-15 16:12:38 +02:00
961b6a0e34
ZSTD_compressBlock() limits block size depending on windowLog parameter
2016-07-15 11:58:49 +02:00
227cc39e15
improved efficiency for large messages with small dictionaries
2016-07-15 11:27:09 +02:00
ea2ecdc315
fixed issue with small dictionary
2016-07-14 23:27:31 +02:00
e9ed5cdc94
fixed minor coverity warning
2016-07-14 21:02:57 +02:00
b23e1ce319
removed debugging traces
2016-07-14 17:46:38 +02:00
17508f1a16
fixed a few minor coverity warnings
2016-07-14 17:18:20 +02:00
8847238cac
simplified ZSTD_estimateCCtxSize()
2016-07-14 17:05:38 +02:00
69c2cdb45c
fixed conversion warning
2016-07-14 16:52:45 +02:00
5e80dd3261
fixed minor coverity warnings
2016-07-13 19:21:57 +02:00
3c174f4da9
fixed minor coverity warning
2016-07-13 17:25:53 +02:00
2b1a3638e6
changed macro name to ZSTDCLI_CLEVEL_DEFAULT
2016-07-13 15:16:00 +02:00
3c242e79d3
updated compression levels table
2016-07-13 14:56:24 +02:00
fbc69f8649
changed for #245
2016-07-13 13:52:58 +02:00
eed2081e55
fixed conversion warning
2016-07-12 15:11:40 +02:00
a43a854cdb
updated paramgrill
2016-07-12 13:42:10 +02:00
73d74a05b9
fixed dfast strategy
2016-07-12 13:03:48 +02:00
45dc35628c
first version of doubleFast
2016-07-12 09:47:31 +02:00
d158c35e9f
added ZSTD_estimateDCtxSize()
2016-07-11 13:46:25 +02:00
8e0ee681b8
added ZSTD_sizeofDCtx()
2016-07-11 13:09:52 +02:00
3ae543ce75
added ZSTD_estimateCCtxSize()
2016-07-11 03:12:17 +02:00
25c506601c
promote ZSTD_getDecompressedSize() to stable API
2016-07-10 01:46:18 +02:00
3b6ae77e15
comment clarification
2016-07-08 23:42:22 +02:00
722e14bb65
fixed compilation error in decompression module
2016-07-08 19:22:16 +02:00
bd10607063
updated spec
2016-07-08 19:16:57 +02:00
c5fb5b7fcd
support offset > 128 MB
2016-07-08 13:13:37 +02:00
ed3845d3fa
introduced ZSTD_WINDOWLOG_MAX_32 ( #239 ), suggested by @GregSlazinski
2016-07-08 12:57:10 +02:00
26f681451f
updated doc
2016-07-08 11:45:08 +02:00
Nick Terrell
Yann Collet
Przemyslaw Skibinski
Yann Collet
Yann Collet
OBATA Akio
Christophe Chevalier
Thomas Klausner
David Lam
Alexander Borzunov
jrmarino
Eric Biggers
Christopher Bergqvist
luben karavelov