Commit Graph

1305 Commits (9050e1925ece4cddd6aa82b1a4526750081c3243)

Author SHA1 Message Date
Yann Collet 2b36b238d3 changed variable name to estimatedSrcSize, to emphasize it does not need to be exact 2016-12-13 17:59:55 +01:00
Yann Collet e795c8a5f6 Added ZSTD_initCStream_srcSize().
Added relevant test cases in zstreamtest
2016-12-13 17:00:14 +01:00
Yann Collet 5397a66b19 minor BMI version check 2016-12-13 15:21:06 +01:00
Yann Collet 35168679bd Merge pull request #478 from terrelln/wildcopy-ub
Fix execSequence wildcopy undefined behavior
2016-12-13 11:33:00 +01:00
Nick Terrell 064a143520 Fix execSequence wildcopy undefined behavior
execSequence relied on pointer overflow to handle cases where
`sequence.matchLength < 8`.  Instead of passing an `size_t` to
wildcopy, pass a `ptrdiff_t`.
2016-12-12 19:01:23 -08:00
Nick Terrell e474aa55b4 Fix decompression buffer overrun
Allows an adversary to write up to 3 bytes beyond the end of the buffer.
Occurs if the match overlaps the `extDict` and `currentPrefix`, and the
match length in the `currentPrefix` is less than `MINMATCH`, and
`op-(16-MINMATCH) >= oMatchEnd > op-16`.
2016-12-12 18:05:30 -08:00
Yann Collet c3a5c4bef8 introduced cycleLog 2016-12-12 00:47:30 +01:00
Yann Collet c261f71f6a minor variation of rescale fix 2016-12-12 00:25:07 +01:00
Nick Terrell 3826207a70 Simplify segfault fix
Take advantage of the fact that `chainLog <= windowLog`.
2016-12-10 18:46:55 -08:00
Nick Terrell 0012332ce0 Fix compression segfault
When the overflow protection kicks in, it makes sure that ip - ctx->base
isn't too large.  However, it didn't ensure that saved offsets are
still valid.  This change ensures that any valid offsets (<= windowLog)
are still representable after the update.

The bug would shop up on line 1056, when `offset_1 > current + 1`, which
causes an underflow.  This in turn, would cause a segfault on line 1063.

The input must necessarily be longer than 1 GB for this issue to occur.
Even then, it only occurs if one of the last 3 matches is larger than
the chain size and block size.
2016-12-09 17:15:33 -08:00
Yann Collet 383b8088a3 minor lib build refactoring 2016-12-08 18:42:27 -08:00
Yann Collet 6e754fe76a fixed lib soname.
example : simple_compression : size overflow check
2016-12-08 18:26:56 -08:00
Przemyslaw Skibinski 7687913178 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2016-12-08 10:42:42 +01:00
Yann Collet 426a9d4b71 changed : dll : only approved ZSTD symbols are now exposed. All other symbols remain internal. 2016-12-07 16:39:34 -08:00
Przemyslaw Skibinski 4da53219a0 zstd Manual updated to 1.1.2 2016-12-07 11:18:40 +01:00
Yann Collet 379908be3d fixed zstd.h for manual 2016-12-06 10:36:15 -08:00
Yann Collet 9dfd0af164 ZBUFF_ as a wrapper to ZSTD streaming API. 2016-12-06 17:16:41 +01:00
Yann Collet 825dffbc43 moved zbuff source files into lib/deprecated 2016-12-05 19:28:19 -08:00
Yann Collet 8f8e2b0b4a fixed initialization warning 2016-12-05 18:00:50 -08:00
Yann Collet e7a41a5955 added : dictID retrieval functions.
added : unit tests for dictID retrieval functions
2016-12-05 16:21:06 -08:00
Yann Collet 9ffbeea875 API : changed : streaming decompression : implicit reset on starting new frames 2016-12-02 18:37:38 -08:00
Yann Collet 2238312c2f fix dict loading 2016-12-02 11:36:11 -08:00
Przemyslaw Skibinski 821bf1febc fixed Doxygen trailing comment 2016-12-02 16:13:41 +01:00
Yann Collet b89af20353 reduced table sizes for HUF_readDTableX4 2016-12-01 18:24:59 -08:00
Yann Collet a0d742b1e4 introduced HUF_buildCTable_wksp(), to reduce stack memory usage 2016-12-01 17:47:30 -08:00
Yann Collet 643d9a234b replaced usage of FSE_buildCTable by FSE_buildCTable_wksp, using less stack space in the process 2016-12-01 16:24:04 -08:00
Yann Collet e928f7e16d introduced ext_wksp variants of count to reduce stack memory usage 2016-12-01 16:13:35 -08:00
Yann Collet 979cab412b fixed some minor visual silent cast warnings.
introduced FSE_count_parallel_wksp().
2016-11-30 18:10:38 -08:00
Yann Collet 5e00b848a8 FSE_compress_wksp() uses less stack space 2016-11-30 16:46:13 -08:00
Yann Collet d79a9a00d9 Introduced FSE_compress_wksp() and FSE_buildCTable_wksp() to reduce stack memory usage 2016-11-30 15:52:20 -08:00
Yann Collet 766431909f introduced FSE_decompress_wksp(), to use less stack space 2016-11-30 12:36:45 -08:00
Yann Collet 95eb43be09 updated pkg config file 2016-11-30 11:06:58 -08:00
Yann Collet 42247705a3 Merge pull request #461 from obache/neatsrc/pkgconfig
libzstd.pc.in: Change to use variables for libdir and includedir
2016-11-30 20:03:42 +01:00
Yann Collet ff504de391 minor decompression speed improvement 2016-11-29 17:42:46 -08:00
Yann Collet 25f46dcc0f minor const 2016-11-29 16:59:27 -08:00
Yann Collet a56ac2815c restored normal decoder speed 2016-11-29 15:30:23 -08:00
Yann Collet 37870d7a66 fixed minor visual warning 2016-11-29 14:31:57 -08:00
Yann Collet 013a2b58ae Merge pull request #464 from terrelln/guards
Fix ZSTD_STATIC_LINKING_ONLY with double include
2016-11-29 23:06:10 +01:00
Yann Collet 167c494748 Merge branch 'dev' of github.com:facebook/zstd into dev 2016-11-29 14:05:15 -08:00
Yann Collet 4f5350f610 long matches support overflow 2016-11-29 13:12:24 -08:00
Nick Terrell 05c00f2ff7 Fix ZSTD_STATIC_LINKING_ONLY with double include 2016-11-29 11:54:39 -08:00
OBATA Akio c53eea7c1a libzstd.pc.in: Change to use variables for libdir and includedir 2016-11-29 16:47:53 +09:00
Yann Collet 52e136ed3d long decoder compatible with round and separate buffers 2016-11-28 19:59:11 -08:00
Yann Collet ce3527ca0c combined normal and long decoder 2016-11-28 18:38:52 -08:00
Yann Collet 8993bee997 restored normal mode 2016-11-28 16:11:30 -08:00
Yann Collet 764e70a4f3 added decodeSequencesLong 2016-11-28 15:50:16 -08:00
Yann Collet 73f88a66f1 added prefetch 2016-11-23 15:43:30 -08:00
Yann Collet 50524bf0da delayed decompression 2016-11-23 15:11:07 -08:00
Przemyslaw Skibinski fc4193bda5 fixed g++ warnings 2016-11-23 18:17:18 +01:00
Przemyslaw Skibinski 9ca65af810 zstd_opt.h: improved price function 2016-11-23 17:22:54 +01:00
Przemyslaw Skibinski ad3e94512c fixed warnings from static analyzer in zstd_opt.h 2016-11-21 20:22:12 +01:00
Przemyslaw Skibinski eec700a3b7 exclude zbuff_compress.c and zbuff_decompress.c 2016-11-21 14:34:03 +01:00
Przemyslaw Skibinski 62d19a6f39 lib\README.md: added Using MinGW+MSYS to create DLL 2016-11-21 14:22:08 +01:00
Przemyslaw Skibinski b85f767743 files to generate ZSTD Windows binary package 2016-11-21 14:10:55 +01:00
Przemyslaw Skibinski 8bb86e330b create DLL with Windows 2016-11-21 12:51:01 +01:00
Przemyslaw Skibinski 93a09eedf1 added libzstd.def 2016-11-21 12:33:27 +01:00
Przemyslaw Skibinski 5a17223691 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2016-11-18 11:47:01 +01:00
Przemyslaw Skibinski 3d18088b38 updated windres 2016-11-17 18:04:41 +01:00
Yann Collet 0d761dbe95 Merge pull request #453 from inikep/dev11
fullbench-dll
2016-11-16 15:45:30 -08:00
Yann Collet 52afb3993e zbuff API now generates deprecation warnings 2016-11-16 08:50:54 -08:00
Przemyslaw Skibinski 179555c1d1 working fullbench-dll 2016-11-15 18:05:46 +01:00
Nick Terrell 4359d21ad7 Merge two memset() calls into one 2016-11-14 17:52:51 -08:00
Nick Terrell 24701de877 Fix uninitialized memory read 2016-11-14 13:57:05 -08:00
Yann Collet 8e4901eccd removed zbuff.h from include installation 2016-11-08 15:45:39 -08:00
Yann Collet fd3be6bc97 bump version number to 1.1.2 2016-11-07 14:35:41 -08:00
Nick Terrell dc904ad17b Fix bug in zstd v0.{5, 6} dictionary decompression
Introduced by bb68062c59.
2016-11-04 16:18:59 -07:00
Przemyslaw Skibinski 38b590ad69 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
# Conflicts:
#	lib/Makefile
2016-11-04 10:10:54 +01:00
Yann Collet 407a11f63e fixed Visual compatibility 2016-11-03 15:52:01 -07:00
Yann Collet 11812260d1 Merge pull request #439 from terrelln/dev
ZSTD_compress_usingDict() specify dict == NULL
2016-11-03 14:15:36 -07:00
Nick Terrell c8a9ac312b Fix dynamic libzstd symlinks 2016-11-03 12:32:48 -07:00
Przemyslaw Skibinski 3a415594b1 fixed MinGW compilation 2016-11-03 12:59:20 +01:00
Yann Collet 7347869fb6 fixed make install 2016-11-02 22:28:37 -07:00
Nick Terrell d82efd8a70 ZSTD_compress_usingDict() when dict gets loaded
Specify that when `dict == NULL || dictSize < 8` no dictionary
gets loaded.
Also add some periods.
2016-11-02 18:07:16 -07:00
Yann Collet 179b19776f fileio.c does no longer need ZSTD_LEGACY_SUPPORT, and does no longer depend on zstd_legacy.h
Added : ZSTD_isFrame() in experimental section
2016-11-02 17:30:49 -07:00
Yann Collet f3f13211ae Fix #419 : no warning when setting custom LDFLAGS 2016-11-02 17:02:45 -07:00
Yann Collet 0a5a5fb7fd Fix #418 : printing selected segments in zdict debug mode can segfault with certain pathological patterns 2016-11-02 13:57:55 -07:00
Yann Collet 31e660e7aa more accurate default maximum window size 2016-10-29 03:56:45 -07:00
Yann Collet 2115724c22 Merge pull request #430 from terrelln/exec-sequences
ZSTD_execSequence() accepts match in last 7 bytes
2016-10-28 10:45:05 -07:00
Nick Terrell 10bfd0c0d5 Fix ZSTD_execSequence() performance regression
Commit ae1cb3b3d0 caused the regression.
It is an instruction alignment issue, because if it is `U64 i` instead
of `U32 i`, the regression returns.  This patch fixes the regression
in gcc, but only gets some of the clang performance back.

Benchmarks:
Run on `silesia.tar`.  I only show levels 1-5 because the performance
regression was uniform across all levels.  I did one run on levels
1-19 and it looked good.

| Build | Level | Before | While | After |
|-------|-------|-------:|------:|------:|
| gcc   |     1 |  931.4 | 904.4 | 932.8 |
| gcc   |     2 |  849.1 | 822.6 | 851.2 |
| gcc   |     3 |  815.6 | 790.6 | 818.9 |
| gcc   |     4 |  794.1 | 770.7 | 798.0 |
| gcc   |     5 |  785.7 | 760.7 | 788.8 |
| clang |     1 |  705.5 | 683.2 | 693.8 |
| clang |     2 |  670.0 | 649.2 | 660.7 |
| clang |     3 |  659.6 | 639.8 | 651.4 |
| clang |     4 |  652.5 | 634.7 | 645.9 |
| clang |     5 |  646.9 | 625.5 | 637.7 |
2016-10-27 16:19:57 -07:00
Yann Collet ee5b725823 ZSTD_initCStream() optimization : do not allocate a CDict when no dictionary used 2016-10-27 14:20:55 -07:00
Nick Terrell eb7873a048 ZSTD_execSequence() accepts match in last 7 bytes
The zstd reference compressor will not emit a match in the last 7
bytes of a block.  The decompressor will also not accept a match
in the last 7 bytes.  This patch makes the decompressor accept a
match in the last 7 bytes.
2016-10-25 21:24:15 -07:00
Yann Collet 335ad5d4d4 added ZSTD_initDStream_usingDDict() .
slightly optimized ZSTD_initDStream() when no dictionary .
fixed ZSTD_sizeof_CStream() .
2016-10-25 17:47:02 -07:00
Yann Collet 9516234e67 first sketch for ZSTD_initCStream_usingCDict() 2016-10-25 16:19:52 -07:00
Yann Collet 62d9a7ddfd Merge pull request #429 from inikep/btopt2
Btopt2
2016-10-25 14:48:43 -07:00
Przemyslaw Skibinski 5c5f01f3da added ZSTD_btopt2 strategy 2016-10-25 12:25:07 +02:00
Yann Collet 7b5948cca7 Merge pull request #426 from terrelln/fixes
Fix various {A, M}SAN bugs
2016-10-24 23:42:26 -07:00
Yann Collet 37d130031d updated comments on context re-use 2016-10-24 17:22:12 -07:00
Nick Terrell b2c39a22b0 Fix compiler narrowing warning 2016-10-24 14:50:13 -07:00
Nick Terrell f698ad6deb Merge remote-tracking branch 'upstream/dev' into fixes
* upstream/dev:
  added doc\zstd_manual.html
  added contrib\gen_html
  zstd_compression_format.md moved to doc/
  Fix small bug in ZSTD_execSequence()
  improved ZSTD_compressBlock_opt_extDict_generic
  protect ZSTD_decodeFrameHeader() from invalid usage, as suggested by @spaskob
  zstd_opt.h: small improvement in compression ratio
  improved dicitonary segment merge
  use implicit rules to compile zstd_decompress.c
  detect early impossible decompression scenario in legacy decoder v0.5
  no repeat mode in legacy v0.5
  fixed invalid invocation of dictionary in legacy decoder v0.5
  fix edge case
  fix command line interpretation
  fixed minor corner case
  zstd.h: added the Introduction section
  fixed clang 3.5 warnings
  zstd.h: updated comments
2016-10-24 13:10:13 -07:00
Yann Collet 4239a207dd Merge pull request #425 from inikep/dev11
Doc
2016-10-24 11:11:40 -07:00
Nick Terrell f9c9af3c2e Reject dictionaries with incomplete entropy tables
If a dictionary specifies that a symbol has probability zero in its
`matchLength`, `literalLength`, or `offset` FSE table, but the symbol
appears when compressing input, the compressor fails.

Ensure that dictionaries support all `matchLength`, and `literalLength`
codes.  They must also support all of the `offset` codes required to
represent every possible offset that can appear in the first block.
2016-10-24 10:42:44 -07:00
Przemyslaw Skibinski 984b66cd72 added contrib\gen_html 2016-10-24 15:59:51 +02:00
Przemyslaw Skibinski 3ee94a7600 zstd_compression_format.md moved to doc/ 2016-10-24 15:58:07 +02:00
Yann Collet 97611611a3 Merge pull request #423 from terrelln/exec-seq-patch
Fix small bug in ZSTD_execSequence()
2016-10-21 17:02:06 -07:00
Nick Terrell ae1cb3b3d0 Fix small bug in ZSTD_execSequence()
`memmove(op, match, sequence.matchLength)` is not the desired behavior.
Overlap is allowed, and handled as if we did `*op++ = *match++`, which
is not how `memmove()` handles overlap.

Only triggered if both of the following conditions are met:
* The match spans extDict & currentPrefixSegment
* `oLitEnd <= oend_w < oLitEnd + length1 < oMatchEnd <= oend`.

These two conditions imply that the block is less than 15 bytes long.
This bug isn't triggered by the streaming API, because it allocates
enough space for the window size + the block size, so there cannot be
a match that is within 8 bytes of the end and overlaps with itself.
It cannot be triggered by the block decompression API because all of
the decompressed data is in the currentPrefixSegment.

Introduced by commit 7158584399
2016-10-21 12:13:44 -07:00
Przemyslaw Skibinski 4732074a71 improved ZSTD_compressBlock_opt_extDict_generic 2016-10-21 11:19:00 +02:00
Yann Collet da3bd8b6de protect ZSTD_decodeFrameHeader() from invalid usage, as suggested by @spaskob 2016-10-20 20:11:00 -07:00
Przemyslaw Skibinski d365ae3497 zstd_opt.h: small improvement in compression ratio 2016-10-20 11:49:02 +02:00
Przemyslaw Skibinski 575ab00db7 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2016-10-20 11:01:52 +02:00
Nick Terrell d760529a05 Fix stack buffer overrun when weightTotal == 0
If `weightTotal == 0`, then `BIT_highbit32(weightTotal)` is
undefined behavior in the case that it calls `__builtin_clz()`.
If `tableLog == HUF_TABLELOG_ABSOLUTEMAX` then we will access one
byte beyond the end of the buffer.
2016-10-19 11:39:11 -07:00
Nick Terrell bb68062c59 Unitialized memory read in ZSTD_decodeSeqHeaders()
Caused by two things:
1. Not checking that `ip` is in range except for the first byte.
2. `ZSTDv0{5,6}_decodeLiteralsBlock()` could return a value larger than `srcSize`.
2016-10-18 16:41:33 -07:00
Yann Collet 52c1bf93fe improved dicitonary segment merge 2016-10-18 16:34:58 -07:00
Nick Terrell 7b06ad7a05 Backport fix from commit 125d817
This fixes a read of unitialized memory.
Full commit hash: 125d81774f.
2016-10-18 14:52:34 -07:00
Nick Terrell f45b157d95 Backport fix from commit 9e8b09a
Fixes uninitialized memory reads.
Full commit hash: 9e8b09a7bd
2016-10-18 14:22:49 -07:00
Yann Collet f7906d5955 detect early impossible decompression scenario in legacy decoder v0.5 2016-10-18 13:48:32 -07:00
Yann Collet 9313c8d953 no repeat mode in legacy v0.5 2016-10-18 13:36:15 -07:00
Yann Collet 83d7bdee4b fixed invalid invocation of dictionary in legacy decoder v0.5 2016-10-18 12:25:43 -07:00
Yann Collet 197a55ee7b fix edge case 2016-10-18 11:27:52 -07:00
Nick Terrell fd98087047 Fix stack buffer overflow in HUF_readCTable()
If `w ==0` on line 153, then `CTable[n].nbBits == tableLog + 1`.
Then `nbPerRank[CTable[n].nbBits]` and `valPerRank[CTable[n].nbBits]`
are stack buffer overflows.
2016-10-17 18:16:59 -07:00
Yann Collet 06573e17be fixed minor corner case 2016-10-17 17:28:28 -07:00
Nick Terrell bfd943ace5 Fix buffer overrun in ZSTD_loadDictEntropyStats()
The table log set by `FSE_readNCount()` was not checked in
`ZSTD_loadDictEntropyStats()`.  This caused `FSE_buildCTable()`
to stack/heap overflow in a few places.

The benchmarks look good, there is no obvious compression performance regression:

  > ./zstds/zstd.opt.0 -i10 -b1 -e10 ~/bench/silesia.tar
   1#silesia.tar       : 211988480 ->  73656930 (2.878), 271.6 MB/s , 716.8 MB/s
   2#silesia.tar       : 211988480 ->  70162842 (3.021), 204.8 MB/s , 671.1 MB/s
   3#silesia.tar       : 211988480 ->  66997986 (3.164), 156.8 MB/s , 658.6 MB/s
   4#silesia.tar       : 211988480 ->  66002591 (3.212), 136.4 MB/s , 665.3 MB/s
   5#silesia.tar       : 211988480 ->  65008480 (3.261),  98.9 MB/s , 647.0 MB/s
   6#silesia.tar       : 211988480 ->  62979643 (3.366),  65.2 MB/s , 670.4 MB/s
   7#silesia.tar       : 211988480 ->  61974560 (3.421),  44.9 MB/s , 688.2 MB/s
   8#silesia.tar       : 211988480 ->  61028308 (3.474),  32.4 MB/s , 711.9 MB/s
   9#silesia.tar       : 211988480 ->  60416751 (3.509),  21.1 MB/s , 718.1 MB/s
  10#silesia.tar       : 211988480 ->  60174239 (3.523),  22.2 MB/s , 721.8 MB/s

  > ./compress_zstds/zstd.opt.1 -i10 -b1 -e10 ~/bench/silesia.tar
   1#silesia.tar       : 211988480 ->  73656930 (2.878), 273.8 MB/s , 722.0 MB/s
   2#silesia.tar       : 211988480 ->  70162842 (3.021), 203.2 MB/s , 666.6 MB/s
   3#silesia.tar       : 211988480 ->  66997986 (3.164), 157.4 MB/s , 666.5 MB/s
   4#silesia.tar       : 211988480 ->  66002591 (3.212), 132.1 MB/s , 661.9 MB/s
   5#silesia.tar       : 211988480 ->  65008480 (3.261),  96.8 MB/s , 641.6 MB/s
   6#silesia.tar       : 211988480 ->  62979643 (3.366),  63.1 MB/s , 677.0 MB/s
   7#silesia.tar       : 211988480 ->  61974560 (3.421),  44.3 MB/s , 678.2 MB/s
   8#silesia.tar       : 211988480 ->  61028308 (3.474),  33.1 MB/s , 708.9 MB/s
   9#silesia.tar       : 211988480 ->  60416751 (3.509),  21.5 MB/s , 710.1 MB/s
  10#silesia.tar       : 211988480 ->  60174239 (3.523),  21.9 MB/s , 723.9 MB/s
2016-10-17 16:55:52 -07:00
Nick Terrell 4db751668f Fix buffer overrun in ZSTD_loadEntropy()
The table log set by `FSE_readNCount()` was not checked in
`ZSTD_loadEntropy()`.  This caused `FSE_buildDTable(dctx->MLTable, ...)`
to overwrite the beginning of `dctx->hufTable`.

The benchmarks look good, there is no obvious performance regression:

  > ./zstds/zstd.opt.0 -i10 -b1 -e5 ~/bench/silesia.tar
   1#silesia.tar       : 211988480 ->  73656930 (2.878), 268.2 MB/s , 701.0 MB/s
   2#silesia.tar       : 211988480 ->  70162842 (3.021), 199.5 MB/s , 666.9 MB/s
   3#silesia.tar       : 211988480 ->  66997986 (3.164), 154.9 MB/s , 655.6 MB/s
   4#silesia.tar       : 211988480 ->  66002591 (3.212), 128.9 MB/s , 648.4 MB/s
   5#silesia.tar       : 211988480 ->  65008480 (3.261),  98.4 MB/s , 633.4 MB/s

  > ./zstds/zstd.opt.2 -i10 -b1 -e5 ~/bench/silesia.tar
   1#silesia.tar       : 211988480 ->  73656930 (2.878), 266.1 MB/s , 703.7 MB/s
   2#silesia.tar       : 211988480 ->  70162842 (3.021), 199.0 MB/s , 666.6 MB/s
   3#silesia.tar       : 211988480 ->  66997986 (3.164), 156.2 MB/s , 656.2 MB/s
   4#silesia.tar       : 211988480 ->  66002591 (3.212), 133.2 MB/s , 647.4 MB/s
   5#silesia.tar       : 211988480 ->  65008480 (3.261),  96.3 MB/s , 633.3 MB/s
2016-10-17 15:51:15 -07:00
Nick Terrell ccfcc643da Check if dict is empty before reading first byte 2016-10-17 11:46:03 -07:00
Yann Collet 2b361cf2f1 minor opt 2016-10-14 16:09:07 -07:00
Yann Collet 7933434fdf Merge branch 'dev' of github.com:facebook/zstd into dev 2016-10-14 13:32:35 -07:00
Yann Collet d4cda27b63 new command -M#, to limit memory usage during decompression (#403) 2016-10-14 13:32:20 -07:00
Nick Terrell 3b9cdf9220 Fix ubsan failures (pass NULL to memcpy) 2016-10-12 20:54:42 -07:00
Yann Collet 5d919e7ac3 added ZSTD_error_frameParameter_windowTooLarge (#403) 2016-10-12 17:29:24 -07:00
Yann Collet e19111c42f make creates libzstd binaries (#415) 2016-10-12 11:09:36 -07:00
Yann Collet 8b70d012f0 fix cmake 2016-10-12 10:23:53 -07:00
Yann Collet 38fb0dc4cf Merge pull request #416 from terrelln/exec-sequence
Fix ZSTD_execSequence() edge case
2016-10-12 10:17:53 -07:00
Nick Terrell 7158584399 Fix ZSTD_execSequence() edge case 2016-10-12 10:05:26 -07:00
Yann Collet f52cd03e73 bumped version number 2016-10-11 17:29:27 -07:00
Yann Collet ef2357d0d3 created error_private.c, so that a single list of error strings get included 2016-10-11 17:24:50 -07:00
Yann Collet 14efab827b added zstd_errors.h to include installation 2016-10-11 16:51:29 -07:00
Yann Collet a17fd7312a changed error_public.h into zstd_errors.h 2016-10-11 16:41:09 -07:00
Yann Collet 18b51b99c0 sync fse 2016-10-11 08:21:09 -07:00
inikep 2d2613399a zstd.h: added the Introduction section 2016-10-06 16:28:21 +02:00
inikep ba1db376ac fixed clang 3.5 warnings 2016-10-06 14:22:48 +02:00
inikep 82057aa7ec zstd.h: updated comments 2016-10-06 13:23:52 +02:00
Yann Collet df6797447f update dictionary builder warning comments 2016-09-27 15:14:32 +02:00
Yann Collet 47094ea66b added comment on filePos 2016-09-26 18:03:33 +02:00
Yann Collet cf409a7e2a fixed : init*_advanced() followed by reset() with different pledgedSrcSiz 2016-09-26 16:41:05 +02:00
Yann Collet 2f2639438a zstreamtest can fuzztest pledgedSrcSize 2016-09-26 14:06:08 +02:00
Christophe Chevalier dc245e91cb Changed to use ZSTDLIBv06_API and ZSTDLIBv07_API for DLL exports to fix warning
- changed name to prevent collision with ZSTDLIB_API used by non-legacy dll exports
2016-09-23 17:09:36 +02:00
Yann Collet 21412bb3f6 Merge branch 'dev' of github.com:Cyan4973/zstd into dev 2016-09-22 15:57:56 +02:00
Yann Collet 51f4d566c2 small decompression speed boost for very small data 2016-09-22 15:57:28 +02:00
Yann Collet 97b378a6f8 Streaming : dictionary compression on multiple files / segments can correctly provide srcSize into header (when provided) using pledgedSrcSize. 2016-09-21 17:20:19 +02:00
Yann Collet 993060e0f2 cli : better adaptation to small files 2016-09-21 16:46:08 +02:00
Yann Collet 1eb2fdc74f bumped version number 2016-09-18 12:21:47 +02:00
Yann Collet a6bdf55759 fixed memory leak 2016-09-15 17:02:06 +02:00
Yann Collet 644a8da88a fixed minor conversion warning 2016-09-15 16:16:21 +02:00
Yann Collet 4cb212938c introduced ZSTD_resetCStream() 2016-09-15 14:54:07 +02:00
Yann Collet fa0c09760c variable renaming 2016-09-15 14:11:01 +02:00
Yann Collet d7c6589df8 support ZSTD_sizeof_*() on NULL
added ZSTD_sizeof_CDict()
2016-09-15 02:57:27 +02:00
Yann Collet e91c4b4cef introduced ZSTD_resetDStream() .
added : ZSTD_sizeof_DDict()
2016-09-14 16:55:44 +02:00
Yann Collet d092d77cfc minor variable renaming 2016-09-14 16:14:57 +02:00
Yann Collet 64deef3bee Fixed srcSize=1 2016-09-14 00:16:07 +02:00
Yann Collet 26ec254066 new strategy for faster DDict decompression 2016-09-13 16:52:16 +02:00
Yann Collet ac175d46d4 updated comments 2016-09-13 00:51:47 +02:00
Yann Collet a3481d6de0 make uninstall 2016-09-12 05:04:26 +02:00
Yann Collet b3060f7a9e changed streaming decoder behavior : now, when all compressed frame is consumed, it means decompression is completed, with regenerated data fully flushed. 2016-09-09 16:44:16 +02:00
Yann Collet 01c199226a updated decompression streaming example 2016-09-08 19:29:04 +02:00
Yann Collet 5c6d244973 Merge branch 'dev' of github.com:facebook/zstd into dev 2016-09-07 14:54:54 +02:00
Yann Collet ac8bace6b1 support large skippable frames 2016-09-07 14:54:23 +02:00
Yann Collet 0e07bf3f60 added comments on searchLength min / max (#337) 2016-09-07 06:33:02 +02:00
Yann Collet 95d07d7447 introduced CHECK_E 2016-09-06 16:38:51 +02:00
Yann Collet 3e21ec5b01 introduced CHECK_F 2016-09-06 15:36:19 +02:00
Yann Collet 5c956d593c FORCE_INLINE common definition 2016-09-06 15:05:19 +02:00
Yann Collet edbcd9f5b2 fixed zbufftest 2016-09-06 14:30:57 +02:00
Yann Collet b624922b14 fixed checksum 2016-09-06 11:16:57 +02:00
Yann Collet a7737f6a60 improved compression on small files when using same parameters 2016-09-06 09:44:59 +02:00
Yann Collet 7ae67bb18a small compression speed gains with using_CDict 2016-09-06 06:28:05 +02:00
Yann Collet 1d4208c029 clarified streaming decompression inlined doc 2016-09-06 05:16:40 +02:00
Yann Collet 7c83dfd5c2 ZSTD_frameHeaderSize_prefix (#340), as result of ZSTD_initStream 2016-09-05 19:47:43 +02:00
Yann Collet fa72f6bdce clarified inline doc for streaming 2016-09-05 17:39:56 +02:00
Yann Collet c73a8109bb Merge pull request #344 from inikep/dev10
unified error codes for legacy decoders
2016-09-05 07:46:33 -07:00
inikep 45db83f98d ZSTD_decodeLiteralsBlock renamed to ZSTDv01_decodeLiteralsBlock 2016-09-05 14:46:24 +02:00
inikep 476964f6a1 ZSTD_decodeSeqHeaders renamed to ZSTDv01_decodeSeqHeaders 2016-09-05 13:34:57 +02:00
inikep c13faa1b0f legacy decoders: restored #include <intrin.h> for VC++ 2016-09-05 13:25:07 +02:00
inikep 8161e7321a unified error codes for legacy decoders 2016-09-05 12:29:51 +02:00
Thomas Klausner b85cdabd50 Enable install targets for NetBSD. 2016-09-04 14:37:57 +02:00
Yann Collet 33a0465a51 fixed a few links 2016-09-02 22:11:49 -07:00
Yann Collet d56dbc02d3 removed g_displayLevel 2016-09-02 17:28:41 -07:00
Yann Collet 855766d73d clarified dictionary in format description 2016-09-02 17:04:49 -07:00
Yann Collet d725427a3c g_time => local displayTime 2016-09-02 15:32:39 -07:00
Yann Collet 1563bfeabc fixing FORCE_INLINE for older compilers (#330) 2016-09-02 11:44:21 -07:00
Yann Collet 7304eb7c09 bumped version number 2016-09-01 15:49:26 -07:00
Yann Collet 901e85fe26 version bump 2016-08-31 07:51:25 -07:00
Yann Collet 1c59c20903 removed redundant files 2016-08-31 07:15:44 -07:00
Yann Collet 599c69d917 minor Makefile updates 2016-08-30 13:33:20 -07:00
David Lam e10f7f3dcb merge 2016-08-30 12:03:36 -07:00
Yann Collet 4ded9e591c added boilerplate 2016-08-30 11:06:28 -07:00
Yann Collet 3b15f1f10f minor refactor 2016-08-30 09:58:50 -07:00
Yann Collet 240795bef7 Merge branch 'dev' of github.com:Cyan4973/zstd into dev 2016-08-30 06:51:55 -07:00
Yann Collet 14200a20f0 Fixed issue #304, reported by @borzunov 2016-08-30 06:51:00 -07:00
David Lam da9d3b7057 Cleanup some errors in typedef comments and remove duplicated HOWTO from zbuff_decompress.c 2016-08-29 17:31:51 -07:00
Yann Collet 09c3c8e885 Merge pull request #307 from inikep/dev08
updated README.md
2016-08-29 16:32:33 -07:00
inikep 6416b0d705 updated README.md 2016-08-29 13:04:26 +02:00
Yann Collet 23b6e05d8e ZSTD_malloc() and ZSTD_free(), to simplify customMem 2016-08-28 21:05:43 -07:00
Yann Collet 5f53b0335e fixed continuation context 2016-08-28 10:00:49 -07:00
Yann Collet 767d8f66fa legacy contexts can be re-used 2016-08-28 08:19:47 -07:00
Yann Collet 4bf317dd00 first version supporting legacy streams (transparent decoding) 2016-08-28 07:43:34 -07:00
Yann Collet e19a9ef05d update compression level table 2016-08-26 20:02:49 +02:00
Yann Collet 9a021c1aae fixed some minor clang warnings 2016-08-26 09:05:06 +02:00
Yann Collet cb5a320705 made -Wdocumentation a clang only flag 2016-08-26 08:06:36 +02:00
Yann Collet 87c18b2ebd fixed multiple minor warnings for XCode 2016-08-26 01:43:47 +02:00
Yann Collet 0d59a6f73a removed debug strings 2016-08-25 22:42:46 +02:00
Yann Collet 5a02b69215 reinforced fix for huge files 2016-08-25 22:24:59 +02:00
Yann Collet 96bdd87de4 fixed : compression bug on very large files 2016-08-25 14:34:42 +02:00