Yann Collet
9313c8d953
no repeat mode in legacy v0.5
2016-10-18 13:36:15 -07:00
Yann Collet
83d7bdee4b
fixed invalid invocation of dictionary in legacy decoder v0.5
2016-10-18 12:25:43 -07:00
Yann Collet
197a55ee7b
fix edge case
2016-10-18 11:27:52 -07:00
Nick Terrell
fd98087047
Fix stack buffer overflow in HUF_readCTable()
...
If `w ==0` on line 153, then `CTable[n].nbBits == tableLog + 1`.
Then `nbPerRank[CTable[n].nbBits]` and `valPerRank[CTable[n].nbBits]`
are stack buffer overflows.
2016-10-17 18:16:59 -07:00
Yann Collet
33fdd099bb
fix command line interpretation
2016-10-17 17:48:48 -07:00
Yann Collet
06573e17be
fixed minor corner case
2016-10-17 17:28:28 -07:00
Nick Terrell
bfd943ace5
Fix buffer overrun in ZSTD_loadDictEntropyStats()
...
The table log set by `FSE_readNCount()` was not checked in
`ZSTD_loadDictEntropyStats()`. This caused `FSE_buildCTable()`
to stack/heap overflow in a few places.
The benchmarks look good, there is no obvious compression performance regression:
> ./zstds/zstd.opt.0 -i10 -b1 -e10 ~/bench/silesia.tar
1#silesia.tar : 211988480 -> 73656930 (2.878), 271.6 MB/s , 716.8 MB/s
2#silesia.tar : 211988480 -> 70162842 (3.021), 204.8 MB/s , 671.1 MB/s
3#silesia.tar : 211988480 -> 66997986 (3.164), 156.8 MB/s , 658.6 MB/s
4#silesia.tar : 211988480 -> 66002591 (3.212), 136.4 MB/s , 665.3 MB/s
5#silesia.tar : 211988480 -> 65008480 (3.261), 98.9 MB/s , 647.0 MB/s
6#silesia.tar : 211988480 -> 62979643 (3.366), 65.2 MB/s , 670.4 MB/s
7#silesia.tar : 211988480 -> 61974560 (3.421), 44.9 MB/s , 688.2 MB/s
8#silesia.tar : 211988480 -> 61028308 (3.474), 32.4 MB/s , 711.9 MB/s
9#silesia.tar : 211988480 -> 60416751 (3.509), 21.1 MB/s , 718.1 MB/s
10#silesia.tar : 211988480 -> 60174239 (3.523), 22.2 MB/s , 721.8 MB/s
> ./compress_zstds/zstd.opt.1 -i10 -b1 -e10 ~/bench/silesia.tar
1#silesia.tar : 211988480 -> 73656930 (2.878), 273.8 MB/s , 722.0 MB/s
2#silesia.tar : 211988480 -> 70162842 (3.021), 203.2 MB/s , 666.6 MB/s
3#silesia.tar : 211988480 -> 66997986 (3.164), 157.4 MB/s , 666.5 MB/s
4#silesia.tar : 211988480 -> 66002591 (3.212), 132.1 MB/s , 661.9 MB/s
5#silesia.tar : 211988480 -> 65008480 (3.261), 96.8 MB/s , 641.6 MB/s
6#silesia.tar : 211988480 -> 62979643 (3.366), 63.1 MB/s , 677.0 MB/s
7#silesia.tar : 211988480 -> 61974560 (3.421), 44.3 MB/s , 678.2 MB/s
8#silesia.tar : 211988480 -> 61028308 (3.474), 33.1 MB/s , 708.9 MB/s
9#silesia.tar : 211988480 -> 60416751 (3.509), 21.5 MB/s , 710.1 MB/s
10#silesia.tar : 211988480 -> 60174239 (3.523), 21.9 MB/s , 723.9 MB/s
2016-10-17 16:55:52 -07:00
Nick Terrell
4db751668f
Fix buffer overrun in ZSTD_loadEntropy()
...
The table log set by `FSE_readNCount()` was not checked in
`ZSTD_loadEntropy()`. This caused `FSE_buildDTable(dctx->MLTable, ...)`
to overwrite the beginning of `dctx->hufTable`.
The benchmarks look good, there is no obvious performance regression:
> ./zstds/zstd.opt.0 -i10 -b1 -e5 ~/bench/silesia.tar
1#silesia.tar : 211988480 -> 73656930 (2.878), 268.2 MB/s , 701.0 MB/s
2#silesia.tar : 211988480 -> 70162842 (3.021), 199.5 MB/s , 666.9 MB/s
3#silesia.tar : 211988480 -> 66997986 (3.164), 154.9 MB/s , 655.6 MB/s
4#silesia.tar : 211988480 -> 66002591 (3.212), 128.9 MB/s , 648.4 MB/s
5#silesia.tar : 211988480 -> 65008480 (3.261), 98.4 MB/s , 633.4 MB/s
> ./zstds/zstd.opt.2 -i10 -b1 -e5 ~/bench/silesia.tar
1#silesia.tar : 211988480 -> 73656930 (2.878), 266.1 MB/s , 703.7 MB/s
2#silesia.tar : 211988480 -> 70162842 (3.021), 199.0 MB/s , 666.6 MB/s
3#silesia.tar : 211988480 -> 66997986 (3.164), 156.2 MB/s , 656.2 MB/s
4#silesia.tar : 211988480 -> 66002591 (3.212), 133.2 MB/s , 647.4 MB/s
5#silesia.tar : 211988480 -> 65008480 (3.261), 96.3 MB/s , 633.3 MB/s
2016-10-17 15:51:15 -07:00
Nick Terrell
ccfcc643da
Check if dict is empty before reading first byte
2016-10-17 11:46:03 -07:00
Yann Collet
2b361cf2f1
minor opt
2016-10-14 16:09:07 -07:00
Nick Terrell
8c6c686d0a
[pzstd] Fix lantent bug in WorkQueue::push()
2016-10-14 15:26:56 -07:00
Nick Terrell
baa152e56e
[pzstd] Add Logger class
2016-10-14 15:26:55 -07:00
Nick Terrell
e9e151ce31
[pzstd] Reuse ZSTD_{C,D}Stream
2016-10-14 15:26:55 -07:00
Nick Terrell
48294b57c3
[pzstd] Put ErrorHolder into SharedState
2016-10-14 15:26:55 -07:00
Nick Terrell
9b603ee284
[pzstd] Run the reading thread separately
2016-10-14 15:26:55 -07:00
Nick Terrell
4cb5e90a5c
[pzstd] Add asan and tsan tests to travis
...
gcc-6 tsan is buggy.
It fails to use the correct linker.
It is also broken with `-pie` with linux kernels newer than 4.1, but previous versions require `-pie`...
2016-10-14 15:26:55 -07:00
Nick Terrell
96e0702c00
[pzstd] Print the correct width ints
2016-10-14 15:26:55 -07:00
Nick Terrell
8b4e84249b
[pzstd] Fix Makefile
2016-10-14 15:26:50 -07:00
Yann Collet
70077bc9bb
refactor for long commands
2016-10-14 14:41:17 -07:00
Yann Collet
d7b120ab5c
added long commands --memory= and --memlimit-decompress=
2016-10-14 14:22:32 -07:00
Yann Collet
1122349ac2
added long comment --memlimit=
2016-10-14 14:07:11 -07:00
Yann Collet
7933434fdf
Merge branch 'dev' of github.com:facebook/zstd into dev
2016-10-14 13:32:35 -07:00
Yann Collet
d4cda27b63
new command -M#, to limit memory usage during decompression ( #403 )
2016-10-14 13:32:20 -07:00
Yann Collet
c8b1ecf4ba
Merge pull request #417 from terrelln/ubsan-failures
...
Fix ubsan failures (pass NULL to memcpy)
2016-10-13 03:37:22 -07:00
Nick Terrell
3b9cdf9220
Fix ubsan failures (pass NULL to memcpy)
2016-10-12 20:54:42 -07:00
Yann Collet
5d919e7ac3
added ZSTD_error_frameParameter_windowTooLarge ( #403 )
2016-10-12 17:29:24 -07:00
Yann Collet
e405c7a126
updated visual projects
2016-10-12 15:29:22 -07:00
Yann Collet
6f2f142b23
Merge pull request #406 from pixelb/pzstd-test-headless
...
pzstd: fix test failure on headless build
2016-10-12 12:23:11 -07:00
Yann Collet
e19111c42f
make creates libzstd binaries ( #415 )
2016-10-12 11:09:36 -07:00
Yann Collet
8b70d012f0
fix cmake
2016-10-12 10:23:53 -07:00
Yann Collet
38fb0dc4cf
Merge pull request #416 from terrelln/exec-sequence
...
Fix ZSTD_execSequence() edge case
2016-10-12 10:17:53 -07:00
Nick Terrell
7158584399
Fix ZSTD_execSequence() edge case
2016-10-12 10:05:26 -07:00
Yann Collet
f52cd03e73
bumped version number
2016-10-11 17:29:27 -07:00
Yann Collet
ef2357d0d3
created error_private.c, so that a single list of error strings get included
2016-10-11 17:24:50 -07:00
Yann Collet
14efab827b
added zstd_errors.h to include installation
2016-10-11 16:51:29 -07:00
Yann Collet
a17fd7312a
changed error_public.h into zstd_errors.h
2016-10-11 16:41:09 -07:00
Yann Collet
564b94183c
Merge pull request #414 from terrelln/license
...
Remove references to GPLv2 license
2016-10-11 16:11:48 -07:00
Nick Terrell
3e82f2b61a
Remove references to GPLv2 license
...
License headers added to Makefiles were taken from `zstd/Makefile`.
2016-10-11 15:28:40 -07:00
Yann Collet
18b51b99c0
sync fse
2016-10-11 08:21:09 -07:00
Nick Terrell
87629978d3
[pzstd] Fix latent bug in WorkQueue
2016-10-06 15:51:58 -07:00
Pádraig Brady
0dd8071e2e
pzstd: fix test failure on headless build
...
Remove this test as pass/fail status is
dependent on whether I/O is connected to a tty.
It currrently passes on a tty but fails on
a package build system for example.
2016-10-06 22:38:27 +01:00
Nick Terrell
b0f6d73002
[pzstd] Remove empty else statement
2016-10-06 12:58:23 -07:00
Nick Terrell
5fa71396e0
[pzstd] Fix typo in readme
2016-10-06 12:45:47 -07:00
inikep
2d2613399a
zstd.h: added the Introduction section
2016-10-06 16:28:21 +02:00
inikep
ba1db376ac
fixed clang 3.5 warnings
2016-10-06 14:22:48 +02:00
inikep
82057aa7ec
zstd.h: updated comments
2016-10-06 13:23:52 +02:00
Yann Collet
f405b8acbe
Merge pull request #401 from inikep/dev
...
Dev
2016-10-05 17:56:47 +02:00
inikep
0fa190b5eb
.travis.yml: added "make clean" before versionsTest
2016-10-05 13:41:37 +02:00
inikep
6ac85a414a
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev
2016-10-05 12:17:51 +02:00
inikep
23cec875cd
fixed Makefile targets: zstd-small, zstd-decompress, zstd-compress
2016-10-05 11:56:22 +02:00
