13bfe885aa
edited ZSTD_initCStream_advanced() comment
2017-10-16 14:06:22 -07:00
241c57adff
playtest : minor cosmetic changes
2017-10-16 14:01:42 -07:00
fc8d293460
dictionary compression use correct file size estimation
...
when determining compression parameters
to compress one file only.
For multiple files, it still "bets" that files are going to be small.
There was also a bug recently added in ZSTD_CCtx_loadDictionary_advanced()
making it incapable to use pledgedSrcSize to determine compression parameters.
2017-10-14 01:21:43 -07:00
5eed8e7a55
changed API comments to invite using macro ZSTD_CONTENTSIZE_UNKNOWN
...
to mean "pledgedSrcSize is not known at init time" instead of `0`.
Note that, a few prototypes created and documented with `0` to mean "unknown" still interpret "0" as unknown,
to avoid breaking 3rd party applications which depend on this behavior.
But this value is no longer recommended to mean "unknown".
In some future version, it might be possible to switch "0" to mean "empty",
as is already the case for several prototypes.
The advantage is that pledgedSrcSize field would have same behavior accross entire API,
making it easier to reason about.
Note that all concerned prototypes belong to the "experimental" API section.
srcSize is controlled at end of compression,
so if someone uses "0" to mean "unknown" while it effectively means "empty",
this is immediately caught by the compression function, which generates an error code : ZSTD_ERROR_srcSize_wrong
2017-10-14 00:32:06 -07:00
9ef32b3cf1
minor : zstd -l -v display each file name
2017-10-14 00:02:32 -07:00
43914f6a20
add test for presence of decompressed size field in header
...
previous test was "display" only, it wouldn't trigger an error in case of missing header.
2017-10-13 23:47:01 -07:00
beb9b4b398
fixed ZSTDMT_initCStream() when contentSizeFlag==1 by default
...
and a wrong test in zstreamtest --mt
2017-10-13 19:09:30 -07:00
213ef3b510
fixed ZSTD_initCStream_advanced() behavior, which depends on contentSizeFlag,
...
and a stream fuzzer test, which was incorrect
(relied on 0 being unconditionnally transformed into `ZSTD_CONTENTSIZE_UNKNOWN`)
2017-10-13 19:01:58 -07:00
3c1e3f8ec9
contentSizeFlag enabled by default would also fail for streaming and MT operations
...
fixed
2017-10-13 18:32:06 -07:00
fb44516641
ensure fParams.contentSizeFlag starts at 1
...
such default was failing for ZSTD_compressBegin/ZSTD_compressContinue
fixed too
2017-10-13 17:39:13 -07:00
dd18d73e7e
fileio: content size is enabled by default
2017-10-13 16:32:18 -07:00
df6b844b4a
Merge branch 'dev' of github.com:facebook/zstd into dev
2017-10-13 02:36:40 -07:00
a9e5705077
minor code formatting
...
added a trace during sequence encoding
2017-10-13 02:36:16 -07:00
5da6693fb6
Merge pull request #885 from terrelln/dev
...
[doc] Add images for release notes
2017-10-09 16:50:09 -07:00
6d8778d8e7
[doc] Add images for release notes
2017-10-09 16:44:31 -07:00
7f6a783862
fixed a small error in decodeCorpus
...
a compressed block must be strictly smaller than its decompressed size.
2017-10-07 15:19:52 -07:00
4252621e26
playtests: do not use cat on large files
...
some target have limitation making cat incompatible with large files
(namely debian hurd-i386)
2017-10-05 20:21:59 -07:00
ba55dc8b5e
Merge pull request #882 from terrelln/list
...
[zstdcli] Add window size to verbose list
2017-10-04 13:39:30 -07:00
6dd958eea2
[zstdcli] Add window size to verbose list
...
```
> zstd --list -v file1 file2 file3
*** zstd command line interface 64-bits v1.3.2, by Yann Collet ***
Window Size: 512.00 KB (524288 B)
Compressed Size: 0.02 KB (19 B)
Check: XXH64
Window Size: 8192.00 KB (8388608 B)
Compressed Size: 0.02 KB (19 B)
Check: XXH64
Window Size: 512.00 KB (524288 B)
Compressed Size: 0.01 KB (15 B)
Check: None
```
2017-10-04 12:26:28 -07:00
c85141caed
Merge pull request #881 from terrelln/dev
...
Ensure dictionary Huff table can encode any symbol
2017-10-03 14:49:07 -07:00
a86a7097ec
Ensure dictionary Huff table can encode any symbol
...
* Ensure that the dictionary Huffman CTable has maxSymbolValue 255.
* Fix a stack buffer overflow during compression dictionary loading.
2017-10-03 13:22:13 -07:00
67478f4cb0
fixed minor conversion warnings for printf
...
in debug mode
2017-10-02 17:28:57 -07:00
9b166d2291
Merge branch 'dev' of github.com:facebook/zstd into dev
2017-10-02 16:34:26 -07:00
3b27ed41fd
Merge branch 'srcSize' into dev
2017-10-02 16:34:14 -07:00
7e00df4a49
bumped version number
...
and updated NEWS in anticipation for release
2017-10-02 16:27:25 -07:00
004fd34fd9
Merge pull request #876 from facebook/srcSize
...
CLI Fix : srcSize written in frame headers when compressing multiple files
2017-10-02 15:02:05 -07:00
7304a63c12
Merge pull request #879 from terrelln/block-size
...
[libzstd] Set CLEVEL_CUSTOM correctly
2017-10-02 14:51:47 -07:00
86e83e926f
[libzstd] Set CLEVEL_CUSTOM correctly
...
In `ZSTD_compressBegin_advanced()`, `ZSTD_parameters` are used to set the
compression parameters, but the level didn't get set to `CLEVEL_CUSTOM`, so
`ZSTD_compressBlock()` used the wrong parameters when checking the source
size.
2017-10-02 13:43:30 -07:00
4946993f87
removed isRegularFile parameter
...
no longer useful : size of src is determined for each file.
2017-10-02 12:29:25 -07:00
7f580f9ee8
interruption handler and variable are static
2017-10-02 11:39:05 -07:00
0d58aaf6f0
/contrib: fixed license header
...
removed last reference to PATENTS file
2017-10-02 02:07:17 -07:00
fe5444bc66
removed the statement for all versions of Visual Studio
2017-10-02 02:02:16 -07:00
51d82d5516
same error in Visual Studio 2012 ...
2017-10-02 01:12:40 -07:00
ed7ae4c9bd
The issue also impacts Visual Studio 2010
2017-10-02 00:45:28 -07:00
6e7ba3df2f
added (void)sig to avoid compilers complaining that sig is not used.
2017-10-02 00:19:47 -07:00
82bc200f82
conditionnally removed invocation that generates a buggy warning with Visual Studio 2008
2017-10-02 00:02:24 -07:00
5e4ad557bc
Merge branch 'dev' of github.com:facebook/zstd into dev
2017-10-01 15:33:57 -07:00
bd18095edc
blindfix for Visual : minor casting issue
...
should not happen since SIGIGN is provided by <signal.h>,
so it should work "ouf of the box"
2017-10-01 15:32:48 -07:00
b600b5bafd
Merge pull request #877 from facebook/compressBound
...
added ZSTD_COMPRESSBOUND() as a macro
2017-10-01 14:59:38 -07:00
00fc1ba8dd
cli: add Ctrl-C support, requested by @mike155 in #854
...
Now, pressing Ctrl-C during compression or decompression
will erase operation artefact (unfinished destination file)
before leaving execution.
2017-10-01 12:10:26 -07:00
5db19b8685
added comment on ZSTD_COMPRESSBOUND()
...
as requested by @terrelln
2017-10-01 11:32:38 -07:00
6e930c13d1
Merge branch 'dev' into compressBound
2017-10-01 11:24:02 -07:00
76ac0b2d99
macro compatible with scenario where windowSize = 1024 (minimum)
2017-09-30 15:34:44 -07:00
dc404119e5
ZSTD_adjustCParams_internal : minor optimization
2017-09-30 15:02:40 -07:00
96bb29aa14
Merge pull request #878 from terrelln/adjust
...
Don't `size -= 1` in ZSTD_adjustCParams()
2017-09-30 14:52:54 -07:00
c5d6dde502
Don't `size -= 1` in ZSTD_adjustCParams()
...
The window size could end up too small if the source size is 2^n + 1.
Credit to OSS-Fuzz
2017-09-30 14:20:06 -07:00
ee1ed78fcb
fix proper naming on FSE_createCTable() arguments in fse.h
2017-09-30 11:08:50 -07:00
5b10345b26
added ZSTD_COMPRESSBOUND() as a macro
...
ZSTD_compressBound() works fine, but is only useful for dynamic allocation.
For static allocation, only a macro can provide the amount during compilation time.
2017-09-29 23:17:41 -07:00
e580dc6a4a
Merge pull request #860 from felixhandte/zstd-lz4-support-tests
...
Add Default LZ4 Support When Available
2017-09-29 22:32:54 -07:00
8afb151c9b
cli: fixed wrong initialization in MT mode
...
It's not good to mix old and new API
ZSTD_resetCStream() doesn't just set pledgedSrcSize :
it also sets the CCtx for a single thread compression.
Problem is, when 2+ threads are defined in cctx->requestedParams,
ZSTD_compress_generic() will want to start MT compression,
since initialization is supposed to have already happened (thanks to ZSTD_resetCStream())
except that the underlying ZSTDMT_CCtx* object is not created,
resulting in a segfault.
This is an invalid construction
(correct one is to use ZSTD_CCtx_setPledgedSrcSize()).
I haven't found a nice way to mitigate this impact if someone makes the same mistake.
At some point, removing the old API to keep only the new API within fileio.c will limit these risks.
2017-09-29 22:14:37 -07:00
Yann Collet
Yann Collet
Nick Terrell