a86a7097ec
Ensure dictionary Huff table can encode any symbol
...
* Ensure that the dictionary Huffman CTable has maxSymbolValue 255.
* Fix a stack buffer overflow during compression dictionary loading.
2017-10-03 13:22:13 -07:00
67478f4cb0
fixed minor conversion warnings for printf
...
in debug mode
2017-10-02 17:28:57 -07:00
9b166d2291
Merge branch 'dev' of github.com:facebook/zstd into dev
2017-10-02 16:34:26 -07:00
3b27ed41fd
Merge branch 'srcSize' into dev
2017-10-02 16:34:14 -07:00
7e00df4a49
bumped version number
...
and updated NEWS in anticipation for release
2017-10-02 16:27:25 -07:00
004fd34fd9
Merge pull request #876 from facebook/srcSize
...
CLI Fix : srcSize written in frame headers when compressing multiple files
2017-10-02 15:02:05 -07:00
7304a63c12
Merge pull request #879 from terrelln/block-size
...
[libzstd] Set CLEVEL_CUSTOM correctly
2017-10-02 14:51:47 -07:00
86e83e926f
[libzstd] Set CLEVEL_CUSTOM correctly
...
In `ZSTD_compressBegin_advanced()`, `ZSTD_parameters` are used to set the
compression parameters, but the level didn't get set to `CLEVEL_CUSTOM`, so
`ZSTD_compressBlock()` used the wrong parameters when checking the source
size.
2017-10-02 13:43:30 -07:00
4946993f87
removed isRegularFile parameter
...
no longer useful : size of src is determined for each file.
2017-10-02 12:29:25 -07:00
7f580f9ee8
interruption handler and variable are static
2017-10-02 11:39:05 -07:00
0d58aaf6f0
/contrib: fixed license header
...
removed last reference to PATENTS file
2017-10-02 02:07:17 -07:00
fe5444bc66
removed the statement for all versions of Visual Studio
2017-10-02 02:02:16 -07:00
51d82d5516
same error in Visual Studio 2012 ...
2017-10-02 01:12:40 -07:00
ed7ae4c9bd
The issue also impacts Visual Studio 2010
2017-10-02 00:45:28 -07:00
6e7ba3df2f
added (void)sig to avoid compilers complaining that sig is not used.
2017-10-02 00:19:47 -07:00
82bc200f82
conditionnally removed invocation that generates a buggy warning with Visual Studio 2008
2017-10-02 00:02:24 -07:00
5e4ad557bc
Merge branch 'dev' of github.com:facebook/zstd into dev
2017-10-01 15:33:57 -07:00
bd18095edc
blindfix for Visual : minor casting issue
...
should not happen since SIGIGN is provided by <signal.h>,
so it should work "ouf of the box"
2017-10-01 15:32:48 -07:00
b600b5bafd
Merge pull request #877 from facebook/compressBound
...
added ZSTD_COMPRESSBOUND() as a macro
2017-10-01 14:59:38 -07:00
00fc1ba8dd
cli: add Ctrl-C support, requested by @mike155 in #854
...
Now, pressing Ctrl-C during compression or decompression
will erase operation artefact (unfinished destination file)
before leaving execution.
2017-10-01 12:10:26 -07:00
5db19b8685
added comment on ZSTD_COMPRESSBOUND()
...
as requested by @terrelln
2017-10-01 11:32:38 -07:00
6e930c13d1
Merge branch 'dev' into compressBound
2017-10-01 11:24:02 -07:00
76ac0b2d99
macro compatible with scenario where windowSize = 1024 (minimum)
2017-09-30 15:34:44 -07:00
dc404119e5
ZSTD_adjustCParams_internal : minor optimization
2017-09-30 15:02:40 -07:00
96bb29aa14
Merge pull request #878 from terrelln/adjust
...
Don't `size -= 1` in ZSTD_adjustCParams()
2017-09-30 14:52:54 -07:00
c5d6dde502
Don't `size -= 1` in ZSTD_adjustCParams()
...
The window size could end up too small if the source size is 2^n + 1.
Credit to OSS-Fuzz
2017-09-30 14:20:06 -07:00
ee1ed78fcb
fix proper naming on FSE_createCTable() arguments in fse.h
2017-09-30 11:08:50 -07:00
5b10345b26
added ZSTD_COMPRESSBOUND() as a macro
...
ZSTD_compressBound() works fine, but is only useful for dynamic allocation.
For static allocation, only a macro can provide the amount during compilation time.
2017-09-29 23:17:41 -07:00
e580dc6a4a
Merge pull request #860 from felixhandte/zstd-lz4-support-tests
...
Add Default LZ4 Support When Available
2017-09-29 22:32:54 -07:00
8afb151c9b
cli: fixed wrong initialization in MT mode
...
It's not good to mix old and new API
ZSTD_resetCStream() doesn't just set pledgedSrcSize :
it also sets the CCtx for a single thread compression.
Problem is, when 2+ threads are defined in cctx->requestedParams,
ZSTD_compress_generic() will want to start MT compression,
since initialization is supposed to have already happened (thanks to ZSTD_resetCStream())
except that the underlying ZSTDMT_CCtx* object is not created,
resulting in a segfault.
This is an invalid construction
(correct one is to use ZSTD_CCtx_setPledgedSrcSize()).
I haven't found a nice way to mitigate this impact if someone makes the same mistake.
At some point, removing the old API to keep only the new API within fileio.c will limit these risks.
2017-09-29 22:14:37 -07:00
657946a913
Merge pull request #875 from facebook/zstdcat
...
decode more data before triggering error
2017-09-29 20:12:25 -07:00
fbd5ab7027
minor fix : no longer use fake srcSize during resource creation
...
srcSize is read and provided at each file, not at resource creation.
This used to be useful with older API, because it could not re-adapt parameters between sessions.
At some point, it will be better to remove the old code, and only keep the new_api.
It works fine by now.
2017-09-29 19:40:27 -07:00
db1668a43b
fix : srcSize written in frame header when multiple files compressed
...
This information used to be disabled when nbFiles>1.
It was badly initialized later in the code, resulting in an error.
2017-09-29 18:05:18 -07:00
7c9669f272
Merge pull request #873 from facebook/shorterTests
...
Leaner tests
2017-09-29 17:26:46 -07:00
1416bc0f07
erase existence of a buffer when it's sent out of the pool
...
In some complex scenario,
the buffer would be freed because it's too large,
another buffer would be allocated, but fail,
trigger an error,
and the general buffer pool would then be freed,
where the definition of the already freed buffer would be found
(beyond total index, but still), and freed again, resulting in double-free error.
2017-09-29 16:27:47 -07:00
8afcc80e07
decode more data before triggering error
...
fixes #874 :
when a frame is not properly terminated by a "last block" signal,
zstd -d used to detect it immediately and error out.
This version will decode and flush the last block, and only then issue an error.
2017-09-29 15:54:09 -07:00
e963800e27
zstdmt : fixed : buffer dst0 wasn't properly set to null after usage
...
now it's possible to unconditionnally invoke ZSTD_releaseAllJobRessources()
wether previous compression was completed correctly or not.
2017-09-28 23:01:31 -07:00
754ae5cc0b
removed ZSTDMT_waitForAllJobsCompleted() from ZSTDMT_freeCCtx()
...
as per @terrelln comment
2017-09-28 20:45:31 -07:00
e0065cf660
make test : removed zstreamtest unit tests for variants
...
slightly reduced time to create dictionary
at beginning of unit tests
2017-09-28 18:34:38 -07:00
47c6a95d07
zstreamtest : run unit tests only during "normal" session
...
not during --mt, --newapi and --opaque
this avoids running them 4x during `make test`
2017-09-28 18:27:22 -07:00
86b4fe5b45
adjustCParams : restored previous behavior
...
unknowns srcSize presumed small if there is a dictionary (dictSize>0)
and presumed large otherwise.
2017-09-28 18:14:28 -07:00
dc27c36495
Update documentation to reflect other format support
2017-09-28 19:43:12 -04:00
d0519d4b0c
Add CLI Program Name Detection for LZ4
2017-09-28 19:18:15 -04:00
5705d9f25a
Add basic tests for the lz4 integration
2017-09-28 19:16:43 -04:00
b93598d6a4
zstdmt : reduced maximum nb of threads
...
to avoid memory address space issues on 32-bits systems
(see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=876416#17 )
2017-09-28 13:49:12 -07:00
e4ec427720
Merge branch 'dev' into shorterTests
...
fixed conflicts
2017-09-28 12:19:28 -07:00
bbef058ae6
zstreamtest --newapi : reduced maximum allocated memory
2017-09-28 11:48:45 -07:00
8074261d00
zstdmt : move on when not enough memory for a new input buffer
...
just continue operations without input forward progress,
instead of an error that stops current compression session.
2017-09-28 11:46:19 -07:00
2cd15dd9a4
fixed minor Visual conversion warning
2017-09-28 02:33:41 -07:00
377abcc02c
zstdmt : better behavior when freeing a context right after a memory allocation error
...
wait for all jobs to be completed, so that freeing can happen safely
2017-09-28 02:23:44 -07:00
Nick Terrell
Yann Collet
Yann Collet
W. Felix Handte