ad4524d605
fix ZSTD_compressBlock() associated with CDict
...
reported by @let-def.
It's actually a bug in ZSTD_compressBegin_usingCDict()
which would pass a wrong pledgedSrcSize value (0 instead of ZSTD_CONTENTSIZE_UNKNOWN)
resulting in wrong window size, resulting in downsized seqStore,
resulting in segfault when writing into the seqStore later in the process.
Added a test in fuzzer to cover this use case (fails before the patch).
2018-05-07 12:54:13 -07:00
64bfdca5b9
Split library install target into pc, static, shared and include only target
...
Signed-off-by: Peter Seiderer <ps.report@gmx.net>
2018-04-30 20:32:32 +02:00
ca77822ddf
Fix parameter adjustment with dictionary
...
The new advanced API basically set `requestedParams = appliedParams` when
using a dictionary. This halted all parameter adjustment, which can hurt
compression ratio if, for example, the window log is small for the first
call, but the rest of the files are large.
This patch fixes the bug, and checks that the `requestedParams` don't change
in the new advanced API when using a dictionary, and generally in the fuzzer.
2018-04-25 16:32:29 -07:00
12f60b8c98
clarified documentation related to refPrefix()
2018-04-25 10:17:06 -07:00
ace856a835
updated documentation of streaming compression api
2018-04-24 14:44:27 -07:00
2c3ad05812
Fix the problem that enables DYNAMIC_BMI2 macro by mistake on ARM architecture with Clang ( #1110 )
2018-04-23 15:41:50 -07:00
e8c9dc5cea
Fix documentation
2018-04-13 12:43:38 -07:00
c0987986e5
Only reset CDict in ZSTD_CCtx_resetParameters()
2018-04-13 11:26:40 -07:00
9f76eebd17
Add ZSTD_CCtx_resetParameters() function
...
* Fix docs for `ZSTD_CCtx_reset()`.
* Add `ZSTD_CCtx_resetParameters()`.
Fixes #1094 .
2018-04-12 16:54:07 -07:00
3c3f59e68f
Enforce pledgeSrcSize whenever known ( #1106 )
...
The test fails before the patch and passes after.
Fixes #1095 .
2018-04-12 16:02:03 -07:00
280a236e9e
Add ZSTD_CCtx(Param)?_getParameter() function
...
Closes #1096 .
2018-04-12 11:50:12 -07:00
04212178b5
doc : clarified advanced API usage
...
sticky parameters only work with `ZSTD_compress_generic()`
2018-04-10 11:40:36 -07:00
ad5ba6cdcf
updated comment on parameters that can be changed during compression
2018-04-09 17:39:07 -07:00
1da629f2ad
Merge pull request #1104 from terrelln/fast-train
...
Allow negative compression levels in training
2018-04-09 14:16:20 -07:00
569e2abccd
Allow negative compression levels in training
...
* Set `dictCLevel` in `zstdcli.c`.
* Only set to default level if the compression level `== 0`, not `<= 0`.
2018-04-09 12:12:03 -07:00
4195b36dd7
Merge pull request #1100 from bket/stable_sort
...
zstd requires a stable sort.
2018-04-05 11:39:27 -07:00
f35b8ba9da
updated ZSTD_p_chainLog description
2018-04-05 11:05:11 -07:00
462aed6811
zstd requires a stable sort.
...
On OpenBSD qsort() is not guaranteed to be stable, their mergesort() is.
This fixes issue #1088 . All the hard work has been done by @terrelln.
2018-04-05 07:59:16 +02:00
55f67502f4
Merge pull request #1098 from terrelln/nd-mt
...
Only load extra table positions for CDicts
2018-04-02 15:38:20 -07:00
295ab0dbfa
Only load extra table positions for CDicts
...
Zstdmt uses prefixes to load the overlap between segments. Loading extra
positions makes compression non-deterministic, depending on the previous
job the context was used for. Since loading extra position takes extra
time as well, only do it when creating a `ZSTD_CDict`.
Fixes #1077 .
2018-04-02 14:41:30 -07:00
5b616fa269
Merge pull request #1090 from bket/openbsd
...
Fix building zstd on OpenBSD.
2018-04-02 14:15:26 -07:00
9d3048346d
Fix building zstd on OpenBSD.
2018-03-31 10:46:20 +02:00
8be984ec45
fixed comments as suggested by @terrelln
2018-03-30 20:09:27 -07:00
e6e848bfe9
added ZSTD_getFrameHeader_advanced()
...
makes it possible to request frame header from a magicless frame
2018-03-29 17:51:08 -06:00
a6694838e1
added more code documentation for ZSTD_getFrameHeader()
2018-03-29 15:24:17 -06:00
21eb26d664
fixed legacy/zstd_v* with older gcc version, by guarding builtin_*
...
like in other files
2018-03-25 20:35:15 +02:00
ad15c1b724
added __has_attribute() define for non-clang compilers
2018-03-23 19:04:48 -07:00
52ca7c6c56
make DYNAMIC_BMI2 support of clang conditional to __has_attribute()
...
to support older clang versions such as 3.4
2018-03-23 18:45:42 -07:00
29b021f9a0
Merge pull request #1067 from facebook/targetLength
...
removed limit ZSTD_TARGETLENGTH_MAX
2018-03-22 10:38:33 -07:00
ad344033df
Fix broken assertion
...
The `avgJobSize` must not be lower than 256 KB for single-pass mode.
In `zstd.h` we say the minimum value for `ZSTD_p_jobSize` is 1 MB,
so ensure that we always pick a size >= 1 MB.
Found by libFuzzer fuzzer tests with large input limits.
2018-03-21 16:20:30 -07:00
153bc1c004
removed limit ZSTD_TARGETLENGTH_MAX
...
this makes it possible to specify extremely large negative compression levels,
achieving the side effect as "no compression".
It will also be possible to define larger targetlength for ultra compression mode.
There is no adverse side effect due to removing this limit.
2018-03-21 15:50:05 -07:00
a99c4a3621
Merge branch 'dev' into advancedDecompress
2018-03-21 06:08:28 -07:00
87b0cf05bd
Merge pull request #1057 from facebook/lrmSettings
...
LRM parameters
2018-03-21 05:59:39 -07:00
d1bf609abf
Merge pull request #1059 from terrelln/mt-ldm
...
Integrate ldm with zstdmt
2018-03-20 17:50:20 -07:00
e0cb8d19c6
fixed legacy test case
2018-03-20 17:48:22 -07:00
878728dc26
fixed several comments by @terrelln
2018-03-20 16:35:14 -07:00
e1c52faace
Merge pull request #1060 from facebook/compressImpl
...
merge bmi2 implementation of encodeSequence into zstd_compress.c
2018-03-20 16:19:42 -07:00
6cda8c932c
added test with ZSTD_decompress_generic() + ZSTD_DCtx_refPrefix()
...
also :
clarified stage condition to accept new parameters,
fixed initializers correspondingly.
2018-03-20 16:16:13 -07:00
0dadb6b70d
implemented ZSTD_DCtx_refPrefix*()
2018-03-20 15:45:56 -07:00
569b8ba4d9
implemented ZSTD_DCtx_refDDict()
2018-03-20 15:43:49 -07:00
a3b76a77ef
Quiet appveyor warnings
2018-03-20 15:34:40 -07:00
6873fec658
changed dictMore for dictContentType
...
which seems clearer to describe what the variable/argument is about.
2018-03-20 15:13:14 -07:00
31b54b6eea
updated ZSTD_initStaticDDict() prototype
...
can also specify dictContentType.
2018-03-20 14:52:02 -07:00
136b9e2392
Fix external sequence corner cases
...
* Clear external sequences when we reset the `ZSTD_CCtx`.
* Skip external sequences when a block is too small to compress.
2018-03-20 14:50:28 -07:00
353117c5d7
implemented ZSTD_DCtx_loadDictionary*()
...
this required updating ZSTD_createDDict_advanced()
to accept a dictContentType parameter (raw, full, auto).
2018-03-20 13:40:29 -07:00
451357f37f
Merge pull request #1058 from facebook/cctxParams
...
updated CCtxParams API
2018-03-20 12:36:12 -07:00
2ed5af0766
merge bmi2 implementation of encodeSequence into zstd_compress.c
2018-03-19 19:10:31 -07:00
d19f803a3b
Fix window size for 1 worker + flushing
2018-03-19 18:56:39 -07:00
24d9edbdd8
Set ldmParams to 0 when disabled
2018-03-19 18:23:54 -07:00
4b92574feb
Fix corner cases exposed by zstreamtest
2018-03-19 17:54:04 -07:00
Yann Collet
Peter Seiderer
Nick Terrell
taigacon
Nick Terrell
Yann Collet
Björn Ketelaars
René Rebe