Commit Graph

355 Commits (0853f8604474d7c519f4e98fd338bd57c5812006)

Author SHA1 Message Date
Yann Collet 2dd76037be zstd cli can increase level when input is too slow 2018-08-09 15:51:30 -07:00
W. Felix Handte 01bb1c1016 Add CCtx Param Controlling Dict Attachment Behavior 2018-06-21 17:29:25 -04:00
Yann Collet b2632bcf6c
Merge pull request #1174 from duc0/document_default_level
Expose ZSTD_CLEVEL_DEFAULT and update documentation
2018-06-12 12:09:01 -07:00
Duc Ngo 869e2718f6 Line break 2018-06-11 10:02:15 -07:00
Duc Ngo e8ef725e13 Address comments 2018-06-11 10:01:35 -07:00
Duc Ngo e34c000e44 Expose ZSTD_CLEVEL_DEFAULT and update documentation 2018-06-08 11:33:44 -07:00
Yann Collet c2c47e24e0 support targetlen==0 with strategy==ZSTD_fast
to mean "normal compression",
targetlen >= 1 now means "disable huffman compression of literals"
2018-06-07 15:49:01 -07:00
Yann Collet a57b4df85f removed literalCompression directive
in this version, literal compression is always disabled for ZSTD_fast strategy.

Performance parity between ZSTD_compress_advanced() and ZSTD_compress_generic()
2018-06-07 15:24:12 -07:00
Yann Collet 24319975b6 bumped version number to v1.3.5 2018-06-06 15:51:55 -07:00
Yann Collet 9b979d0e33 minor : improved API code comment
Extend guarantee that ZSTD_getFrameContentSize() will delivering the decompressed size
to any single-pass compression function.

Answer #1156
2018-05-31 11:12:18 -07:00
Yann Collet 12f60b8c98 clarified documentation related to refPrefix() 2018-04-25 10:17:06 -07:00
Yann Collet ace856a835 updated documentation of streaming compression api 2018-04-24 14:44:27 -07:00
Nick Terrell e8c9dc5cea Fix documentation 2018-04-13 12:43:38 -07:00
Nick Terrell 9f76eebd17 Add ZSTD_CCtx_resetParameters() function
* Fix docs for `ZSTD_CCtx_reset()`.
* Add `ZSTD_CCtx_resetParameters()`.

Fixes #1094.
2018-04-12 16:54:07 -07:00
Nick Terrell 280a236e9e Add ZSTD_CCtx(Param)?_getParameter() function
Closes #1096.
2018-04-12 11:50:12 -07:00
Yann Collet 04212178b5 doc : clarified advanced API usage
sticky parameters only work with `ZSTD_compress_generic()`
2018-04-10 11:40:36 -07:00
Yann Collet ad5ba6cdcf updated comment on parameters that can be changed during compression 2018-04-09 17:39:07 -07:00
Yann Collet f35b8ba9da updated ZSTD_p_chainLog description 2018-04-05 11:05:11 -07:00
Yann Collet 8be984ec45 fixed comments as suggested by @terrelln 2018-03-30 20:09:27 -07:00
Yann Collet e6e848bfe9 added ZSTD_getFrameHeader_advanced()
makes it possible to request frame header from a magicless frame
2018-03-29 17:51:08 -06:00
Yann Collet a6694838e1 added more code documentation for ZSTD_getFrameHeader() 2018-03-29 15:24:17 -06:00
Yann Collet 153bc1c004 removed limit ZSTD_TARGETLENGTH_MAX
this makes it possible to specify extremely large negative compression levels,
achieving the side effect as "no compression".

It will also be possible to define larger targetlength for ultra compression mode.

There is no adverse side effect due to removing this limit.
2018-03-21 15:50:05 -07:00
Yann Collet a99c4a3621 Merge branch 'dev' into advancedDecompress 2018-03-21 06:08:28 -07:00
Yann Collet 87b0cf05bd
Merge pull request #1057 from facebook/lrmSettings
LRM parameters
2018-03-21 05:59:39 -07:00
Yann Collet 0dadb6b70d implemented ZSTD_DCtx_refPrefix*() 2018-03-20 15:45:56 -07:00
Yann Collet 569b8ba4d9 implemented ZSTD_DCtx_refDDict() 2018-03-20 15:43:49 -07:00
Yann Collet 6873fec658 changed dictMore for dictContentType
which seems clearer to describe what the variable/argument is about.
2018-03-20 15:13:14 -07:00
Yann Collet 31b54b6eea updated ZSTD_initStaticDDict() prototype
can also specify dictContentType.
2018-03-20 14:52:02 -07:00
Yann Collet 353117c5d7 implemented ZSTD_DCtx_loadDictionary*()
this required updating ZSTD_createDDict_advanced()
to accept a dictContentType parameter (raw, full, auto).
2018-03-20 13:40:29 -07:00
Yann Collet c8b3d389fd updated CCtxParams API
to respect naming convention :
ZSTD_CCtxParams_*()
2018-03-19 15:07:26 -07:00
Yann Collet 6f4d0778a5 make it possible to express compression parameters in any order 2018-03-19 14:41:23 -07:00
Yann Collet 9618c0c804 make it possible to specify LDM parameters in any order 2018-03-19 11:07:04 -07:00
Yann Collet cbc71e40f6 moving LRM parameters out of experimental section
into "normal" range,
start pinned at 160.
2018-03-15 17:22:40 -07:00
Yann Collet 50f763ec44 fixed several comments are underlined by @terrelln 2018-03-13 14:23:14 -07:00
Yann Collet 2291b85a1e changed ZSTD_p_literalCompression into ZSTD_p_compressLiterals
prefer verb+object construction
2018-03-12 11:44:10 -07:00
Yann Collet a57d43d4d4 updated documentation of targetLength 2018-03-12 11:35:01 -07:00
Yann Collet 6a9b41b731 create command --fast[=#]
access negative compression levels from command line
for both compression and benchmark modes.

also : ensure proper propagation of parameters
through ZSTD_compress_generic() interface.

added relevant cli tests.
2018-03-11 20:01:23 -07:00
Yann Collet a146ee04ae added negative compression levels
negative compression level trade compression ratio for more compression speed.
They turn off huffman compression of literals,
and use row 0 as baseline with a stepSize = -cLevel.

added associated test in fuzzer

also added : new advanced parameter ZSTD_p_literalCompression
2018-03-11 05:21:53 -07:00
Nick Terrell 6b88d592fd Reduce ZSTD_CHAINLOG_MAX to 29 in 32-bit mode 2018-02-26 13:30:24 -08:00
Yann Collet 90eca318a7 fileio: create dedicated function to generate zstd frames
like other formats
2018-02-02 14:24:56 -08:00
Yann Collet 209df52ba2 Changed nbThreads for nbWorkers
This makes it easier to explain that nbWorkers=0 --> single-threaded mode,
while nbWorkers=1 --> asynchronous mode (one mode thread on top of the "main" caller thread).
No need for an additional asynchronous mode flag.
nbWorkers>=2 works the same as nbThreads>=2 previously.
2018-02-01 19:29:30 -08:00
Yann Collet 4b6a94f0cc clarified comments on LDM parameters 2018-02-01 17:07:27 -08:00
Yann Collet 60fa90b6c0 zstdmt: added ability to change compression parameters during compression 2018-02-01 16:13:31 -08:00
Yann Collet 823a28a1f4
Merge pull request #1000 from facebook/progressiveFlush
Progressive flush
2018-01-30 22:49:47 -08:00
Yann Collet a2ba629971 fixed function declaration ZSTD_getBlockSize() 2018-01-30 15:03:39 -08:00
Yann Collet 2cb0740b6b zstdmt: changed naming convention
to avoid confusion with blocks.

also:
- jobs are cut into chunks of 512KB now, to reduce nb of mutex calls.
- fix function declaration ZSTD_getBlockSizeMax()
- fix outdated comment
2018-01-30 14:43:36 -08:00
Yann Collet 9f8ed23b5b bumped version number to v1.3.4
also added a paragraph on using compression level with training mode
as this is a recurrent question (see for example #1004)
2018-01-27 22:23:26 -08:00
Yann Collet 5f349b129c zstdmt : correctly set end of frame 2018-01-23 15:52:40 -08:00
Yann Collet c1cc57f270 zstdmt : fix end condition (ZSTD_e_end)
When ZSTD_e_end directive is provided,
the question is not only "are internal buffers completely flushed",
it is also "is current frame completed".

In some rare cases,
it was possible for internal buffers to be completely flushed,
triggering a @return == 0,
but frame was not completed as it needed a last null-size block to mark the end,
resulting in an unfinished frame.
2018-01-23 15:19:11 -08:00
Yann Collet d19dc1903c
Merge pull request #995 from facebook/progressiveMT
Progressive mt
2018-01-18 17:59:49 -08:00