Commit Graph

178 Commits (8f27e8cf3de23b87fc3185145316c278f7315488)

Author SHA1 Message Date
Yann Collet 7269fe6cd3 minor code comment update 2018-09-14 16:06:35 -07:00
Yann Collet 0403148315
Merge pull request #1295 from felixhandte/hdr-intro-comment-negative-lvls
Proposed Update to Zstd.h Introduction Comment
2018-09-14 15:29:19 -07:00
Yann Collet 674dd21bd0 final parameter tuning 2018-09-12 17:25:34 -07:00
Yann Collet 55a8f84a2c spec clarification
following #1305 comments from @ulikunitz
2018-09-05 12:31:33 -07:00
W. Felix Handte 8e944ccade Regenerate Documentation 2018-08-22 17:07:34 -07:00
Nick Terrell c1a7defee1 Small fixes to zstd specification
Update to keep in sync with the RFC.
2018-07-10 15:07:36 -07:00
Yann Collet a55ffbb31b removed unused imaged 2018-06-27 16:03:46 -07:00
Yann Collet c0007893b4 added graph for cdict speed v1.3.5 improvements 2018-06-27 15:58:34 -07:00
Yann Collet 4489daec09 slightly adjusted default-distribution threshold
depending on strategy.
fast favors faster compression and decompression speeds.
2018-06-26 20:10:45 -07:00
Yann Collet d70c4a5074
Merge pull request #1201 from facebook/rfcUpdate
updated Zstandard frame format
2018-06-22 11:53:50 -07:00
Yann Collet c1e6347717 fixed minor typos, detected by @terrelln 2018-06-21 18:08:11 -07:00
Yann Collet 7639db939f updated Zstandard frame format
adding clarifications from IETF RFC DISCUSS.
2018-06-21 17:55:55 -07:00
Yann Collet 2d76defbfe grouped all histogram functions into hist.c
renamed functions with HIST_* prefix
2018-06-13 19:49:31 -04:00
Yann Collet a57b4df85f removed literalCompression directive
in this version, literal compression is always disabled for ZSTD_fast strategy.

Performance parity between ZSTD_compress_advanced() and ZSTD_compress_generic()
2018-06-07 15:24:12 -07:00
Yann Collet 24319975b6 bumped version number to v1.3.5 2018-06-06 15:51:55 -07:00
Yann Collet 830fd4468f better make -j all behavior
avoid concurrent compilation of libzstd
2018-06-06 12:47:16 -07:00
Yann Collet a4c9c4defe update Zstandard format specification
answering a few questions from IETF RFC Discuss stage.
2018-05-31 10:47:44 -07:00
Nick Terrell 73f4c890cd Clarify what happens when Number_of_Sequences == 0 2018-05-22 16:12:33 -07:00
Yann Collet 82ad249645 Clarifications of Zstandard format specification
from IETF RFC review
2018-04-30 12:36:55 -07:00
Yann Collet 12f60b8c98 clarified documentation related to refPrefix() 2018-04-25 10:17:06 -07:00
Yann Collet ace856a835 updated documentation of streaming compression api 2018-04-24 14:44:27 -07:00
Nick Terrell 6bb6e6178a Add zstdmt long range mode graphs 2018-03-23 14:06:05 -07:00
Yann Collet d6c6b62958 updated graph for dictionary compression 2018-03-22 18:44:56 -07:00
Yann Collet 184e1c8b8c updated benchmark figures 2018-03-22 18:22:10 -07:00
Yann Collet b7b80e1569 fixed paramgrill 2018-03-21 16:09:15 -07:00
Yann Collet a99c4a3621 Merge branch 'dev' into advancedDecompress 2018-03-21 06:08:28 -07:00
Yann Collet 6873fec658 changed dictMore for dictContentType
which seems clearer to describe what the variable/argument is about.
2018-03-20 15:13:14 -07:00
Yann Collet 31b54b6eea updated ZSTD_initStaticDDict() prototype
can also specify dictContentType.
2018-03-20 14:52:02 -07:00
Yann Collet 353117c5d7 implemented ZSTD_DCtx_loadDictionary*()
this required updating ZSTD_createDDict_advanced()
to accept a dictContentType parameter (raw, full, auto).
2018-03-20 13:40:29 -07:00
Yann Collet c8b3d389fd updated CCtxParams API
to respect naming convention :
ZSTD_CCtxParams_*()
2018-03-19 15:07:26 -07:00
Yann Collet 97816400ca added /contrib projects to `make all` 2018-03-15 16:40:14 -07:00
Yann Collet 2291b85a1e changed ZSTD_p_literalCompression into ZSTD_p_compressLiterals
prefer verb+object construction
2018-03-12 11:44:10 -07:00
Yann Collet a57d43d4d4 updated documentation of targetLength 2018-03-12 11:35:01 -07:00
Yann Collet 564cb1b640 update doc/README.md on provided tools to test 3rd party implementations 2018-02-27 17:37:05 -08:00
Yann Collet 9945e60ac4 Merge branch 'dev' into flexibleLevel 2018-02-10 11:54:49 -08:00
Yann Collet 209df52ba2 Changed nbThreads for nbWorkers
This makes it easier to explain that nbWorkers=0 --> single-threaded mode,
while nbWorkers=1 --> asynchronous mode (one mode thread on top of the "main" caller thread).
No need for an additional asynchronous mode flag.
nbWorkers>=2 works the same as nbThreads>=2 previously.
2018-02-01 19:29:30 -08:00
Yann Collet 823a28a1f4
Merge pull request #1000 from facebook/progressiveFlush
Progressive flush
2018-01-30 22:49:47 -08:00
Yann Collet b173b8d8da updated zstd api manual 2018-01-30 15:05:12 -08:00
Yann Collet 2cb0740b6b zstdmt: changed naming convention
to avoid confusion with blocks.

also:
- jobs are cut into chunks of 512KB now, to reduce nb of mutex calls.
- fix function declaration ZSTD_getBlockSizeMax()
- fix outdated comment
2018-01-30 14:43:36 -08:00
Yann Collet 9f8ed23b5b bumped version number to v1.3.4
also added a paragraph on using compression level with training mode
as this is a recurrent question (see for example #1004)
2018-01-27 22:23:26 -08:00
Yann Collet 8e128eaf05 zstdmt : refactor job members
grouped by sharing properties
2018-01-26 10:20:38 -08:00
Yann Collet 997e4d0ccd added POOL_tryAdd() 2018-01-18 14:39:51 -08:00
Yann Collet 394eec697b Introduce ZSTD_getFrameProgression()
Produces 3 statistics for ongoing frame compression :
- ingested
- consumed (effectively compressed)
- produced

Ingested can be larger than consumed due to buffering effect.

For the time being, this patch mostly fixes the % ratio issue,
since it computes consumed / produced,
instead of ingested / produced.

That being said, update is not "smooth",
because on a slow enough setting,
fileio spends most of its time waiting for a worker to complete its job.

This could be improved thanks to more granular flushing
i.e. start flushing before ongoing job is fully completed.
2018-01-17 16:39:02 -08:00
Shawn Landden 914d983879
fix unbounded range
I think you meant 8 MiB or smaller, instead of an unbounded (and illogical) range
2017-12-21 16:15:12 -08:00
Yann Collet 78de28239f minor readme formatting update 2017-12-15 13:26:39 -08:00
Yann Collet 2cff66b62f version bump to v1.3.3 2017-12-14 16:11:20 -08:00
Yann Collet bd88f633ac zstreamtest : in `-T#s`, s considered a suffix meaning "seconds"
avoid unintentionnally triggering `seedset`,
so that seed gets automatically determined when not set.
2017-11-27 12:15:23 -08:00
Yann Collet 4154aec679 fixed comment, as suggested by @terrelln 2017-11-21 10:26:17 -08:00
Yann Collet fccb46fbe0 minor spelling fixes 2017-11-18 11:28:00 -08:00
Yann Collet ca1a9ebac5 fixed zlib wrapper
it was invoking ZSTD_initCStream_advanced() with pledgedSrcSize==0 and contentSizeFlag=1
which means "empty"
while the intention was to mean "unknown".

The contentSizeFlag==1 is new, it is a consequence of setting this value to 1 by default.

The solution selected here is to pass ZSTD_CONTENTSIZE_UNKNOWN to mean "unknown".
So contentSizeFlag remains set (it wasn't in previous versions).
2017-10-18 11:22:23 -07:00