768a264200
Merge branch 'dev' of github.com:facebook/zstd into dev
2018-11-13 15:56:36 -08:00
092c4abd4c
bumped version number to v1.3.8
2018-11-13 15:53:38 -08:00
fc20b3c441
added flag -Wc++-compat
...
for library and cli
2018-10-26 16:38:23 -07:00
9bf00707c7
minor clarifications of history update rules
2018-10-26 15:51:51 -07:00
f0fe9b0f02
Reverted removal of a trailing space.
...
My editor removes trailing spaces while saving. Not confusing things I
reverted that change.
2018-10-23 08:43:19 +02:00
4f702e4445
Fixed a typo
...
I fixed a typo in the last commit. Many thanks to @terrelin for pointing
that out.
2018-10-23 08:36:50 +02:00
c7942caff0
Clarify special case of offset history update
...
If the current sequence has literal length of zero then an offset value
of three is handled in a special manner. While I implemented a golang
decoder I had to consult the educational decoder for clarification on
the update of the offset history in that case. This commit provides the
clarification that the offset value Repeated_Offset1-1 is handled as a
new offset is added to the offset history accordingly.
2018-10-22 23:46:43 +02:00
1e6208e75e
bumped version number to v1.3.7
...
updated documentation
2018-10-11 14:40:12 -07:00
68bec4c5bb
added graph for ZSTD_compress_usingCDict() in v1.3.6
2018-10-04 14:39:11 -07:00
8883af6a1e
Merge pull request #1327 from facebook/adapt
...
Adaptive compression
2018-09-26 14:39:08 -07:00
72a3adf826
updated format documentation
...
to match last edits of RFC8478.
2018-09-25 16:34:26 -07:00
04f47bbdd2
Merge branch 'dev' into adapt
2018-09-24 16:56:45 -07:00
a54c86cfc6
defined a minimum negative level
...
which can be probed using new function ZSTD_minCLevel().
Also : redefined ZSTD_TARGETLENGTH_MIN/MAX for consistency
used the opportunity to bump version number to v1.3.6
2018-09-20 16:52:03 -07:00
2f78228f65
Merge branch 'dev' into adapt
2018-09-19 12:43:42 -07:00
005f000aed
updated documentation of *refPrefix()
...
indicating the equivalence with `diff` operation.
2018-09-18 13:07:08 -07:00
7269fe6cd3
minor code comment update
2018-09-14 16:06:35 -07:00
0403148315
Merge pull request #1295 from felixhandte/hdr-intro-comment-negative-lvls
...
Proposed Update to Zstd.h Introduction Comment
2018-09-14 15:29:19 -07:00
674dd21bd0
final parameter tuning
2018-09-12 17:25:34 -07:00
55a8f84a2c
spec clarification
...
following #1305 comments from @ulikunitz
2018-09-05 12:31:33 -07:00
8e944ccade
Regenerate Documentation
2018-08-22 17:07:34 -07:00
33f7709c71
fileio: changed parameter type from ptr to plain structure
...
safer : this parameter is read-only,
we don't want original structure to be modified
2018-08-13 13:02:03 -07:00
c1a7defee1
Small fixes to zstd specification
...
Update to keep in sync with the RFC.
2018-07-10 15:07:36 -07:00
a55ffbb31b
removed unused imaged
2018-06-27 16:03:46 -07:00
c0007893b4
added graph for cdict speed v1.3.5 improvements
2018-06-27 15:58:34 -07:00
4489daec09
slightly adjusted default-distribution threshold
...
depending on strategy.
fast favors faster compression and decompression speeds.
2018-06-26 20:10:45 -07:00
d70c4a5074
Merge pull request #1201 from facebook/rfcUpdate
...
updated Zstandard frame format
2018-06-22 11:53:50 -07:00
c1e6347717
fixed minor typos, detected by @terrelln
2018-06-21 18:08:11 -07:00
7639db939f
updated Zstandard frame format
...
adding clarifications from IETF RFC DISCUSS.
2018-06-21 17:55:55 -07:00
2d76defbfe
grouped all histogram functions into hist.c
...
renamed functions with HIST_* prefix
2018-06-13 19:49:31 -04:00
a57b4df85f
removed literalCompression directive
...
in this version, literal compression is always disabled for ZSTD_fast strategy.
Performance parity between ZSTD_compress_advanced() and ZSTD_compress_generic()
2018-06-07 15:24:12 -07:00
24319975b6
bumped version number to v1.3.5
2018-06-06 15:51:55 -07:00
830fd4468f
better make -j all behavior
...
avoid concurrent compilation of libzstd
2018-06-06 12:47:16 -07:00
a4c9c4defe
update Zstandard format specification
...
answering a few questions from IETF RFC Discuss stage.
2018-05-31 10:47:44 -07:00
73f4c890cd
Clarify what happens when Number_of_Sequences == 0
2018-05-22 16:12:33 -07:00
82ad249645
Clarifications of Zstandard format specification
...
from IETF RFC review
2018-04-30 12:36:55 -07:00
12f60b8c98
clarified documentation related to refPrefix()
2018-04-25 10:17:06 -07:00
ace856a835
updated documentation of streaming compression api
2018-04-24 14:44:27 -07:00
6bb6e6178a
Add zstdmt long range mode graphs
2018-03-23 14:06:05 -07:00
d6c6b62958
updated graph for dictionary compression
2018-03-22 18:44:56 -07:00
184e1c8b8c
updated benchmark figures
2018-03-22 18:22:10 -07:00
b7b80e1569
fixed paramgrill
2018-03-21 16:09:15 -07:00
a99c4a3621
Merge branch 'dev' into advancedDecompress
2018-03-21 06:08:28 -07:00
6873fec658
changed dictMore for dictContentType
...
which seems clearer to describe what the variable/argument is about.
2018-03-20 15:13:14 -07:00
31b54b6eea
updated ZSTD_initStaticDDict() prototype
...
can also specify dictContentType.
2018-03-20 14:52:02 -07:00
353117c5d7
implemented ZSTD_DCtx_loadDictionary*()
...
this required updating ZSTD_createDDict_advanced()
to accept a dictContentType parameter (raw, full, auto).
2018-03-20 13:40:29 -07:00
c8b3d389fd
updated CCtxParams API
...
to respect naming convention :
ZSTD_CCtxParams_*()
2018-03-19 15:07:26 -07:00
97816400ca
added /contrib projects to `make all`
2018-03-15 16:40:14 -07:00
2291b85a1e
changed ZSTD_p_literalCompression into ZSTD_p_compressLiterals
...
prefer verb+object construction
2018-03-12 11:44:10 -07:00
a57d43d4d4
updated documentation of targetLength
2018-03-12 11:35:01 -07:00
564cb1b640
update doc/README.md on provided tools to test 3rd party implementations
2018-02-27 17:37:05 -08:00
9945e60ac4
Merge branch 'dev' into flexibleLevel
2018-02-10 11:54:49 -08:00
209df52ba2
Changed nbThreads for nbWorkers
...
This makes it easier to explain that nbWorkers=0 --> single-threaded mode,
while nbWorkers=1 --> asynchronous mode (one mode thread on top of the "main" caller thread).
No need for an additional asynchronous mode flag.
nbWorkers>=2 works the same as nbThreads>=2 previously.
2018-02-01 19:29:30 -08:00
823a28a1f4
Merge pull request #1000 from facebook/progressiveFlush
...
Progressive flush
2018-01-30 22:49:47 -08:00
b173b8d8da
updated zstd api manual
2018-01-30 15:05:12 -08:00
2cb0740b6b
zstdmt: changed naming convention
...
to avoid confusion with blocks.
also:
- jobs are cut into chunks of 512KB now, to reduce nb of mutex calls.
- fix function declaration ZSTD_getBlockSizeMax()
- fix outdated comment
2018-01-30 14:43:36 -08:00
9f8ed23b5b
bumped version number to v1.3.4
...
also added a paragraph on using compression level with training mode
as this is a recurrent question (see for example #1004 )
2018-01-27 22:23:26 -08:00
8e128eaf05
zstdmt : refactor job members
...
grouped by sharing properties
2018-01-26 10:20:38 -08:00
997e4d0ccd
added POOL_tryAdd()
2018-01-18 14:39:51 -08:00
394eec697b
Introduce ZSTD_getFrameProgression()
...
Produces 3 statistics for ongoing frame compression :
- ingested
- consumed (effectively compressed)
- produced
Ingested can be larger than consumed due to buffering effect.
For the time being, this patch mostly fixes the % ratio issue,
since it computes consumed / produced,
instead of ingested / produced.
That being said, update is not "smooth",
because on a slow enough setting,
fileio spends most of its time waiting for a worker to complete its job.
This could be improved thanks to more granular flushing
i.e. start flushing before ongoing job is fully completed.
2018-01-17 16:39:02 -08:00
914d983879
fix unbounded range
...
I think you meant 8 MiB or smaller, instead of an unbounded (and illogical) range
2017-12-21 16:15:12 -08:00
78de28239f
minor readme formatting update
2017-12-15 13:26:39 -08:00
2cff66b62f
version bump to v1.3.3
2017-12-14 16:11:20 -08:00
bd88f633ac
zstreamtest : in `-T#s`, s considered a suffix meaning "seconds"
...
avoid unintentionnally triggering `seedset`,
so that seed gets automatically determined when not set.
2017-11-27 12:15:23 -08:00
4154aec679
fixed comment, as suggested by @terrelln
2017-11-21 10:26:17 -08:00
fccb46fbe0
minor spelling fixes
2017-11-18 11:28:00 -08:00
ca1a9ebac5
fixed zlib wrapper
...
it was invoking ZSTD_initCStream_advanced() with pledgedSrcSize==0 and contentSizeFlag=1
which means "empty"
while the intention was to mean "unknown".
The contentSizeFlag==1 is new, it is a consequence of setting this value to 1 by default.
The solution selected here is to pass ZSTD_CONTENTSIZE_UNKNOWN to mean "unknown".
So contentSizeFlag remains set (it wasn't in previous versions).
2017-10-18 11:22:23 -07:00
5eed8e7a55
changed API comments to invite using macro ZSTD_CONTENTSIZE_UNKNOWN
...
to mean "pledgedSrcSize is not known at init time" instead of `0`.
Note that, a few prototypes created and documented with `0` to mean "unknown" still interpret "0" as unknown,
to avoid breaking 3rd party applications which depend on this behavior.
But this value is no longer recommended to mean "unknown".
In some future version, it might be possible to switch "0" to mean "empty",
as is already the case for several prototypes.
The advantage is that pledgedSrcSize field would have same behavior accross entire API,
making it easier to reason about.
Note that all concerned prototypes belong to the "experimental" API section.
srcSize is controlled at end of compression,
so if someone uses "0" to mean "unknown" while it effectively means "empty",
this is immediately caught by the compression function, which generates an error code : ZSTD_ERROR_srcSize_wrong
2017-10-14 00:32:06 -07:00
6d8778d8e7
[doc] Add images for release notes
2017-10-09 16:44:31 -07:00
7f6a783862
fixed a small error in decodeCorpus
...
a compressed block must be strictly smaller than its decompressed size.
2017-10-07 15:19:52 -07:00
ecf1778e23
updated ZSTD_format_e value validation
...
also updated manual
2017-09-27 11:19:21 -07:00
52a1d1c6dc
added ZSTD_DCtx_reset()
2017-09-25 16:56:48 -07:00
f2a913862c
added ZSTD_decompress_generic_simpleArgs()
2017-09-25 15:46:34 -07:00
6ee05a02b8
added ZSTD_decompress_generic()
...
same as ZSTD_decompressStream(),
just for a similar feeling as the compression side, which uses ZSTD_compress_generic()
2017-09-25 15:41:48 -07:00
96f0cde31a
minor function rename
...
ZSTD_estimateCStreamSize_advanced_usingCParams -> ZSTD_estimateCStreamSize_usingCParams
_usingX is clear.
_advanced feels redundant
2017-09-24 16:47:02 -07:00
7c3dea42ce
added prototypes for advanced parameters for decompression API
...
required to decode custom formats
2017-09-24 15:57:29 -07:00
086b9597d9
added ability to split input files for dictionary training
...
using command -B#
This is the same behavior as benchmark module,
which can also split input into arbitrary size blocks, using -B#.
2017-09-14 16:45:10 -07:00
f1571dad8f
Merge pull request #838 from stellamplau/ldm-mergeDev
...
Add long distance matcher
2017-09-13 13:24:08 -07:00
ce31004f20
fix following suggestions by @terrelln
2017-09-11 13:12:52 -07:00
058ed2ad33
ZSTD_decodingBufferSize_min()
...
supporting function for bufferless streaming API (ZSTD_decompressContinue())
makes it possible to correctly size a round buffer for decoding using this API.
also : added field blockSizeMax within ZSTD_frameHeader,
as it's a necessary information to know when to restart at beginning of decoding buffer.
2017-09-09 01:03:29 -07:00
eeff55dfa8
Merge remote-tracking branch 'upstream/dev' into ldm-mergeDev
2017-09-06 15:56:32 -07:00
8c33cfe0bc
Add ldm documentation in README
2017-09-06 15:21:01 -07:00
0c314cde4b
updated zstd API manual
...
for new CCtxParams object
2017-08-31 18:28:19 -07:00
b0cb081dc8
last batch of header files changed to reflect new license ( #825 )
...
only remains to update contrib/linux-kernel (@terrelln)
2017-08-31 12:20:50 -07:00
e21384fffb
fixed more file headers after license change ( #825 )
2017-08-31 12:11:57 -07:00
e8d35cc5e9
minor formulation change, recommended by @ulikunitz
2017-08-20 10:39:20 -07:00
d0d06e421f
added alternative representation for huffman bistream
2017-08-19 12:26:09 -07:00
8b12812147
fix #803 : wrong example in huffman bitstream section, reported by @ulikunitz
2017-08-19 12:17:57 -07:00
bef5eda8d9
const vars, change copy_literals() to only take size_t literal_length
2017-08-16 11:11:52 -07:00
8d3f18af2c
renamed IO functions for clarity
2017-08-14 17:51:51 -07:00
6aebcfa0bc
added comments for new functions
2017-08-14 15:11:01 -07:00
9d56c21279
added separate function for executing match copy command
2017-08-14 15:06:03 -07:00
d3e57db0bd
created separate function for offset computation
2017-08-14 14:20:12 -07:00
b6d6be58c9
created separate function for copying literals during sequence execution
2017-08-14 14:05:16 -07:00
93c1309fd4
added free to free_dictionary()
2017-08-14 13:08:30 -07:00
38f4e43381
added error checking for dictionary initialized with null src
2017-08-14 09:41:04 -07:00
7ef9c6f4b2
made separate API for dictionary management
2017-08-11 18:40:19 -07:00
bfc6db8d6a
exposed dictionary functions/types
2017-08-11 17:53:37 -07:00
bd308d806b
remove debug symbols when cleaning, added a simple test
2017-08-11 14:42:15 -07:00
9f67e8652e
fixed warnings shown by compiler
2017-08-11 14:41:44 -07:00
d0dc675596
add makefile
2017-08-11 14:35:13 -07:00
Yann Collet
Ulrich Kunitz
Yann Collet
W. Felix Handte
Nick Terrell
Shawn Landden
Stella Lau
Paul Cruz