Commit Graph

289 Commits (22ff60b3aae01b3d8a51fbda27f331e26d4e243e)

Author SHA1 Message Date
Stella Lau 5bc2c1e982 Add prototype support for customMem with cctxParams 2017-08-23 12:03:30 -07:00
Stella Lau 6f1a21c7e9 Remove formatting-only changes 2017-08-23 10:24:19 -07:00
Stella Lau 5b956f4753 Comment out CCtx_param versions of CDict functions 2017-08-21 14:49:16 -07:00
Stella Lau 1c0dbe81b1 Add documentation for CCtx_params 2017-08-21 13:18:00 -07:00
Stella Lau 939f954285 Pass ZSTD_CCtx_params as const ptr when possible 2017-08-21 12:57:18 -07:00
Stella Lau 73c73bf16a Reduce code duplication in zstreamtest 2017-08-21 12:41:19 -07:00
Stella Lau 91b30dbe84 Remove test parameter 2017-08-21 10:09:06 -07:00
Stella Lau f181f33bdf Disable tests and refactor 2017-08-21 01:59:08 -07:00
Stella Lau 023b24e6d4 Add cctx param tests 2017-08-20 22:55:07 -07:00
Yann Collet 32fb407c9d updated a bunch of headers
for the new license
2017-08-18 16:52:05 -07:00
Stella Lau 2300c58a6f Move dictContentByRef to cctx params 2017-08-18 12:03:16 -07:00
Stella Lau b6cb2ed8cb Move dictMode to cctxParams 2017-08-18 11:43:31 -07:00
Stella Lau 97e27affcb Move compression level to cctx params 2017-08-18 11:20:08 -07:00
Stella Lau c0221124d5 Add function to set opaque parameters 2017-08-17 19:30:22 -07:00
Stella Lau 4169f49171 Add initialization/allocation functions for opaque params 2017-08-17 18:45:04 -07:00
Stella Lau ade95b8bed Add opaque interfaces for static initialization 2017-08-17 18:13:08 -07:00
Stella Lau 699f11b4f7 Create opaque parameter structure 2017-08-17 17:33:46 -07:00
Yann Collet e1222544be Merge pull request #753 from paulcruz74/adapt-approach-3
adaptive compression v1
2017-07-27 10:00:10 -07:00
Paul Cruz 6945b3c43d removed previous version of completion for compression 2017-07-19 11:51:50 -07:00
Paul Cruz 50ce4eaeb6 added error detection for pthread initialization, added compression completion measurement, fixed const values 2017-07-17 10:12:44 -07:00
Yann Collet 4e77f7761d clarified comment on ZSTD_p_contentSizeFlag 2017-07-13 02:09:07 -07:00
Yann Collet 40156a4967 bumped version nb to v1.3.1 2017-07-08 04:55:09 -07:00
Yann Collet ead4dd48f6 new field frameHeader.headerSize 2017-07-07 15:51:24 -07:00
Yann Collet 990449b89d new field : ZSTD_frameHeader.frameType
Makes frame type (zstd,skippable) detection more straighforward.
ZSTD_getFrameHeader set frameContentSize=ZSTD_CONTENTSIZE_UNKNOWN to mean "field not present"
2017-07-07 15:21:35 -07:00
Yann Collet e622330a3b extended frameHeader.windowSize to unsigned long long 2017-07-07 14:19:01 -07:00
Yann Collet 97f2bf66da minor : fix typo 2017-06-29 11:31:40 -07:00
Yann Collet acbef3decd ZSTD_getFrameContentSize() is promoted to "stable" status 2017-06-29 05:19:51 -07:00
Yann Collet 2bf428df45 Merge branch 'advancedAPI2' into refPrefix 2017-06-28 16:35:49 -07:00
Yann Collet 813535105b added function to control monotonic memory budget increase of ZSTD_defaultCParameters[0]
It's a runtime test, based on assert(),
played once, on first ZSTD_getCParams() usage,
when ZSTD_DEBUG is enabled.
2017-06-28 15:34:56 -07:00
Yann Collet b7372933b8 implemented ZSTD_refPrefix() 2017-06-27 15:49:12 -07:00
Yann Collet 7d3816183f exposed ZSTD_MAGIC_DICTIONARY in zstd.h
makes it easier to explain ZSTD_dictMode
2017-06-27 13:50:34 -07:00
Yann Collet fecc721fd9 added parameter ZSTD_p_refDictContent 2017-06-27 11:46:39 -07:00
Yann Collet dde10b23fe refactored ZSTD_estimateDStreamSize()
now uses windowSize as argument.
Also : created ZSTD_estimateDStreamSize_fromFrame()
2017-06-26 17:44:26 -07:00
Yann Collet 09ae03a570 ZSTD_estimateCDictSize_advanced()
ZSTD_estimateCDictSize() now uses same arguments as ZSTD_createCDict()
ZSTD_estimateCDictSize_advanced() uses same arguments as ZSTD_createCDict_advanced()
2017-06-26 16:47:32 -07:00
Yann Collet 0c9a915a28 ZSTD_estimateCStreamSize_advanced() 2017-06-26 16:02:25 -07:00
Yann Collet 31af8290d1 ZSTD_estimateCCtx_advanced()
ZSTD_estimateCCtx() is now a "simple" function,
taking int compressionLevel as single argument.

ZSTD_estimateCCtx_advanced() takes a CParams argument,
which is both more complete and more complex to generate.
2017-06-26 15:52:39 -07:00
Yann Collet ef269c1b68 Merge pull request #725 from facebook/advancedAPI2
New Advanced API
2017-06-23 09:50:47 -07:00
Yann Collet 7bd1a2900e added ZSTD_dictMode_e to control dictionary loading mode 2017-06-21 11:50:33 -07:00
Yann Collet 78b8234554 fixed comments, following suggestion by @terrelln 2017-06-20 14:26:48 -07:00
Yann Collet fe234bf48b fix attempts : fullbench for VS2008 2017-06-19 15:23:19 -07:00
Yann Collet bf99150be3 update new api presentation in zstd.h and manual 2017-06-19 12:56:25 -07:00
Yann Collet d7a3bffba9 new api : setting compression parameters is refused if a dictionary is already loaded 2017-06-19 11:53:01 -07:00
Yann Collet 381e66cfbd added ZSTD_clampCParams()
now ZSTD_adjustCParams() is always successful,
it always produces a valid CParams
2017-06-16 17:34:54 -07:00
Yann Collet 559ee82e90 fixed : calling ZSTD_compress_generic() to end-flush a stream in multiple steps 2017-06-16 11:58:21 -07:00
Yann Collet bd18c885a3 added ZSTD_CCtx_reset 2017-06-16 10:17:50 -07:00
Yann Collet 05ae4b2190 added protection : MT incompatible with Static allocation 2017-06-15 18:03:34 -07:00
Paul Cruz a9b77c83e5 cleaning up code for analyzing frames 2017-06-15 14:13:28 -07:00
Yann Collet f35e2de61c linked newAPI to ZSTDMT 2017-06-05 18:32:48 -07:00
Yann Collet 8c910d2097 updated ZSTDMT streaming API
ZSTDMT streaming API is now similar
and has same capabilites as single-thread streaming API.
It makes it easier to blend them together.
2017-06-03 01:15:02 -07:00
Yann Collet c35e535002 added support for multithreading parameters 2017-06-01 18:44:06 -07:00
Yann Collet deee6e523f expose ZSTD_compress_generic_simpleArgs()
which is a binding towards ZSTD_compress_generic()
using only integral types for arguments.
2017-05-30 17:42:00 -07:00
Yann Collet 44e45e8423 added ZSTDMT_createCCtx_advanced()
make it possible to use custom allocators
2017-05-30 16:12:06 -07:00
Yann Collet f45ca527a1 Merge branch 'advancedAPI2' of github.com:facebook/zstd into advancedAPI2 2017-05-30 10:02:03 -07:00
Yann Collet b8136f019a static dctx is incompatible with legacy support
documented, and runtime tested
2017-05-27 00:03:08 -07:00
Yann Collet 7028cbd7fd fixed a few code comments : ZSTD_getFrameParams => ZSTD_getFrameHeader 2017-05-25 18:29:08 -07:00
Yann Collet cdf7e82222 Added ZSTD_initStaticCDict() 2017-05-25 18:05:49 -07:00
Yann Collet 57827f906f added ZSTD_initStaticDDict() 2017-05-25 15:44:06 -07:00
Yann Collet 25989e361c updated ZSTD_estimate?DictSize() to pass parameter byReference
resulting ?Dict object is smaller when created byReference.
Seems better than a documentation note.
2017-05-25 15:07:37 -07:00
Yann Collet 0fdc71c3dc added ZSTD_initStaticDCtx() 2017-05-24 17:41:41 -07:00
Yann Collet 11ea2f7fda Merged ZSTD_DCtx and ZSTD_DStream objects
They are now the same object.
It's recommended to keep both types in source code
as previous versions of library (<v1.3.0)
still need this differentiation.
2017-05-23 16:19:43 -07:00
Yann Collet c7fe262dc9 added ZSTD_initStaticCCtx()
makes it possible to statically or externally allocate CCtx.
static CCtx will only use provided memory area,
it will never resize nor malloc.
2017-05-23 13:20:41 -07:00
Yann Collet 24de7b0346 Implemented ZSTD_CCtx_refCDict() 2017-05-22 13:05:45 -07:00
Yann Collet ee970398b2 Merge branch 'dev' into advancedAPI2 2017-05-22 12:33:56 -07:00
Nick Terrell a1280406b0 [libzstd] Allow users to define custom visibility 2017-05-19 18:01:59 -07:00
Yann Collet fa3671eac7 changed ZSTD_BLOCKSIZE_ABSOLUTEMAX into ZSTD_BLOCKSIZE_MAX
Also :
change ZSTD_getBlockSizeMax() into ZSTD_getBlockSize()
created ZSTD_BLOCKSIZELOG_MAX
2017-05-19 10:51:30 -07:00
Yann Collet 009d604e00 ZSTD_compress_generic() supports multiple successive frames
also : clarified streaming API implementation
2017-05-19 10:17:59 -07:00
Yann Collet 6d4fef36de Added ZSTD_compress_generic()
Used in fileio.c (zstd cli).
Need to set macro ZSTD_NEWAPI to trigger it.
2017-05-17 18:36:15 -07:00
Yann Collet 23c256e44b removed useless variable from CCtx
CStream's pledgedSrcSize is no longer necessary
srcSize control is realized within bufferless interface.
2017-05-16 18:10:11 -07:00
Yann Collet 9f95e445ab minor comment clarifications 2017-05-16 17:26:43 -07:00
Yann Collet 0bdb575c31 Merge branch 'dev' into advancedAPI2 2017-05-16 16:32:29 -07:00
Yann Collet bfff8999c5 added prototype ZSTD_versionString() 2017-05-16 16:12:23 -07:00
Yann Collet 4eff8136aa added prototype ZSTD_decompressBegin_usingDDict (#700) 2017-05-16 16:05:27 -07:00
Yann Collet a5ffe3d370 pushed enum values for strategy by one (ZSTD_fast==1)
this makes it possible to use `0` to mean:
"do not change strategy"
2017-05-12 16:29:19 -07:00
Yann Collet add66f816d changed macro LOADCPARAMS by static function ZSTD_cLevelToCParams()
for improved compiler checks.
Also : ensure most parameters can receive value "0"
to mean "do not change".
2017-05-12 16:01:15 -07:00
Yann Collet b0edb7fb0e added ZSTD_CCtx_setParameter() 2017-05-12 15:31:53 -07:00
Yann Collet db8e21d5a0 made ZSTD_compress_generic() definition accessible
note that the implementation is not done yet.
2017-05-12 13:46:49 -07:00
Yann Collet 33eb7ac6b6 updated Advanced API proposal
only declarations in zstd.h
2017-05-12 12:36:11 -07:00
Nick Terrell 5f2c7213c7 Merge remote-tracking branch 'upstream/dev' into btopt
* upstream/dev: (305 commits)
  added test for ZSTD_estimateCStreamSize()
  changed variable name, for clarity
  fixed ZSTD_estimateCStreamSize()
  shortened ZSTD_createCStream_Advanced()
  fixed symbols test
  added ZSTD_estimateDStreamSize()
  changed name frameParams into frameHeader
  regroup memory usage function declarations
  separated ZSTD_estimateCStreamSize() from ZSTD_estimateCCtxSize()
  bumped version number
  added ZSTD_estimateCDictSize() and ZSTD_estimateDDictSize()
  Updated ZSTD_freeCCtx()
  updated ZSTD_estimateCCtxSize()
  Updated ZSTD_sizeof_CCtx()
  merged CCtx and CStream as a single same object
  cli : -d and -t do not stop after a failed decompression
  added dev branch CircleCI badge
  added dev branch Appveyor badge
  keep dev branch status only
  creates a binary archive without the `programs` directory
  ...
2017-05-10 16:49:58 -07:00
Yann Collet cef02d9317 changed variable name, for clarity
fhiPtr -> zfhPtr
https://github.com/facebook/zstd/pull/689#discussion_r115638676
2017-05-10 11:14:08 -07:00
Yann Collet f16f4497ca added ZSTD_estimateDStreamSize() 2017-05-09 16:18:17 -07:00
Yann Collet 542c9dfcf8 changed name frameParams into frameHeader
ZSTD_frameParams => ZSTD_frameHeader
ZSTD_getFrameParams() -> ZSTD_getFrameHeader()

The new naming is more distinctive from ZSTD_frameParameters,
which is used during compression.

ZSTD_frameHeader is clearer in its intention to described frame header content.
It also implies we are decoding a ZSTD frame, hence we are at decoding stage.
2017-05-09 15:46:07 -07:00
Yann Collet 5a36c069e7 regroup memory usage function declarations
in a single paragraph in zstd.h, for clarity
2017-05-09 15:11:30 -07:00
Yann Collet fa8dadb294 separated ZSTD_estimateCStreamSize() from ZSTD_estimateCCtxSize()
for clarity
2017-05-08 18:24:16 -07:00
Yann Collet 51652522a2 bumped version number 2017-05-08 17:52:46 -07:00
Yann Collet a1d6704d7f added ZSTD_estimateCDictSize() and ZSTD_estimateDDictSize()
it complements ZSTD_estimateCCtxSize()
for the special case of ZSTD_initCStream_usingDict()
2017-05-08 17:51:49 -07:00
Yann Collet fc5145955a updated ZSTD_estimateCCtxSize()
added a parameter streaming,
to estimate memory allocation size
when the CCtx is used for streaming (CStream).

Note : this function is not able to estimate
memory cost of a potential internal CDict
which can only happen when starting with ZSTD_initCStream_usingDict()
2017-05-08 17:07:59 -07:00
Yann Collet 791d744279 Updated ZSTD_sizeof_CCtx()
can now contain buffers if object used as CStream.
ZSTD_sizeof_CStream() is now just a thin wrapper of ZSTD_sizeof_CCtx().
2017-05-08 16:17:30 -07:00
Yann Collet 0be6fd3429 merged CCtx and CStream as a single same object
To be changed : ZSTD_sizeof_CCtx(), ZSTD_estimateCCtxSize()
2017-05-08 16:08:01 -07:00
Yann Collet a51cab6e68 Merge pull request #678 from facebook/apiChange
Breaking API Change around CDict
2017-04-28 10:02:45 -07:00
Yann Collet a92cbb7004 Added a secondary test, checking dictID presence after setting noDictIdFLag=1 2017-04-27 15:08:56 -07:00
Yann Collet 8b669535f8 bumped version number to v1.2.0 2017-04-27 12:50:20 -07:00
Yann Collet 77bf59ef50 added ZSTD_initCStream_usingCDict_advanced() 2017-04-27 11:43:04 -07:00
Yann Collet f4bd857d81 created ZSTD_compress_usingCDict_advanced() 2017-04-27 11:31:55 -07:00
Yann Collet 31533bacce Changed ZSTD_createCDict_advanced()
It now only uses compressionParameters as argument.
It produces many changes throughout user code,
though hopefully they tend to be simple :
just provide the cParams part from existing ZSTD_parameters.

Some programs might depend on ZSTD_createCDict_advanced() to pass frame parameters.
This change will force them to revisit this strategy and fix it,
since frame parameters are effectively silently ignored in current version.
2017-04-27 00:29:04 -07:00
Yann Collet 768df129d2 changed ZSTD_compressBegin_usingCDict()
No longer takes `pledgedSrcSize` as argument
this is in line with similar functions ZSTD_compress_usingCDict()
and ZSTD_initCStream_usingCDict().
2017-04-26 15:42:10 -07:00
Yann Collet 71ddeb67b1 made room in workspace for FSE tables
still need to be transferred from CCtx into workspace
2017-04-20 22:54:54 -07:00
Yann Collet 715b9aa113 created ZSTD_compressBegin_usingCDict_advanced() 2017-04-18 13:55:53 -07:00
Yann Collet af4f45b682 Improved code comments for block functions 2017-04-18 03:17:44 -07:00
Yann Collet 4f818182b8 clarified frame parameters for ZSTD_compress*_usingCDict()
created ZSTD_compressBegin_usingCDict_internal(),
which gives direct control to frame Parameters.
ZSTD_resetCStream_internal() now points into it.
2017-04-17 18:29:06 -07:00
Yann Collet 36c2a03757 updated comments for ZSTD_resetCStream() 2017-04-05 22:06:21 -07:00