facebook/zstd
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

added ZSTD_decompress_generic()

Browse Source 
same as ZSTD_decompressStream(),
just for a similar feeling as the compression side, which uses ZSTD_compress_generic()
This commit is contained in:
Yann Collet 2017-09-25 15:41:48 -07:00
parent b8d4a3887f
commit 6ee05a02b8
4 changed files with 53 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

33
doc/zstd_manual.html
View File

@ -27,8 +27,8 @@
<li><a href="#Chapter17">Buffer-less and synchronous inner streaming functions</a></li> <li><a href="#Chapter17">Buffer-less and synchronous inner streaming functions</a></li>
<li><a href="#Chapter18">Buffer-less streaming compression (synchronous mode)</a></li> <li><a href="#Chapter18">Buffer-less streaming compression (synchronous mode)</a></li>
<li><a href="#Chapter19">Buffer-less streaming decompression (synchronous mode)</a></li> <li><a href="#Chapter19">Buffer-less streaming decompression (synchronous mode)</a></li>
<li><a href="#Chapter20">=== New advanced API (experimental) ===</a></li> <li><a href="#Chapter20">New advanced API (experimental)</a></li>
<li><a href="#Chapter21">=== Block level API ===</a></li> <li><a href="#Chapter21">Block level API</a></li>
</ol> </ol>
<hr> <hr>
<a name="Chapter1"></a><h2>Introduction</h2><pre> <a name="Chapter1"></a><h2>Introduction</h2><pre>
@ -783,7 +783,7 @@ size_t ZSTD_decodingBufferSize_min(unsigned long long windowSize, unsigned long
</pre></b><BR> </pre></b><BR>
<pre><b>typedef enum { ZSTDnit_frameHeader, ZSTDnit_blockHeader, ZSTDnit_block, ZSTDnit_lastBlock, ZSTDnit_checksum, ZSTDnit_skippableFrame } ZSTD_nextInputType_e; <pre><b>typedef enum { ZSTDnit_frameHeader, ZSTDnit_blockHeader, ZSTDnit_block, ZSTDnit_lastBlock, ZSTDnit_checksum, ZSTDnit_skippableFrame } ZSTD_nextInputType_e;
</b></pre><BR> </b></pre><BR>
<a name="Chapter20"></a><h2>=== New advanced API (experimental) ===</h2><pre></pre> <a name="Chapter20"></a><h2>New advanced API (experimental)</h2><pre></pre>
<pre><b>typedef enum { <pre><b>typedef enum {
ZSTD_f_zstd1 = 0, </b>/* Normal zstd frame format, specified in zstd_compression_format.md (default) */<b> ZSTD_f_zstd1 = 0, </b>/* Normal zstd frame format, specified in zstd_compression_format.md (default) */<b>
@ -1070,9 +1070,9 @@ size_t ZSTD_CCtx_refPrefix_advanced(ZSTD_CCtx* cctx, const void* prefix, size_t
</p></pre><BR> </p></pre><BR>
<h3>Advanced parameters for decompression API</h3><pre></pre><b><pre></pre></b><BR> <h3>Advanced parameters for decompression API</h3><pre></pre><b><pre></pre></b><BR>
<pre><b>size_t ZSTD_DCtx_loadDictionary(ZSTD_DCtx* dctx, const void* dict, size_t dictSize); <pre><b>size_t ZSTD_DCtx_loadDictionary(ZSTD_DCtx* dctx, const void* dict, size_t dictSize); </b>/* not implemented */<b>
size_t ZSTD_DCtx_loadDictionary_byReference(ZSTD_DCtx* dctx, const void* dict, size_t dictSize); size_t ZSTD_DCtx_loadDictionary_byReference(ZSTD_DCtx* dctx, const void* dict, size_t dictSize); </b>/* not implemented */<b>
size_t ZSTD_DCtx_loadDictionary_advanced(ZSTD_DCtx* dctx, const void* dict, size_t dictSize, ZSTD_dictLoadMethod_e dictLoadMethod, ZSTD_dictMode_e dictMode); size_t ZSTD_DCtx_loadDictionary_advanced(ZSTD_DCtx* dctx, const void* dict, size_t dictSize, ZSTD_dictLoadMethod_e dictLoadMethod, ZSTD_dictMode_e dictMode); </b>/* not implemented */<b>
</b><p> Create an internal DDict from dict buffer, </b><p> Create an internal DDict from dict buffer,
to be used to decompress next frames. to be used to decompress next frames.
@result : 0, or an error code (which can be tested with ZSTD_isError()). @result : 0, or an error code (which can be tested with ZSTD_isError()).
@ -1089,7 +1089,7 @@ size_t ZSTD_DCtx_loadDictionary_advanced(ZSTD_DCtx* dctx, const void* dict, size
</p></pre><BR> </p></pre><BR>
<pre><b>size_t ZSTD_DCtx_refDDict(ZSTD_DCtx* dctx, const ZSTD_DDict* ddict); <pre><b>size_t ZSTD_DCtx_refDDict(ZSTD_DCtx* dctx, const ZSTD_DDict* ddict); </b>/* not implemented */<b>
</b><p> Reference a prepared dictionary, to be used to decompress next frames. </b><p> Reference a prepared dictionary, to be used to decompress next frames.
The dictionary remains active for decompression of future frames using same DCtx. The dictionary remains active for decompression of future frames using same DCtx.
@result : 0, or an error code (which can be tested with ZSTD_isError()). @result : 0, or an error code (which can be tested with ZSTD_isError()).
@ -1100,8 +1100,8 @@ size_t ZSTD_DCtx_loadDictionary_advanced(ZSTD_DCtx* dctx, const void* dict, size
</p></pre><BR> </p></pre><BR>
<pre><b>size_t ZSTD_DCtx_refPrefix(ZSTD_DCtx* dctx, const void* prefix, size_t prefixSize); <pre><b>size_t ZSTD_DCtx_refPrefix(ZSTD_DCtx* dctx, const void* prefix, size_t prefixSize); </b>/* not implemented */<b>
size_t ZSTD_DCtx_refPrefix_advanced(ZSTD_DCtx* dctx, const void* prefix, size_t prefixSize, ZSTD_dictMode_e dictMode); size_t ZSTD_DCtx_refPrefix_advanced(ZSTD_DCtx* dctx, const void* prefix, size_t prefixSize, ZSTD_dictMode_e dictMode); </b>/* not implemented */<b>
</b><p> Reference a prefix (single-usage dictionary) for next compression job. </b><p> Reference a prefix (single-usage dictionary) for next compression job.
Prefix is **only used once**. It must be explicitly referenced before each frame. Prefix is **only used once**. It must be explicitly referenced before each frame.
If there is a need to use same prefix multiple times, consider embedding it into a ZSTD_DDict instead. If there is a need to use same prefix multiple times, consider embedding it into a ZSTD_DDict instead.
@ -1110,7 +1110,7 @@ size_t ZSTD_DCtx_refPrefix_advanced(ZSTD_DCtx* dctx, const void* prefix, size_t
Note 2 : Prefix buffer is referenced. It must outlive compression job. Note 2 : Prefix buffer is referenced. It must outlive compression job.
Note 3 : By default, the prefix is treated as raw content (ZSTD_dm_rawContent). Note 3 : By default, the prefix is treated as raw content (ZSTD_dm_rawContent).
Use ZSTD_CCtx_refPrefix_advanced() to alter dictMode. Use ZSTD_CCtx_refPrefix_advanced() to alter dictMode.
Note 4 : Referencing a raw content prefix costs almost nothing cpu and memory wise. Note 4 : Referencing a raw content prefix has almost no cpu nor memory cost.
</p></pre><BR> </p></pre><BR>
@ -1131,7 +1131,18 @@ size_t ZSTD_DCtx_refPrefix_advanced(ZSTD_DCtx* dctx, const void* prefix, size_t
</p></pre><BR> </p></pre><BR>
<a name="Chapter21"></a><h2>=== Block level API ===</h2><pre></pre> <pre><b>size_t ZSTD_decompress_generic(ZSTD_DCtx* dctx,
ZSTD_outBuffer* output,
ZSTD_inBuffer* input);
</b><p> Behave the same as ZSTD_decompressStream.
Decompression parameters cannot be changed once decompression is started.
@return : an error code, which can be tested using ZSTD_isError()
if >0, a hint, nb of expected input bytes for next invocation.
`0` means : a frame has just been fully decoded and flushed.
</p></pre><BR>
<a name="Chapter21"></a><h2>Block level API</h2><pre></pre>
<pre><b></b><p> Frame metadata cost is typically ~18 bytes, which can be non-negligible for very small blocks (< 100 bytes). <pre><b></b><p> Frame metadata cost is typically ~18 bytes, which can be non-negligible for very small blocks (< 100 bytes).
User will have to take in charge required information to regenerate data, such as compressed and content sizes. User will have to take in charge required information to regenerate data, such as compressed and content sizes.

1
lib/compress/zstd_compress.c
View File

@ -2574,6 +2574,7 @@ MEM_STATIC size_t ZSTD_limitCopy(void* dst, size_t dstCapacity,
/** ZSTD_compressStream_generic(): /** ZSTD_compressStream_generic():
* internal function for all *compressStream*() variants and *compress_generic() * internal function for all *compressStream*() variants and *compress_generic()
* non-static, because can be called from zstdmt.c
* @return : hint size for next input */ * @return : hint size for next input */
size_t ZSTD_compressStream_generic(ZSTD_CStream* zcs, size_t ZSTD_compressStream_generic(ZSTD_CStream* zcs,
ZSTD_outBuffer* output, ZSTD_outBuffer* output,

9
lib/decompress/zstd_decompress.c
View File

@ -2379,7 +2379,10 @@ size_t ZSTD_decompressStream(ZSTD_DStream* zds, ZSTD_outBuffer* output, ZSTD_inB
U32 someMoreWork = 1; U32 someMoreWork = 1;
DEBUGLOG(5, "ZSTD_decompressStream"); DEBUGLOG(5, "ZSTD_decompressStream");
if (input->pos > input->size) return ERROR(GENERIC); /* forbidden */
if (output->pos > output->size) return ERROR(GENERIC); /* forbidden */
DEBUGLOG(5, "input size : %u", (U32)(input->size - input->pos)); DEBUGLOG(5, "input size : %u", (U32)(input->size - input->pos));
#if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT>=1) #if defined(ZSTD_LEGACY_SUPPORT) && (ZSTD_LEGACY_SUPPORT>=1)
if (zds->legacyVersion) { if (zds->legacyVersion) {
/* legacy support is incompatible with static dctx */ /* legacy support is incompatible with static dctx */
@ -2590,3 +2593,9 @@ size_t ZSTD_decompressStream(ZSTD_DStream* zds, ZSTD_outBuffer* output, ZSTD_inB
return nextSrcSizeHint; return nextSrcSizeHint;
} }
} }
size_t ZSTD_decompress_generic(ZSTD_DCtx* dctx, ZSTD_outBuffer* output, ZSTD_inBuffer* input)
{
return ZSTD_decompressStream(dctx, output, input);
}

32
lib/zstd.h
View File

@ -908,7 +908,9 @@ ZSTDLIB_API ZSTD_nextInputType_e ZSTD_nextInputType(ZSTD_DCtx* dctx);
/** === New advanced API (experimental) === **/ /* ============================================ */
/** New advanced API (experimental) */
/* ============================================ */
/* notes on API design : /* notes on API design :
* In this proposal, parameters are pushed one by one into an existing context, * In this proposal, parameters are pushed one by one into an existing context,
@ -1295,23 +1297,31 @@ ZSTDLIB_API size_t ZSTD_DCtx_setMaxWindowSize(ZSTD_DCtx* dctx, size_t maxWindowS
* such ZSTD_f_zstd1_magicless for example. * such ZSTD_f_zstd1_magicless for example.
* @return : 0, or an error code (which can be tested using ZSTD_isError()). * @return : 0, or an error code (which can be tested using ZSTD_isError()).
*/ */
ZSTDLIB_API size_t ZSTD_DCtx_setFormat(ZSTD_DCtx* dctx, ZSTD_format_e format); /* implemented, but not functional */ ZSTDLIB_API size_t ZSTD_DCtx_setFormat(ZSTD_DCtx* dctx, ZSTD_format_e format);
/* How to decompress ? /*! ZSTD_decompress_generic() :
* * Behave the same as ZSTD_decompressStream.
* currently, use ZSTD_decompressStream(). * Decompression parameters cannot be changed once decompression is started.
* We could also create a ZSTD_decompress_generic(), * @return : an error code, which can be tested using ZSTD_isError()
* for an API experience similar to the compression one. * if >0, a hint, nb of expected input bytes for next invocation.
* It would effectively works exactly the same as ZSTD_decompressStream(). * `0` means : a frame has just been fully decoded and flushed.
* */
ZSTDLIB_API size_t ZSTD_decompress_generic(ZSTD_DCtx* dctx,
ZSTD_outBuffer* output,
ZSTD_inBuffer* input);
/*
* Also : to re-init a decoding context, use ZSTD_initDStream(). * Also : to re-init a decoding context, use ZSTD_initDStream().
* Here also, for a similar API logic, we could create ZSTD_DCtx_reset(). * Here for a similar API logic, we could create ZSTD_DCtx_reset().
* It would behave the same. * It would behave the same.
*/ */
/** === Block level API === **/ /* ============================ */
/** Block level API */
/* ============================ */
/*! /*!
Block functions produce and decode raw zstd blocks, without frame metadata. Block functions produce and decode raw zstd blocks, without frame metadata.