facebook/zstd
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

stronger tests for zbuff decompression

Browse Source
This commit is contained in:
Yann Collet 2015-11-27 14:30:23 +01:00
parent 4a7bb12320
commit 800fa6c378
2 changed files with 14 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
lib/zstd_static.h
View File

@ -126,17 +126,17 @@ size_t ZSTD_decompressContinue(ZSTD_DCtx* dctx, void* dst, size_t maxDstSize, co
First operation is to retrieve frame parameters, using ZSTD_getFrameParams(). First operation is to retrieve frame parameters, using ZSTD_getFrameParams().
This function doesn't consume its input. It needs enough input data to properly decode the frame header. This function doesn't consume its input. It needs enough input data to properly decode the frame header.
The objective is to retrieve *params.windowlog, to know how much memory is required during decoding. The objective is to retrieve *params.windowlog, to know minimum amount of memory required during decoding.
Result : 0 if successfull, it means the ZSTD_parameters structure has been filled. Result : 0 when successful, it means the ZSTD_parameters structure has been filled.
>0 : means there is not enough data into src. Provides the expected size to successfully decode header. >0 : means there is not enough data into src. Provides the expected size to successfully decode header.
errorCode, which can be tested using ZSTD_isError() (For example, if it's not a ZSTD header) errorCode, which can be tested using ZSTD_isError() (For example, if it's not a ZSTD header)
Then it's possible to start decompression. Then it's possible to start decompression.
Use ZSTD_nextSrcSizeToDecompress() and ZSTD_decompressContinue() alternatively. Use ZSTD_nextSrcSizeToDecompress() and ZSTD_decompressContinue() alternatively.
ZSTD_nextSrcSizeToDecompress() tells how much bytes to provide as 'srcSize' to ZSTD_decompressContinue(). ZSTD_nextSrcSizeToDecompress() tells how much bytes to provide as 'srcSize' to ZSTD_decompressContinue().
ZSTD_decompressContinue() will use previous data blocks during decompress. ZSTD_decompressContinue() requires this exact amount of bytes, or just fails.
They should be located contiguously prior to current block. Alternatively, a round buffer is possible. ZSTD_decompressContinue() needs previous data blocks during decompression, up to (1 << windowlog).
Just make sure that the combined of current and accessible past blocks is a minimum of (1 << windowlog). They should preferably be located contiguously, prior to current block. Alternatively, a round buffer is also possible.
@result of ZSTD_decompressContinue() is the number of bytes regenerated within 'dst'. @result of ZSTD_decompressContinue() is the number of bytes regenerated within 'dst'.
It can be zero, which is not an error; it just means ZSTD_decompressContinue() has decoded some header. It can be zero, which is not an error; it just means ZSTD_decompressContinue() has decoded some header.

12
programs/zbufftest.c
View File

@ -371,7 +371,10 @@ int fuzzerTests(U32 seed, U32 nbTests, unsigned startTest, double compressibilit
sampleSize = (size_t)1 << sampleSizeLog; sampleSize = (size_t)1 << sampleSizeLog;
sampleSize += FUZ_rand(&lseed) & (sampleSize-1); sampleSize += FUZ_rand(&lseed) & (sampleSize-1);
readSize = sampleSize; readSize = sampleSize;
genSize = dstBufferSize - totalGenSize; sampleSizeLog = FUZ_rand(&lseed) % maxSampleLog;
sampleSize = (size_t)1 << sampleSizeLog;
sampleSize += FUZ_rand(&lseed) & (sampleSize-1);
genSize = MIN(sampleSize, dstBufferSize - totalGenSize);
errorCode = ZBUFF_decompressContinue(zd, dstBuffer+totalGenSize, &genSize, cBuffer+totalCSize, &readSize); errorCode = ZBUFF_decompressContinue(zd, dstBuffer+totalGenSize, &genSize, cBuffer+totalCSize, &readSize);
CHECK (ZBUFF_isError(errorCode), "decompression error : %s", ZBUFF_getErrorName(errorCode)); CHECK (ZBUFF_isError(errorCode), "decompression error : %s", ZBUFF_getErrorName(errorCode));
totalGenSize += genSize; totalGenSize += genSize;
@ -405,13 +408,16 @@ int fuzzerTests(U32 seed, U32 nbTests, unsigned startTest, double compressibilit
ZBUFF_decompressInit(zd); ZBUFF_decompressInit(zd);
totalCSize = 0; totalCSize = 0;
totalGenSize = 0; totalGenSize = 0;
while (totalCSize < cSize) while ( (totalCSize < cSize) && (totalGenSize < dstBufferSize) )
{ {
sampleSizeLog = FUZ_rand(&lseed) % maxSampleLog; sampleSizeLog = FUZ_rand(&lseed) % maxSampleLog;
sampleSize = (size_t)1 << sampleSizeLog; sampleSize = (size_t)1 << sampleSizeLog;
sampleSize += FUZ_rand(&lseed) & (sampleSize-1); sampleSize += FUZ_rand(&lseed) & (sampleSize-1);
readSize = sampleSize; readSize = sampleSize;
genSize = dstBufferSize - totalGenSize; sampleSizeLog = FUZ_rand(&lseed) % maxSampleLog;
sampleSize = (size_t)1 << sampleSizeLog;
sampleSize += FUZ_rand(&lseed) & (sampleSize-1);
genSize = MIN(sampleSize, dstBufferSize - totalGenSize);
errorCode = ZBUFF_decompressContinue(zd, dstBuffer+totalGenSize, &genSize, cBuffer+totalCSize, &readSize); errorCode = ZBUFF_decompressContinue(zd, dstBuffer+totalGenSize, &genSize, cBuffer+totalCSize, &readSize);
if (ZBUFF_isError(errorCode)) break; /* error correctly detected */ if (ZBUFF_isError(errorCode)) break; /* error correctly detected */
totalGenSize += genSize; totalGenSize += genSize;