Commit Graph

349 Commits (adbe74a8ac099d9f97929478b9ba45c9fa2cdc27)

Author SHA1 Message Date
Yann Collet dde10b23fe refactored ZSTD_estimateDStreamSize()
now uses windowSize as argument.
Also : created ZSTD_estimateDStreamSize_fromFrame()
2017-06-26 17:44:26 -07:00
Yann Collet 09ae03a570 ZSTD_estimateCDictSize_advanced()
ZSTD_estimateCDictSize() now uses same arguments as ZSTD_createCDict()
ZSTD_estimateCDictSize_advanced() uses same arguments as ZSTD_createCDict_advanced()
2017-06-26 16:47:32 -07:00
Yann Collet 0c9a915a28 ZSTD_estimateCStreamSize_advanced() 2017-06-26 16:02:25 -07:00
Yann Collet 31af8290d1 ZSTD_estimateCCtx_advanced()
ZSTD_estimateCCtx() is now a "simple" function,
taking int compressionLevel as single argument.

ZSTD_estimateCCtx_advanced() takes a CParams argument,
which is both more complete and more complex to generate.
2017-06-26 15:52:39 -07:00
Yann Collet ef269c1b68 Merge pull request #725 from facebook/advancedAPI2
New Advanced API
2017-06-23 09:50:47 -07:00
Yann Collet cca1ed14db Merge branch 'advancedAPI2' of github.com:facebook/zstd into advancedAPI2 2017-06-23 00:13:03 -07:00
Yann Collet 32f546c763 zstreamtest : disabled multi-threading tests with --newapi --no-big-tests
--no-big-tests is typically used in combination with qemu-user-static
qemu-user-static allocated 4 GB of RAM upfront.
On 2 GB VM, this can degenerate into a crash.
It's not a problem as long as memory is not used.
But with multi-threading enabled, memory fragmentation kicks in,
so the amoung of RAM effectively touched increases,
and can pass beyond the 2 GB limit of the VM.

In single-threaded mode, there is no such issue :
memory requirement is smaller, and remains well-located,
so very little fragmentation is expected.

This modification should make `qemu-arm-static zstreamtests --newapi --no-big-tests`
work fine on Travis CI.
2017-06-23 00:09:02 -07:00
Yann Collet f99c2c1a22 reduced --no-big-tests even more
to pass tests on qemu-aarch64
2017-06-21 23:35:58 -07:00
Yann Collet 49f8459d11 fixed minor cast warning 2017-06-21 18:43:39 -07:00
Yann Collet 4d3bdcf130 reduced CLevelMax for --no-big-tests 2017-06-21 18:20:02 -07:00
Yann Collet bfc2f00080 --no-big-tests for zstreamtest
Apply --no-big-tests for tsan tests
2017-06-21 17:57:14 -07:00
Yann Collet dce789281b fixed : decompression of skippable frames in streaming mode 2017-06-21 15:53:42 -07:00
Yann Collet c16748b137 added test cases for new command --list 2017-06-21 12:09:53 -07:00
Yann Collet 7bd1a2900e added ZSTD_dictMode_e to control dictionary loading mode 2017-06-21 11:50:33 -07:00
Paul Cruz db3606e249 added test for files with no frame content size 2017-06-20 17:43:36 -07:00
Paul Cruz a73c2a444a added tests with null files, changed condition to check that the file is more than 0 bytes 2017-06-20 14:33:08 -07:00
Paul Cruz 58c19b4202 spacing matters for test scripts 2017-06-20 14:14:53 -07:00
Yann Collet ff8f83bd47 fixed fuzzer test 2017-06-20 12:17:32 -07:00
Paul Cruz b07d0af2ba added additional tests for error detection 2017-06-20 11:54:44 -07:00
Paul Cruz 6a99de2dd8 change test to list info on .zst files 2017-06-20 10:45:06 -07:00
Yann Collet 688952062b minor declaration statement warning fix 2017-06-19 11:53:42 -07:00
Yann Collet d7a3bffba9 new api : setting compression parameters is refused if a dictionary is already loaded 2017-06-19 11:53:01 -07:00
Yann Collet ed1d039127 newapi fuzzer tests : random flush orders in main loop 2017-06-19 11:07:33 -07:00
Yann Collet f5deae8a67 new api fuzzer tests : frame parameters are randonly set 2017-06-18 23:41:38 -07:00
Yann Collet ea3630a889 new api fuzzer set : dictionary is randomly set 2017-06-18 23:31:55 -07:00
Yann Collet 8dee0ec99e new api fuzzer tests : compression parameters are randomly set 2017-06-18 23:25:15 -07:00
Yann Collet bd05b19713 removed macro CLAMP from decodecorpus
never used,
and would duplicate with CLAMP from zstd_compress.c
2017-06-16 18:16:27 -07:00
Yann Collet 01743a36e7 fuzzer tests for new API 2017-06-16 17:56:41 -07:00
Paul Cruz ddd1ab710c added tests in playTest.sh 2017-06-15 16:53:03 -07:00
Paul Cruz 37e1b1488b removed previous double free crash, used new crash function 2017-06-09 13:57:18 -07:00
Paul Cruz be9b0ae627 added error function for different compilation modes 2017-06-09 13:41:34 -07:00
Yann Collet 8c910d2097 updated ZSTDMT streaming API
ZSTDMT streaming API is now similar
and has same capabilites as single-thread streaming API.
It makes it easier to blend them together.
2017-06-03 01:15:02 -07:00
Yann Collet cdf7e82222 Added ZSTD_initStaticCDict() 2017-05-25 18:05:49 -07:00
Yann Collet 57827f906f added ZSTD_initStaticDDict() 2017-05-25 15:44:06 -07:00
Yann Collet 25989e361c updated ZSTD_estimate?DictSize() to pass parameter byReference
resulting ?Dict object is smaller when created byReference.
Seems better than a documentation note.
2017-05-25 15:07:37 -07:00
Yann Collet 0fdc71c3dc added ZSTD_initStaticDCtx() 2017-05-24 17:41:41 -07:00
Yann Collet c7fe262dc9 added ZSTD_initStaticCCtx()
makes it possible to statically or externally allocate CCtx.
static CCtx will only use provided memory area,
it will never resize nor malloc.
2017-05-23 13:20:41 -07:00
Yann Collet 60a557e7fd fixing symbol.c test
I believe it would be better to rely on fuzzer tests compiled with dll
2017-05-19 11:27:43 -07:00
Yann Collet fa3671eac7 changed ZSTD_BLOCKSIZE_ABSOLUTEMAX into ZSTD_BLOCKSIZE_MAX
Also :
change ZSTD_getBlockSizeMax() into ZSTD_getBlockSize()
created ZSTD_BLOCKSIZELOG_MAX
2017-05-19 10:51:30 -07:00
Yann Collet 009d604e00 ZSTD_compress_generic() supports multiple successive frames
also : clarified streaming API implementation
2017-05-19 10:17:59 -07:00
Yann Collet 0bdb575c31 Merge branch 'dev' into advancedAPI2 2017-05-16 16:32:29 -07:00
Yann Collet 2d4d31c18a removed gcc compilation flag -Wbad-function-cast
It makes it more difficult to directly cast the result of a function,
requiring to store the result in an intermediate variable.
It does not necessarily help readability,
and this restriction can be difficult to overcome in some constructions,
like some macros.

also : fixed minor Visual conversion warnings in datagencli.c
2017-05-16 11:34:38 -07:00
Yann Collet 83d0c764dc added several compilation flags 2017-05-15 17:15:46 -07:00
Yann Collet a5ffe3d370 pushed enum values for strategy by one (ZSTD_fast==1)
this makes it possible to use `0` to mean:
"do not change strategy"
2017-05-12 16:29:19 -07:00
Nick Terrell 5f2c7213c7 Merge remote-tracking branch 'upstream/dev' into btopt
* upstream/dev: (305 commits)
  added test for ZSTD_estimateCStreamSize()
  changed variable name, for clarity
  fixed ZSTD_estimateCStreamSize()
  shortened ZSTD_createCStream_Advanced()
  fixed symbols test
  added ZSTD_estimateDStreamSize()
  changed name frameParams into frameHeader
  regroup memory usage function declarations
  separated ZSTD_estimateCStreamSize() from ZSTD_estimateCCtxSize()
  bumped version number
  added ZSTD_estimateCDictSize() and ZSTD_estimateDDictSize()
  Updated ZSTD_freeCCtx()
  updated ZSTD_estimateCCtxSize()
  Updated ZSTD_sizeof_CCtx()
  merged CCtx and CStream as a single same object
  cli : -d and -t do not stop after a failed decompression
  added dev branch CircleCI badge
  added dev branch Appveyor badge
  keep dev branch status only
  creates a binary archive without the `programs` directory
  ...
2017-05-10 16:49:58 -07:00
Yann Collet 30ab64e21d added test for ZSTD_estimateCStreamSize() 2017-05-10 11:30:19 -07:00
Yann Collet 461a9cc7c6 fixed symbols test 2017-05-09 16:53:09 -07:00
Yann Collet f16f4497ca added ZSTD_estimateDStreamSize() 2017-05-09 16:18:17 -07:00
Yann Collet 542c9dfcf8 changed name frameParams into frameHeader
ZSTD_frameParams => ZSTD_frameHeader
ZSTD_getFrameParams() -> ZSTD_getFrameHeader()

The new naming is more distinctive from ZSTD_frameParameters,
which is used during compression.

ZSTD_frameHeader is clearer in its intention to described frame header content.
It also implies we are decoding a ZSTD frame, hence we are at decoding stage.
2017-05-09 15:46:07 -07:00
Yann Collet fa8dadb294 separated ZSTD_estimateCStreamSize() from ZSTD_estimateCCtxSize()
for clarity
2017-05-08 18:24:16 -07:00