Commit Graph

3569 Commits (1fd2df9c2cfacb84fe7340842ea80b7e8a97cdc1)

Author SHA1 Message Date
Yann Collet 1fd2df9c2c fixed zstreamtest
modifying compression parameters is not allowed after ZSTD_CCtx_loadDictionary()
but it is still allowed after ZSTD_refPrefix()
2017-06-28 09:57:59 -07:00
Yann Collet 2e4274262d controlled dictMode 2017-06-27 17:09:12 -07:00
Yann Collet b7372933b8 implemented ZSTD_refPrefix() 2017-06-27 15:49:12 -07:00
Yann Collet 7d3816183f exposed ZSTD_MAGIC_DICTIONARY in zstd.h
makes it easier to explain ZSTD_dictMode
2017-06-27 13:50:34 -07:00
Yann Collet fecc721fd9 added parameter ZSTD_p_refDictContent 2017-06-27 11:46:39 -07:00
Yann Collet c7fb884eea fixed minor conversion warning 2017-06-26 18:02:23 -07:00
Yann Collet dde10b23fe refactored ZSTD_estimateDStreamSize()
now uses windowSize as argument.
Also : created ZSTD_estimateDStreamSize_fromFrame()
2017-06-26 17:44:26 -07:00
Yann Collet 09ae03a570 ZSTD_estimateCDictSize_advanced()
ZSTD_estimateCDictSize() now uses same arguments as ZSTD_createCDict()
ZSTD_estimateCDictSize_advanced() uses same arguments as ZSTD_createCDict_advanced()
2017-06-26 16:47:32 -07:00
Yann Collet 0c9a915a28 ZSTD_estimateCStreamSize_advanced() 2017-06-26 16:02:25 -07:00
Yann Collet 31af8290d1 ZSTD_estimateCCtx_advanced()
ZSTD_estimateCCtx() is now a "simple" function,
taking int compressionLevel as single argument.

ZSTD_estimateCCtx_advanced() takes a CParams argument,
which is both more complete and more complex to generate.
2017-06-26 15:52:39 -07:00
Yann Collet 379f9d874d Merge pull request #733 from terrelln/pzstd-deadlock
[pzstd] Fix potential deadlock in the case of errors
2017-06-23 14:18:55 -07:00
Yann Collet ef269c1b68 Merge pull request #725 from facebook/advancedAPI2
New Advanced API
2017-06-23 09:50:47 -07:00
Yann Collet 6122af9cf5 --no-big-tests for msan
msan tests require too much RAM for 2 GB VM
2017-06-23 00:37:15 -07:00
Yann Collet cca1ed14db Merge branch 'advancedAPI2' of github.com:facebook/zstd into advancedAPI2 2017-06-23 00:13:03 -07:00
Yann Collet 32f546c763 zstreamtest : disabled multi-threading tests with --newapi --no-big-tests
--no-big-tests is typically used in combination with qemu-user-static
qemu-user-static allocated 4 GB of RAM upfront.
On 2 GB VM, this can degenerate into a crash.
It's not a problem as long as memory is not used.
But with multi-threading enabled, memory fragmentation kicks in,
so the amoung of RAM effectively touched increases,
and can pass beyond the 2 GB limit of the VM.

In single-threaded mode, there is no such issue :
memory requirement is smaller, and remains well-located,
so very little fragmentation is expected.

This modification should make `qemu-arm-static zstreamtests --newapi --no-big-tests`
work fine on Travis CI.
2017-06-23 00:09:02 -07:00
Nick Terrell f570609286 [pzstd] Fix deadlock in the case of errors
Fixes #720.
2017-06-22 18:09:42 -07:00
Yann Collet 0d0fea96bc Merge pull request #732 from terrelln/dev
[linux] Update patches
2017-06-21 23:46:54 -07:00
Yann Collet f99c2c1a22 reduced --no-big-tests even more
to pass tests on qemu-aarch64
2017-06-21 23:35:58 -07:00
Nick Terrell aff86be75d [linux] Write all the patch summaries 2017-06-21 20:23:44 -07:00
Yann Collet 49f8459d11 fixed minor cast warning 2017-06-21 18:43:39 -07:00
Yann Collet 4d3bdcf130 reduced CLevelMax for --no-big-tests 2017-06-21 18:20:02 -07:00
Yann Collet bfc2f00080 --no-big-tests for zstreamtest
Apply --no-big-tests for tsan tests
2017-06-21 17:57:14 -07:00
Yann Collet ecb0f46866 add controls over streaming buffers 2017-06-21 17:25:01 -07:00
Yann Collet 224e7a1053 added --list command contrib by @paulcruz74 2017-06-21 15:58:36 -07:00
Yann Collet a69b6ec503 Merge pull request #730 from paulcruz74/dev
Patching ratio and non-existent file bugs
2017-06-21 15:56:05 -07:00
Yann Collet dce789281b fixed : decompression of skippable frames in streaming mode 2017-06-21 15:53:42 -07:00
Nick Terrell 0a94fead41 [linux] Rename last 2 diffs to patch names 2017-06-21 15:27:07 -07:00
Nick Terrell f3ad4062ba [linux] Add summaries to xxhash and zstd patches 2017-06-21 15:27:07 -07:00
Nick Terrell 34b0a0883a [linux] Rename diffs to patch names 2017-06-21 15:27:07 -07:00
Nick Terrell b789860db0 [linux] Switch BtrFS compression level to 1 2017-06-21 15:27:02 -07:00
Yann Collet 991f9dfcde switched fileio.c to ZSTD_DEBUG trigger macro 2017-06-21 15:16:13 -07:00
Yann Collet 204b6b7ef6 fixed streaming buffered allocation with CDict compression 2017-06-21 15:13:00 -07:00
Yann Collet 1e4129b27b fixed dangling pointer risk, detected by @terrelln 2017-06-21 13:26:10 -07:00
Paul Cruz d549254d3b Merge pull request #3 from facebook/dev
pulling changes
2017-06-21 12:42:38 -07:00
Paul Cruz 5be0f5544e added newline to align output 2017-06-21 12:41:10 -07:00
Paul Cruz 6f5fe71041 added error check for when file could not be opened 2017-06-21 12:37:23 -07:00
Paul Cruz 74a725da69 reversed calculation of ratio 2017-06-21 12:27:20 -07:00
Yann Collet 83095970e6 free cdictLocal faster, suggested by @terrelln 2017-06-21 12:26:40 -07:00
Yann Collet c16748b137 added test cases for new command --list 2017-06-21 12:09:53 -07:00
Yann Collet 7bd1a2900e added ZSTD_dictMode_e to control dictionary loading mode 2017-06-21 11:50:33 -07:00
Yann Collet 9c56b12938 Merge pull request #723 from paulcruz74/dev
Adding zstd -l
2017-06-21 09:41:55 -07:00
Yann Collet e51d51bdf7 fixed memcpy() overlap 2017-06-20 17:44:55 -07:00
Paul Cruz db3606e249 added test for files with no frame content size 2017-06-20 17:43:36 -07:00
Yann Collet 466f92eaa6 removed one useless streaming compression stage, detected by @terrelln 2017-06-20 16:25:29 -07:00
Yann Collet c3bce24ef4 fixed potential dangling pointer, detected by @terrelln 2017-06-20 16:09:11 -07:00
Paul Cruz a73c2a444a added tests with null files, changed condition to check that the file is more than 0 bytes 2017-06-20 14:33:08 -07:00
Yann Collet 78b8234554 fixed comments, following suggestion by @terrelln 2017-06-20 14:26:48 -07:00
Paul Cruz 58c19b4202 spacing matters for test scripts 2017-06-20 14:14:53 -07:00
Yann Collet b44ab82f7a ensure new ZSTD_strategy starts at value 1 2017-06-20 14:11:49 -07:00
Paul Cruz 0765602903 fixed error where extremely small files were not being detected as not compressed with zstd 2017-06-20 14:04:46 -07:00