e7e5a8cef7
made fullbench compatible with multi-threading
...
fullbench 61/62 measure speed of ZSTD_compress_generic with 2 threads
2017-06-29 18:56:24 -07:00
afb0aca739
zstreamtest : big tests are only enabled in 64-bits mode
...
to avoid requesting too much memory in 32-bits mode during MT tests
2017-06-29 18:19:09 -07:00
2e84bec9ac
updated fullbench to also measure ZSTD_compress_generic()
...
will make it possible to visualize
optimization opportunity for ZSTD_e_end
2017-06-29 13:03:10 -07:00
7303ed5d74
minor : specify compression level in tags
2017-06-29 11:30:31 -07:00
590937df20
Merge pull request #739 from facebook/refPrefix
...
ZSTD_refPrefix
2017-06-29 04:36:03 -07:00
811deaea6f
Merge pull request #736 from terrelln/cover-default-api
...
[zdict] Make COVER the default algorithm
2017-06-28 20:25:36 -07:00
ad5f29918e
Merge pull request #737 from facebook/advancedAPI2
...
simplified API for *initStatic*()
2017-06-28 20:25:22 -07:00
62f7efc04a
added a test to ensure ZSTD_refPrefix() doesn't impact the following compression job
2017-06-28 16:25:13 -07:00
7ac4724bd2
removed fnum from DISPLAY statements
2017-06-28 13:00:49 -07:00
e667d33b0b
fixed generation of buggy test, corrected DISPLAY statements for errors
2017-06-28 12:19:37 -07:00
33a6639039
fixed ZSTD_refPrefix with Multithread-enabled CCtx
2017-06-28 11:09:43 -07:00
1fd2df9c2c
fixed zstreamtest
...
modifying compression parameters is not allowed after ZSTD_CCtx_loadDictionary()
but it is still allowed after ZSTD_refPrefix()
2017-06-28 09:57:59 -07:00
2e4274262d
controlled dictMode
2017-06-27 17:09:12 -07:00
b7372933b8
implemented ZSTD_refPrefix()
2017-06-27 15:49:12 -07:00
fecc721fd9
added parameter ZSTD_p_refDictContent
2017-06-27 11:46:39 -07:00
09a5bbe22e
Merge pull request #735 from terrelln/xz-default
...
[zstdcli] Support xz by default when liblzma is available
2017-06-27 09:42:02 -07:00
5b7fd7c422
[zdict] Make COVER the default algorithm
2017-06-26 21:09:22 -07:00
20eeb243d1
Merge pull request #729 from paulcruz74/corpus
...
Corpus
2017-06-26 17:47:28 -07:00
dde10b23fe
refactored ZSTD_estimateDStreamSize()
...
now uses windowSize as argument.
Also : created ZSTD_estimateDStreamSize_fromFrame()
2017-06-26 17:44:26 -07:00
09ae03a570
ZSTD_estimateCDictSize_advanced()
...
ZSTD_estimateCDictSize() now uses same arguments as ZSTD_createCDict()
ZSTD_estimateCDictSize_advanced() uses same arguments as ZSTD_createCDict_advanced()
2017-06-26 16:47:32 -07:00
0c9a915a28
ZSTD_estimateCStreamSize_advanced()
2017-06-26 16:02:25 -07:00
31af8290d1
ZSTD_estimateCCtx_advanced()
...
ZSTD_estimateCCtx() is now a "simple" function,
taking int compressionLevel as single argument.
ZSTD_estimateCCtx_advanced() takes a CParams argument,
which is both more complete and more complex to generate.
2017-06-26 15:52:39 -07:00
6aeb50ea0d
[zstdcli] Add unlzma and unxz symlinks
2017-06-26 11:24:36 -07:00
298fe9f23b
added cli tests to test-all
2017-06-23 17:10:41 -07:00
3a295a91f8
added additional condition so large offsets into the dictionary are not generated past windowSize
2017-06-23 15:54:51 -07:00
2085375816
fixed bug detected by the API test
2017-06-23 13:44:24 -07:00
8cd134559d
type warnings
2017-06-23 12:00:48 -07:00
4219acc60a
fixed bus error bug
2017-06-23 11:22:29 -07:00
ef269c1b68
Merge pull request #725 from facebook/advancedAPI2
...
New Advanced API
2017-06-23 09:50:47 -07:00
cca1ed14db
Merge branch 'advancedAPI2' of github.com:facebook/zstd into advancedAPI2
2017-06-23 00:13:03 -07:00
32f546c763
zstreamtest : disabled multi-threading tests with --newapi --no-big-tests
...
--no-big-tests is typically used in combination with qemu-user-static
qemu-user-static allocated 4 GB of RAM upfront.
On 2 GB VM, this can degenerate into a crash.
It's not a problem as long as memory is not used.
But with multi-threading enabled, memory fragmentation kicks in,
so the amoung of RAM effectively touched increases,
and can pass beyond the 2 GB limit of the VM.
In single-threaded mode, there is no such issue :
memory requirement is smaller, and remains well-located,
so very little fragmentation is expected.
This modification should make `qemu-arm-static zstreamtests --newapi --no-big-tests`
work fine on Travis CI.
2017-06-23 00:09:02 -07:00
2e8cc6f12a
added sizeof for clarity
2017-06-22 15:52:33 -07:00
6d7f442120
changed makefile test
2017-06-22 15:44:01 -07:00
b325a2e4db
changed assignment
2017-06-22 15:36:28 -07:00
2935239544
added description for dictionary cli tests
2017-06-22 15:23:59 -07:00
97a8f89bec
made changes to cli test so it runs properly
2017-06-22 15:13:41 -07:00
829eb29033
added cli test for decodecorpus inside tests/Makefile. Also changed calculation of offset
2017-06-22 14:43:44 -07:00
028fb9d01b
added cli test case in Makefile
2017-06-22 13:53:39 -07:00
98751f69e7
should be updating seed whenever multiple files are generated
2017-06-22 10:23:36 -07:00
84cfa07d2d
changed format of command to --use-dict=#
2017-06-22 10:04:14 -07:00
f99c2c1a22
reduced --no-big-tests even more
...
to pass tests on qemu-aarch64
2017-06-21 23:35:58 -07:00
04094f37e9
fixed offset in this case os that it always goes past src start
2017-06-21 18:47:40 -07:00
49f8459d11
fixed minor cast warning
2017-06-21 18:43:39 -07:00
0950b3159a
more meaningful names for count variables
2017-06-21 18:30:27 -07:00
0b6eedeace
malloc samples instead of static allocation
2017-06-21 18:24:19 -07:00
4d3bdcf130
reduced CLevelMax for --no-big-tests
2017-06-21 18:20:02 -07:00
52a13abf3f
removed comment regarding windowSize requirement
2017-06-21 18:13:52 -07:00
32e36c2d9a
removed requirement that offset be less than windowSize
2017-06-21 18:00:47 -07:00
bfc2f00080
--no-big-tests for zstreamtest
...
Apply --no-big-tests for tsan tests
2017-06-21 17:57:14 -07:00
eeff2d0e3c
added const x2
2017-06-21 17:48:37 -07:00
04253e21bb
added const
2017-06-21 17:48:01 -07:00
2ceaec1aad
got rid of unnecessary casting to void*
2017-06-21 17:46:37 -07:00
ccae9ec807
added comment addressing use of bits in frame header descriptor
2017-06-21 17:43:21 -07:00
dce789281b
fixed : decompression of skippable frames in streaming mode
2017-06-21 15:53:42 -07:00
c16748b137
added test cases for new command --list
2017-06-21 12:09:53 -07:00
7bd1a2900e
added ZSTD_dictMode_e to control dictionary loading mode
2017-06-21 11:50:33 -07:00
db3606e249
added test for files with no frame content size
2017-06-20 17:43:36 -07:00
acf0df678c
fixed strange spacing
2017-06-20 15:41:43 -07:00
69bc4fab25
free the context object once done with decompression
2017-06-20 15:37:14 -07:00
ceb4f65620
allocate statically so we don't have to deal with freeing
2017-06-20 15:31:54 -07:00
a73c2a444a
added tests with null files, changed condition to check that the file is more than 0 bytes
2017-06-20 14:33:08 -07:00
58c19b4202
spacing matters for test scripts
2017-06-20 14:14:53 -07:00
ff8f83bd47
fixed fuzzer test
2017-06-20 12:17:32 -07:00
b07d0af2ba
added additional tests for error detection
2017-06-20 11:54:44 -07:00
43ffcdde40
const dictOffset
2017-06-20 11:27:39 -07:00
1e0d125e76
const value
2017-06-20 11:15:26 -07:00
0034129270
made changes to accommodate single file use case
2017-06-20 11:07:38 -07:00
6a99de2dd8
change test to list info on .zst files
2017-06-20 10:45:06 -07:00
36e14b1963
added description to advanced usage, changed to use-dict
2017-06-20 10:34:34 -07:00
15102069a1
removed testing code from generation code
2017-06-20 10:25:16 -07:00
ff5589c091
added comments for clarity
2017-06-20 10:17:38 -07:00
551fb7ca05
changed error condition
2017-06-20 10:14:03 -07:00
b8237c6c66
refactoring for readability
2017-06-20 10:12:13 -07:00
6f17de2132
changed computation for readability
2017-06-20 09:50:28 -07:00
4a4f9ba262
const value that doesn't change
2017-06-19 17:46:21 -07:00
3a6856edca
changed scope to avoid errors
2017-06-19 17:43:16 -07:00
7d2b5613b4
adhere to min dict size
2017-06-19 17:37:14 -07:00
ea2af1ffc4
changed dictionary to stay reasonable size during test
2017-06-19 17:34:50 -07:00
fdbb07283a
added test to runTestMode
2017-06-19 17:29:15 -07:00
bdec7b8ce7
fixed for loop declaration, fixed test case
2017-06-19 17:18:43 -07:00
23098b53e8
wrote a test for dictionary corpus
2017-06-19 17:13:00 -07:00
7a26390989
changing for loop declaration
2017-06-19 16:39:57 -07:00
147ef05f12
for loop declaration not allowed
2017-06-19 16:38:51 -07:00
791352ad0e
scope change
2017-06-19 16:32:38 -07:00
365e265a8e
changed scope so that warning disappears
2017-06-19 16:28:40 -07:00
f4f928595b
changed scope to get rid of mixed code/variables warning
2017-06-19 16:27:05 -07:00
7ea025cc7d
added in struct to keep functions more orderly
2017-06-19 16:23:00 -07:00
b91b810c64
made some small changes to dictInfo struct and initDictInfo
2017-06-19 16:05:05 -07:00
1ee4f6b36c
cleaning up code
2017-06-19 15:55:01 -07:00
2a52d72712
added comment to anotate decompressed error code
2017-06-19 14:21:35 -07:00
73ebb646d8
small stylistic changes
2017-06-19 14:20:35 -07:00
34f4e5cab2
changed if statement styling
2017-06-19 14:07:56 -07:00
1fc1a35dc4
added calls to free to match calls to malloc, made some stylistic changes, added init function for new struct
2017-06-19 14:02:27 -07:00
8f4fa43fe8
got files to decompress
2017-06-19 13:21:50 -07:00
688952062b
minor declaration statement warning fix
2017-06-19 11:53:42 -07:00
d7a3bffba9
new api : setting compression parameters is refused if a dictionary is already loaded
2017-06-19 11:53:01 -07:00
da4ddc1f76
fixed decompression so that it uses compressed version
2017-06-19 11:40:45 -07:00
33467add60
made changes so that HUF_writeCTable no longer fails
2017-06-19 11:27:50 -07:00
ed1d039127
newapi fuzzer tests : random flush orders in main loop
2017-06-19 11:07:33 -07:00
09ac51b56b
small style changes
2017-06-19 09:24:38 -07:00
f5deae8a67
new api fuzzer tests : frame parameters are randonly set
2017-06-18 23:41:38 -07:00
ea3630a889
new api fuzzer set : dictionary is randomly set
2017-06-18 23:31:55 -07:00
8dee0ec99e
new api fuzzer tests : compression parameters are randomly set
2017-06-18 23:25:15 -07:00
bd05b19713
removed macro CLAMP from decodecorpus
...
never used,
and would duplicate with CLAMP from zstd_compress.c
2017-06-16 18:16:27 -07:00
01743a36e7
fuzzer tests for new API
2017-06-16 17:56:41 -07:00
2f30433cd7
fixing general style
2017-06-15 18:06:32 -07:00
ddd1ab710c
added tests in playTest.sh
2017-06-15 16:53:03 -07:00
623b8fce85
add print statements for debugging and change malloc with updated size
2017-06-15 11:37:24 -07:00
52934fbfc8
added print statements for debugging, fixed first segfault
2017-06-15 09:47:29 -07:00
03a208e27c
changed samples to geometric distribution so that it is huffman compressible
2017-06-14 18:06:16 -07:00
d93207a79f
changed dictionary from using fixed amount of bytes for the header / entropy tables
2017-06-14 17:23:56 -07:00
664ed05ff6
changed randomly generated samples to randomly selected from the dictionary content
2017-06-14 16:42:51 -07:00
3b0e24c06b
added error message to catch case where size not given
2017-06-14 12:04:22 -07:00
aa95bc0a5f
code refactoring and adding some print statements for debugging
2017-06-14 10:54:47 -07:00
07cfc975bb
debugging error with finalizeDictionary() not writing dictionary properly
2017-06-13 17:27:02 -07:00
8dd621f788
changed makefile to include zdict functions, wrote out dictionary to path
2017-06-13 14:19:35 -07:00
f35f252e36
added code to generate dictionary using finalizeDictionary
2017-06-13 11:54:43 -07:00
37e1b1488b
removed previous double free crash, used new crash function
2017-06-09 13:57:18 -07:00
be9b0ae627
added error function for different compilation modes
2017-06-09 13:41:34 -07:00
11c3987baf
added code to extend the offset when a dictionary is detected
2017-06-09 09:48:56 -07:00
125ed59968
made sure dictionary ID was being written in the frame header of each file
2017-06-08 17:27:15 -07:00
233ee5334e
set the lower bits of frame header descriptor if dictionary is needed
2017-06-08 17:11:33 -07:00
c2d909e396
added code for generating dictionary/test files randomly. Still need to make sure dictionary ID matches
2017-06-08 17:06:30 -07:00
8c910d2097
updated ZSTDMT streaming API
...
ZSTDMT streaming API is now similar
and has same capabilites as single-thread streaming API.
It makes it easier to blend them together.
2017-06-03 01:15:02 -07:00
cdf7e82222
Added ZSTD_initStaticCDict()
2017-05-25 18:05:49 -07:00
57827f906f
added ZSTD_initStaticDDict()
2017-05-25 15:44:06 -07:00
25989e361c
updated ZSTD_estimate?DictSize() to pass parameter byReference
...
resulting ?Dict object is smaller when created byReference.
Seems better than a documentation note.
2017-05-25 15:07:37 -07:00
0fdc71c3dc
added ZSTD_initStaticDCtx()
2017-05-24 17:41:41 -07:00
c7fe262dc9
added ZSTD_initStaticCCtx()
...
makes it possible to statically or externally allocate CCtx.
static CCtx will only use provided memory area,
it will never resize nor malloc.
2017-05-23 13:20:41 -07:00
60a557e7fd
fixing symbol.c test
...
I believe it would be better to rely on fuzzer tests compiled with dll
2017-05-19 11:27:43 -07:00
fa3671eac7
changed ZSTD_BLOCKSIZE_ABSOLUTEMAX into ZSTD_BLOCKSIZE_MAX
...
Also :
change ZSTD_getBlockSizeMax() into ZSTD_getBlockSize()
created ZSTD_BLOCKSIZELOG_MAX
2017-05-19 10:51:30 -07:00
009d604e00
ZSTD_compress_generic() supports multiple successive frames
...
also : clarified streaming API implementation
2017-05-19 10:17:59 -07:00
0bdb575c31
Merge branch 'dev' into advancedAPI2
2017-05-16 16:32:29 -07:00
2d4d31c18a
removed gcc compilation flag -Wbad-function-cast
...
It makes it more difficult to directly cast the result of a function,
requiring to store the result in an intermediate variable.
It does not necessarily help readability,
and this restriction can be difficult to overcome in some constructions,
like some macros.
also : fixed minor Visual conversion warnings in datagencli.c
2017-05-16 11:34:38 -07:00
83d0c764dc
added several compilation flags
2017-05-15 17:15:46 -07:00
a5ffe3d370
pushed enum values for strategy by one (ZSTD_fast==1)
...
this makes it possible to use `0` to mean:
"do not change strategy"
2017-05-12 16:29:19 -07:00
5f2c7213c7
Merge remote-tracking branch 'upstream/dev' into btopt
...
* upstream/dev: (305 commits)
added test for ZSTD_estimateCStreamSize()
changed variable name, for clarity
fixed ZSTD_estimateCStreamSize()
shortened ZSTD_createCStream_Advanced()
fixed symbols test
added ZSTD_estimateDStreamSize()
changed name frameParams into frameHeader
regroup memory usage function declarations
separated ZSTD_estimateCStreamSize() from ZSTD_estimateCCtxSize()
bumped version number
added ZSTD_estimateCDictSize() and ZSTD_estimateDDictSize()
Updated ZSTD_freeCCtx()
updated ZSTD_estimateCCtxSize()
Updated ZSTD_sizeof_CCtx()
merged CCtx and CStream as a single same object
cli : -d and -t do not stop after a failed decompression
added dev branch CircleCI badge
added dev branch Appveyor badge
keep dev branch status only
creates a binary archive without the `programs` directory
...
2017-05-10 16:49:58 -07:00
30ab64e21d
added test for ZSTD_estimateCStreamSize()
2017-05-10 11:30:19 -07:00
461a9cc7c6
fixed symbols test
2017-05-09 16:53:09 -07:00
f16f4497ca
added ZSTD_estimateDStreamSize()
2017-05-09 16:18:17 -07:00
542c9dfcf8
changed name frameParams into frameHeader
...
ZSTD_frameParams => ZSTD_frameHeader
ZSTD_getFrameParams() -> ZSTD_getFrameHeader()
The new naming is more distinctive from ZSTD_frameParameters,
which is used during compression.
ZSTD_frameHeader is clearer in its intention to described frame header content.
It also implies we are decoding a ZSTD frame, hence we are at decoding stage.
2017-05-09 15:46:07 -07:00
fa8dadb294
separated ZSTD_estimateCStreamSize() from ZSTD_estimateCCtxSize()
...
for clarity
2017-05-08 18:24:16 -07:00
a1d6704d7f
added ZSTD_estimateCDictSize() and ZSTD_estimateDDictSize()
...
it complements ZSTD_estimateCCtxSize()
for the special case of ZSTD_initCStream_usingDict()
2017-05-08 17:51:49 -07:00
fc5145955a
updated ZSTD_estimateCCtxSize()
...
added a parameter streaming,
to estimate memory allocation size
when the CCtx is used for streaming (CStream).
Note : this function is not able to estimate
memory cost of a potential internal CDict
which can only happen when starting with ZSTD_initCStream_usingDict()
2017-05-08 17:07:59 -07:00
0be6fd3429
merged CCtx and CStream as a single same object
...
To be changed : ZSTD_sizeof_CCtx(), ZSTD_estimateCCtxSize()
2017-05-08 16:08:01 -07:00
01a1abfdb5
cli : -d and -t do not stop after a failed decompression
...
The problematic srcfile will be named on console/log,
but decompression/test will continue onto next file in the list.
2017-05-05 19:15:24 -07:00
f376d47c11
[CLI] Switch dictionary builder on CLI to cover
2017-05-02 11:18:27 -07:00
a51cab6e68
Merge pull request #678 from facebook/apiChange
...
Breaking API Change around CDict
2017-04-28 10:02:45 -07:00
36e79a1a1f
tests : -g is part of DEBUGFLAGS
2017-04-27 17:29:40 -07:00
c6915429f2
shortened Appveyor release tests
...
fuzzer supports time suffix `s` for "seconds"
2017-04-27 16:24:53 -07:00
a92cbb7004
Added a secondary test, checking dictID presence after setting noDictIdFLag=1
2017-04-27 15:08:56 -07:00
7d283cdfa2
added test for ZSTD_initCStream_usingCDict_advanced()
2017-04-27 14:48:34 -07:00
2f73427d35
added test for ZSTD_compress_usingCDict_advanced()
2017-04-27 14:39:39 -07:00
7321345fd2
fixed another VS2008 init error
2017-04-27 14:19:34 -07:00
f4bd857d81
created ZSTD_compress_usingCDict_advanced()
2017-04-27 11:31:55 -07:00
31533bacce
Changed ZSTD_createCDict_advanced()
...
It now only uses compressionParameters as argument.
It produces many changes throughout user code,
though hopefully they tend to be simple :
just provide the cParams part from existing ZSTD_parameters.
Some programs might depend on ZSTD_createCDict_advanced() to pass frame parameters.
This change will force them to revisit this strategy and fix it,
since frame parameters are effectively silently ignored in current version.
2017-04-27 00:29:04 -07:00
4de8632957
Add LZ4 compress/decompress support to CLI
2017-04-24 16:48:25 -07:00
230d7acc7d
cli : add support for --threads=# command
...
updated documentation
add relevant test case
2017-04-21 11:38:13 -07:00
c17e020c9a
disable assert when compiling paramgrill
...
paramgrill is a benchmark calibration function.
Speed accuracy is critical, it cannot be altered by assert.
2017-04-20 12:50:02 -07:00
2c5514c759
fixed ZSTDMT_initCStream_advanced()
...
Must use the new ZSTD_compressBegin_usingCDict_advanced()
to enforce correct frame parameters
2017-04-18 22:52:41 -07:00
0bb381dad8
added test for ZSTD_initCStream_advanced()
...
when params.fParams.contentSizeFlags = 1
and pledgedSrcSize = 0
then srcSize should be considered 0 (empty), and not "unknown"
2017-04-18 15:08:52 -07:00
4f818182b8
clarified frame parameters for ZSTD_compress*_usingCDict()
...
created ZSTD_compressBegin_usingCDict_internal(),
which gives direct control to frame Parameters.
ZSTD_resetCStream_internal() now points into it.
2017-04-17 18:29:06 -07:00
9c4c970399
Merge branch 'dev' into compressionFlow
2017-04-17 16:33:09 -07:00
d845dab69c
Fix input size too small to trigger zstdmt
2017-04-17 12:10:58 -07:00
c424ec2eae
Add multithreading tests to playTests.sh
2017-04-17 11:41:46 -07:00
2afb075382
Merge branch 'dev' into compressionFlow
2017-04-14 15:08:31 -07:00
f913cbed33
fixed : memory leak in fuzzer test
2017-04-13 22:46:51 -07:00
20d5e03893
content size is controlled at bufferless level
...
so it's active for all entry points
Also : added relevant test (wrong content size) in fuzzer
2017-04-11 18:34:02 -07:00
d37e1df2ab
Fix message
2017-04-11 17:33:26 -07:00
eb70d219fd
Add test of file > 4GB to playTests
2017-04-11 17:15:13 -07:00
4ee6b15dac
force contentSizeFlag=0 when using ZSTD_initCStream_usingCDict()
...
because by definition srcSize is not known when using this prototype.
added relevant test
Note : this use was already working, because at a later stage
(both ZSTD_compressBegin_usingCDict() and ZSTD_copyCCtx())
pledgedSrcSize=0 is translated into "unknown", no matter the frame parameter.
This is not correct, but of little importance,
as the medium term plan is to no longer set fParams within CDict
2017-04-11 11:59:44 -07:00
4b987ad8ce
Introduce ZSTD_initCStream_internal()
...
This is now the regroup point for ZSTD_initCStream*() functions
ZSTD_initCStream_advanced() now properly checks for parameters validity.
Also : added <assert.h> usage inside zstd_compress.c
Needs ZSTD_DEBUG=1 macro to be triggered.
Will be triggered by default from `tests` directory
2017-04-10 17:50:44 -07:00
1fa3b75369
Merge pull request #648 from facebook/zstdmt
...
improved zstreamtest --mt to trap bug #644
2017-04-07 06:09:52 -07:00
1652172b2d
Add refuse to overwrite test and fix format
2017-04-06 17:06:30 -07:00
9da11c6aae
Add test for overwriting read-only
2017-04-06 12:58:49 -07:00
ce80098f14
improved zstreamtest --mt to trap bug #644
2017-04-05 16:34:09 -07:00
9631e603ae
paramgrill : results from optimizer use cli format
...
Best found configuration is displayed
using documented cli advanced parameter syntax.
2017-04-04 16:54:33 -07:00
89b32f3db0
fix paramgrill -O#
...
ensure proposed config does not require more memory than necessary
2017-04-04 16:41:11 -07:00
2724f25567
fixed paramgrill -O#
...
find optimal settings for a minimum speed
2017-04-04 16:31:17 -07:00
c2007388a5
fixed bench.c : optional advanced parameters applied
...
before creating cdict
2017-04-04 15:35:06 -07:00
39a6cc5172
Make ZSTD_compress_usingCDict() respect contentSizeFlag
2017-04-03 21:09:55 -07:00
62ecad3819
Fix ZSTD_initCStream_usingCDict() to use dictionary
2017-04-03 21:05:59 -07:00
30c7698970
optimize ZSTDMT_compress() memory usage
...
does no longer allocate temporary buffers
when there is enough room in dstBuffer to decompress directly there.
(previous method would skip that for 1st chunk only).
Also : fix ZSTD_compressBound() for small srcSize
2017-03-31 18:27:03 -07:00
6476c51b86
Merge pull request #637 from facebook/zstdmt
...
Zstdmt
2017-03-30 14:18:37 -07:00
5152fb2cb2
Convert all tabs to spaces
2017-03-29 18:51:58 -07:00
1e1e26f2ac
fixed #634 : ZSTDMT_compressCCtx() doesn't provide frame content size in header
2017-03-29 17:09:59 -07:00
0ef6803707
added ZSTD_getFrameContentSize() to ZSTDMT unit test in fuzzer
...
Now tests fail, because ZSTDMT_compress() doesn't fill frame content size correctly.
2017-03-29 16:58:57 -07:00
41fefd573a
Improved speed tests
...
Better compatibility with Mac OS-X
Force attribution to a selected core # (Linux)
2017-03-26 23:52:19 -07:00
4c41d37fcc
changed test for new syntax
...
--dictID= and --maxdict=
2017-03-24 18:36:56 -07:00
f332ece468
dictBuilder fails to create dictionary on certain input
...
Properly expressed with an error code (see zstd_errors.h)
and a cli return code != 0
2017-03-23 16:24:02 -07:00
680e4e0953
Ignore symbolic links unless --force specified
2017-03-23 11:52:09 -07:00
042ba122ae
Change g_displayLevel to int and fix DISPLAYUPDATE flush
2017-03-23 11:21:59 -07:00
1b418bd3da
Merge remote-tracking branch 'origin/dev' into appveyor
2017-03-21 10:59:54 -07:00
5a2b857029
Update appveyor.yml to have short and long tests
2017-03-20 17:04:44 -07:00
c771977489
Fix decodecorpus clang compile errors
2017-03-20 16:02:24 -07:00
2f6c7e6a53
Fix windows compiler warnings for decodecorpus
2017-03-20 13:45:12 -07:00
170861da70
Merge pull request #610 from iburinoc/slowfuzzer
...
Strongly limit compression level when source is large for tests running on QEMU
2017-03-20 13:36:01 -07:00
0b0b10ec83
Add --no-big-tests flag for qemu fuzz-tests
2017-03-20 11:32:24 -07:00
7ebf2de02d
Add ability to strongly limit fuzzer test size with flag
2017-03-20 11:25:00 -07:00
35c147ab6a
Merge pull request #605 from iburinoc/term
...
Fix zstd not erroring on compressing to terminal without forceStdout
2017-03-17 14:02:48 -07:00
9a38dfa794
Only run IS_CONSOLE tests with a TTY
2017-03-17 12:32:18 -07:00
d973071e90
Add tests for compressed data on console
2017-03-16 16:25:19 -07:00
f5e50519e0
Prevent fuzz testers from combining large dicts with high clevel
2017-03-15 15:04:54 -07:00
3773bd318f
Merge pull request #601 from terrelln/nxz
...
Add xz and lzma support.
2017-03-13 22:00:03 -07:00
aa8bcf360f
Add xz and lzma support.
...
Finish feature started by @inikep.
* Add xz and lzma compression and decompression support to target `xzstd`.
* Fix bug in gzip decompression that silently accepted truncated files.
* Add gzip frame composition tests.
* Add xz/lzma compatibility tests.
* Add xz/lzma frame composition tests.
2017-03-13 18:45:41 -07:00
120df494e9
Update builds to not support legacy v01-v03
2017-03-13 14:44:08 -07:00
7c8f5d5bc7
Make test times overwritable
2017-03-09 16:05:23 -08:00
daec40db24
Update .travis.yml and Makefile for medium tests
2017-03-09 16:05:22 -08:00
eeb31eed39
s/ZSTD_btopt2/ZSTD_btultra/g
2017-03-09 11:44:25 -08:00
8b1d004031
added -Wformat-security flag, as recommended by @pixelb
2017-03-05 21:17:32 -08:00
27526c7201
make : added target shortest
...
shortest only run fast part of playTests.sh .
cc @iburinoc
2017-03-01 17:02:49 -08:00
c1c040eae1
added gzip tests
...
also : made sure zstd --format=gzip -V
would fail if gzip compatibility is not supported
2017-03-01 16:49:20 -08:00
51598510c0
Merge pull request #580 from facebook/speedStream
...
Improve streaming decompression speed
2017-03-01 10:59:51 -08:00
a33ae64204
fixed decoding skippable frames
2017-02-28 01:15:28 -08:00
c0b1731bce
added test for decompression with NULL dict and NULL DDict
...
previous version of ZSTD_decompressMultiFrame() would fail that test
2017-02-28 01:02:46 -08:00
8b3560e196
update gzip tests
2017-02-28 09:41:23 +01:00
1d7f30f9d4
Merge branch 'decompressStream' into dev
2017-02-27 20:55:22 -08:00
a81c2e7e44
Merge pull request #573 from facebook/ddict
...
Improved DDict memory usage
2017-02-27 20:54:42 -08:00
952d06fa9c
fullbench : -i0 displays list of functions to bench
2017-02-27 17:58:02 -08:00
67d86a74a5
added test case : --rm on stdin
...
must remain silent (instead of failing)
2017-02-27 16:09:20 -08:00
ea7589ce07
Merge pull request #571 from inikep/dev11
...
gzip tests
2017-02-27 13:54:33 -08:00
5d848527e6
use "./gzip" for gzip tests
2017-02-27 22:02:03 +01:00
b43d75154d
update gzip tests
2017-02-27 09:07:35 +01:00
1479c98661
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
2017-02-27 08:56:43 +01:00
2302bfa4bf
Merge branch 'dev' into testcorpus
2017-02-26 22:27:31 -08:00
bd7fa21deb
added ZSTD_refDDict()
...
Now DDict does no longer depends on DCtx duplication
2017-02-26 14:43:07 -08:00
8dff956dbf
Added DDict unit test in fuzzer
...
also : slightly modified loadEntropy :
know src must points at start of dictionary
2017-02-25 10:11:15 -08:00
682ae8a10e
Merge pull request #567 from inikep/dev11
...
faster Travis tests
2017-02-25 06:57:31 -08:00
22468b0cc3
Fix some more ARM compile errors
...
https://travis-ci.org/facebook/zstd/jobs/204807461
Can't get them to reproduce the compile errors locally,
tested fix by forcing that test to run on Travis.
2017-02-24 10:55:42 -08:00
a66b764d79
added tests for gzip
2017-02-24 16:09:17 +01:00
8740d6bcf5
fix uninitialized value warning
2017-02-24 09:24:55 +01:00
df9f9296e3
attempt to fix pthreat linking error
...
replacing -lpthread by -pthread
2017-02-24 00:16:05 -08:00
831b4890ce
minor tests/Makefile refactoring
...
and update of zstd_manual,html
2017-02-23 23:09:10 -08:00
d590291d86
Fix -Wsign-compare issues in decodecorpus.c
...
https://travis-ci.org/facebook/zstd/jobs/204423280
2017-02-23 16:11:08 -08:00
7e224ea03e
Merge pull request #562 from iburinoc/testcorpus
...
Create a tool that generates random, valid, Zstd frames for decoder testing
2017-02-23 15:19:36 -08:00
3cd8d50c34
Update CLI and link from educational decoder
2017-02-23 13:06:50 -08:00
485ca8c352
Update tests/README.md
2017-02-23 10:27:00 -08:00
f119b62055
Create a tool that generates random, valid, Zstd frames for decoder testing
...
Note: Does not handle dictionaries currently
2017-02-22 16:08:34 -08:00
9050e1925e
Change name to to findFrameCompressedSize and add skippable support
2017-02-22 12:12:34 -08:00
337ec875b6
minor tweaks
2017-02-22 10:31:30 +01:00
3d836bfd18
travis.yml: fix versionsTest target
2017-02-22 09:36:42 +01:00
346ce32ade
legacy.c: fix memory leaks
2017-02-21 20:10:21 +01:00
684858e7b7
fix memory leaks
2017-02-21 18:17:24 +01:00
f2da1b1f22
Merge pull request #550 from iburinoc/matchoverlenwrite
...
execSequence copies up to 2*WILDCOPY_OVERLENGTH extra
2017-02-16 18:14:47 -08:00
ff176d3208
Merge pull request #548 from inikep/gz_compress
...
Circle CI tests
2017-02-16 14:57:59 -08:00
0ed3901b05
Update overlength match test case
2017-02-16 13:36:57 -08:00
887eaa9e21
Fix wildcopy overwriting data still in window
2017-02-15 16:43:45 -08:00
9b5a1e9d97
added circle.yml
2017-02-14 20:06:41 +01:00
74b81ada25
Don't run test-pool with QEMU
...
> make test -n
...
./pool
> make test -n QEMU_SYS=valgrind
...
./legacy
# ./pool not run
2017-02-14 10:08:18 -08:00
5069b6c2c3
Merge branch 'dev' into multiframe
2017-02-10 10:08:55 -08:00
Yann Collet
Yann Collet
Paul Cruz
Nick Terrell
Yann Collet
Sean Purcell
Przemyslaw Skibinski