b5b9275e67
Rename estimateCCtxSize_advanced() and estimateCStreamSize_advanced()
2017-08-29 10:49:29 -07:00
18224608ff
Remove ZSTD_setCCtxParameter()
2017-08-25 13:58:41 -07:00
9911153723
Move jobSize and overlapLog in zstdmt to cctxParams
2017-08-25 13:14:51 -07:00
eb7bbab36a
Remove ZSTD_p_refDictContent and dictContentByRef
2017-08-25 11:11:45 -07:00
15fdeb9e41
Enforce nbThreads<=1 for estimateCCtxSize
2017-08-24 16:28:49 -07:00
1c81f725ff
Remove duplicated testing code
2017-08-23 15:47:15 -07:00
6f1a21c7e9
Remove formatting-only changes
2017-08-23 10:24:19 -07:00
8fd1636776
Remove unused functions
2017-08-22 13:33:58 -07:00
73c73bf16a
Reduce code duplication in zstreamtest
2017-08-21 12:41:19 -07:00
3587556873
[cover] Test small maxdict
2017-08-21 11:16:47 -07:00
91b30dbe84
Remove test parameter
2017-08-21 10:09:06 -07:00
f181f33bdf
Disable tests and refactor
2017-08-21 01:59:08 -07:00
023b24e6d4
Add cctx param tests
2017-08-20 22:55:07 -07:00
d6394cc4c3
fixed test-zstd-nolegacy
2017-08-20 10:15:44 -07:00
32fb407c9d
updated a bunch of headers
...
for the new license
2017-08-18 16:52:05 -07:00
8049556928
Merge pull request #778 from terrelln/bad-huff
...
[libzstd] Fix bug in Huffman decompresser
2017-08-07 14:05:58 -07:00
abe12b3399
[libzstd] Fix bug in Huffman decompresser
...
The zstd format specification doesn't enforce that Huffman compressed
literals (including the table) have to be smaller than the uncompressed
literals. The compressor will never Huffman compress literals if the
compressed size is larger than the uncompressed size. The decompresser
doesn't accept Huffman compressed literals with 4 streams whose compressed
size is at least as large as the uncompressed size.
* Make the decompresser accept Huffman compressed literals whose size
increases.
* Add a test case that exposes the bug. The compressed file has to be
statically generated, since the compressor won't normally produce files
that expose the bug.
2017-08-07 12:37:48 -07:00
e1abc2a367
Switch the sleep function to UTIL_sleepMilli
2017-08-07 11:49:13 -07:00
1e366f9dea
Add test for deadlock
2017-08-02 11:27:50 -07:00
5adceeed01
Allow queueSize=0 in pool.c and update poolTests
2017-07-31 10:10:16 -07:00
38ba7002f2
fixed minor warning on unused variable in shell function
2017-07-20 18:39:04 -07:00
5e6c5203f3
fixed fuzzer test for non OS-X platforms
2017-07-20 15:11:56 -07:00
1ca1288689
added --memtest=# command to fuzzer
...
to jump directly to relevant test section
2017-07-19 16:01:16 -07:00
44b0838253
Merge pull request #770 from terrelln/test-mode
...
[zstdcli] Fix -t in streaming mode
2017-07-18 15:40:59 -07:00
d0b27483ae
[zstdcli] Fix -t in streaming mode
2017-07-18 14:45:49 -07:00
cc1522351f
[libzstd] Fix bug in Huffman encoding
...
Summary:
Huffman encoding with a bad dictionary can encode worse than the
HUF_BLOCKBOUND(srcSize), since we don't filter out incompressible
input, and even if we did, the dictionaries Huffman table could be
ill suited to compressing actual data.
The fast optimization doesn't seem to improve compression speed,
even when I hard coded fast = 1, the speed didn't improve over hard coding
it to 0.
Benchmarks:
$ ./zstd.dev -b1e5
Benchmarking levels from 1 to 5
1#Synthetic 50% : 10000000 -> 3139163 (3.186), 524.8 MB/s ,1890.0 MB/s
2#Synthetic 50% : 10000000 -> 3115138 (3.210), 372.6 MB/s ,1830.2 MB/s
3#Synthetic 50% : 10000000 -> 3222672 (3.103), 223.3 MB/s ,1400.2 MB/s
4#Synthetic 50% : 10000000 -> 3276678 (3.052), 198.0 MB/s ,1280.1 MB/s
5#Synthetic 50% : 10000000 -> 3271570 (3.057), 107.8 MB/s ,1200.0 MB/s
$ ./zstd -b1e5
Benchmarking levels from 1 to 5
1#Synthetic 50% : 10000000 -> 3139163 (3.186), 524.8 MB/s ,1870.2 MB/s
2#Synthetic 50% : 10000000 -> 3115138 (3.210), 370.0 MB/s ,1810.3 MB/s
3#Synthetic 50% : 10000000 -> 3222672 (3.103), 223.3 MB/s ,1380.1 MB/s
4#Synthetic 50% : 10000000 -> 3276678 (3.052), 196.1 MB/s ,1270.0 MB/s
5#Synthetic 50% : 10000000 -> 3271570 (3.057), 106.8 MB/s ,1180.1 MB/s
$ ./zstd.dev -b1e5 ../silesia.tar
Benchmarking levels from 1 to 5
1#silesia.tar : 211988480 -> 73651685 (2.878), 429.7 MB/s ,1096.5 MB/s
2#silesia.tar : 211988480 -> 70158785 (3.022), 321.2 MB/s ,1029.1 MB/s
3#silesia.tar : 211988480 -> 66993813 (3.164), 243.7 MB/s , 981.4 MB/s
4#silesia.tar : 211988480 -> 66306481 (3.197), 226.7 MB/s , 972.4 MB/s
5#silesia.tar : 211988480 -> 64757852 (3.274), 150.3 MB/s , 963.6 MB/s
$ ./zstd -b1e5 ../silesia.tar
Benchmarking levels from 1 to 5
1#silesia.tar : 211988480 -> 73651685 (2.878), 429.7 MB/s ,1087.1 MB/s
2#silesia.tar : 211988480 -> 70158785 (3.022), 318.8 MB/s ,1029.1 MB/s
3#silesia.tar : 211988480 -> 66993813 (3.164), 246.5 MB/s , 981.4 MB/s
4#silesia.tar : 211988480 -> 66306481 (3.197), 229.2 MB/s , 972.4 MB/s
5#silesia.tar : 211988480 -> 64757852 (3.274), 149.3 MB/s , 963.6 MB/s
Test Plan:
I added a test case to the fuzzer which crashed with ASAN before the patch
and succeeded after.
2017-07-18 13:20:40 -07:00
fa3aa04ccd
Merge pull request #761 from paulcruz74/file-rename
...
renamed pool.c to poolTests.c
2017-07-14 09:09:45 -07:00
3a60efd3a9
policy change : ZSTDMT automatically caps nbThreads to ZSTDMT_NBTHREADS_MAX ( #760 )
...
Previously, ZSTDMT would refuse to create the compressor.
Also : increased ZSTDMT_NBTHREADS_MAX to 256,
updated doc,
and added relevant test
2017-07-13 10:17:23 -07:00
052a95f77c
fix : ZSTDMT_compress_advanced() correctly generates checksum
...
when params.fParams.checksumFlag==1.
This use case used to be impossible when only ZSTD_compress() was available
2017-07-11 17:18:26 -07:00
ef0ff7fe7f
zstdmt: removed margin for improved memory usage
2017-07-11 08:54:29 -07:00
4616fad18b
improved ZSTDMT_compress() memory usage
...
does not need the input buffer for streaming operations
also : reduced a few tests time length
2017-07-10 17:16:41 -07:00
670b1fc547
optimized memory usage for ZSTDMT_compress()
...
Previously, each job would reserve a CCtx right before being posted.
The CCtx would be "part of the job description",
and only released when the job is completed (aka flushed).
For ZSTDMT_compress(), which creates all jobs first and only join at the end,
that meant one CCtx per job.
The nb of jobs used to be == nb of threads,
but since latest modification,
which reduces the size of jobs in order to spread the load of difficult areas,
it also increases the nb of jobs for large sources / small compression level.
This resulted in many more CCtx being created.
In this new version, CCtx are reserved within the worker thread.
It guaranteea there cannot be more CCtx reserved than workers (<= nb threads).
To do that, it required to make the CCtx Pool multi-threading-safe :
it can now be called from multiple threads in parallel.
2017-07-10 16:30:55 -07:00
3510efb02d
fix : custom allocator correctly propagated to child contexts
2017-07-10 14:21:40 -07:00
ee3423d709
extended fuzzer MT memory tests
2017-07-10 14:09:16 -07:00
88da8f1816
fix : propagate custom allocator to ZSTDMT though ZSTD_CCtx_setParameter()
...
also : compile fuzzer with MT enabled
2017-07-10 14:02:33 -07:00
f9524cf366
added --memtest to fuzzer
2017-07-10 13:48:41 -07:00
e32fb0c1fe
added ZSTD_sizeof_CCtx() test
2017-07-10 12:29:57 -07:00
89190ef07d
renamed pool.c to poolTests.c
2017-07-10 11:32:30 -07:00
ed0243a63c
removed zbufftest from list of `all` tests
2017-07-07 16:16:14 -07:00
990449b89d
new field : ZSTD_frameHeader.frameType
...
Makes frame type (zstd,skippable) detection more straighforward.
ZSTD_getFrameHeader set frameContentSize=ZSTD_CONTENTSIZE_UNKNOWN to mean "field not present"
2017-07-07 15:21:35 -07:00
7758ed8458
fixed fullbench, part 2
2017-07-06 02:48:00 -07:00
9b2c1acfc0
fixed fullbench
2017-07-06 02:22:57 -07:00
27e883371d
fixed wrong assert() condition
...
A single job created by ZSTDMT_compress() can be < 256KB
if data to compress is < 256 KB
(in which case it is delegated to single thread mode)
2017-07-04 19:33:16 -07:00
2cb9774f5e
more precise estimation of amount to flush at end of stream (single thread mode)
...
also : can use DEBUGLEVEL variable in /tests
2017-07-04 12:39:26 -07:00
5051dd39ca
Merge pull request #743 from facebook/fullbench
...
compress_generic() automatic optimization opportunities
2017-07-03 21:26:38 -07:00
95c4a6e2c8
Merge pull request #745 from terrelln/libfuzzer
...
[fuzz] Add libFuzzer targets
2017-07-03 15:15:20 -07:00
bea0f0cfa0
[fuzz] Move from fuzz/ to tests/fuzz/
2017-07-03 12:40:12 -07:00
4b26306cb8
blindfix : fullbench's one-time leak, detected by valgrind
2017-07-01 08:03:59 -07:00
c07e43c2b5
added --show-leak-kind=all to valgrind tests
2017-07-01 07:05:11 -07:00
b5bb7c6d95
fixed Visual compilation of fullbench-dll
2017-06-29 19:59:37 -07:00
e7e5a8cef7
made fullbench compatible with multi-threading
...
fullbench 61/62 measure speed of ZSTD_compress_generic with 2 threads
2017-06-29 18:56:24 -07:00
afb0aca739
zstreamtest : big tests are only enabled in 64-bits mode
...
to avoid requesting too much memory in 32-bits mode during MT tests
2017-06-29 18:19:09 -07:00
2e84bec9ac
updated fullbench to also measure ZSTD_compress_generic()
...
will make it possible to visualize
optimization opportunity for ZSTD_e_end
2017-06-29 13:03:10 -07:00
7303ed5d74
minor : specify compression level in tags
2017-06-29 11:30:31 -07:00
590937df20
Merge pull request #739 from facebook/refPrefix
...
ZSTD_refPrefix
2017-06-29 04:36:03 -07:00
811deaea6f
Merge pull request #736 from terrelln/cover-default-api
...
[zdict] Make COVER the default algorithm
2017-06-28 20:25:36 -07:00
ad5f29918e
Merge pull request #737 from facebook/advancedAPI2
...
simplified API for *initStatic*()
2017-06-28 20:25:22 -07:00
62f7efc04a
added a test to ensure ZSTD_refPrefix() doesn't impact the following compression job
2017-06-28 16:25:13 -07:00
7ac4724bd2
removed fnum from DISPLAY statements
2017-06-28 13:00:49 -07:00
e667d33b0b
fixed generation of buggy test, corrected DISPLAY statements for errors
2017-06-28 12:19:37 -07:00
33a6639039
fixed ZSTD_refPrefix with Multithread-enabled CCtx
2017-06-28 11:09:43 -07:00
1fd2df9c2c
fixed zstreamtest
...
modifying compression parameters is not allowed after ZSTD_CCtx_loadDictionary()
but it is still allowed after ZSTD_refPrefix()
2017-06-28 09:57:59 -07:00
2e4274262d
controlled dictMode
2017-06-27 17:09:12 -07:00
b7372933b8
implemented ZSTD_refPrefix()
2017-06-27 15:49:12 -07:00
fecc721fd9
added parameter ZSTD_p_refDictContent
2017-06-27 11:46:39 -07:00
09a5bbe22e
Merge pull request #735 from terrelln/xz-default
...
[zstdcli] Support xz by default when liblzma is available
2017-06-27 09:42:02 -07:00
5b7fd7c422
[zdict] Make COVER the default algorithm
2017-06-26 21:09:22 -07:00
20eeb243d1
Merge pull request #729 from paulcruz74/corpus
...
Corpus
2017-06-26 17:47:28 -07:00
dde10b23fe
refactored ZSTD_estimateDStreamSize()
...
now uses windowSize as argument.
Also : created ZSTD_estimateDStreamSize_fromFrame()
2017-06-26 17:44:26 -07:00
09ae03a570
ZSTD_estimateCDictSize_advanced()
...
ZSTD_estimateCDictSize() now uses same arguments as ZSTD_createCDict()
ZSTD_estimateCDictSize_advanced() uses same arguments as ZSTD_createCDict_advanced()
2017-06-26 16:47:32 -07:00
0c9a915a28
ZSTD_estimateCStreamSize_advanced()
2017-06-26 16:02:25 -07:00
31af8290d1
ZSTD_estimateCCtx_advanced()
...
ZSTD_estimateCCtx() is now a "simple" function,
taking int compressionLevel as single argument.
ZSTD_estimateCCtx_advanced() takes a CParams argument,
which is both more complete and more complex to generate.
2017-06-26 15:52:39 -07:00
6aeb50ea0d
[zstdcli] Add unlzma and unxz symlinks
2017-06-26 11:24:36 -07:00
298fe9f23b
added cli tests to test-all
2017-06-23 17:10:41 -07:00
3a295a91f8
added additional condition so large offsets into the dictionary are not generated past windowSize
2017-06-23 15:54:51 -07:00
2085375816
fixed bug detected by the API test
2017-06-23 13:44:24 -07:00
8cd134559d
type warnings
2017-06-23 12:00:48 -07:00
4219acc60a
fixed bus error bug
2017-06-23 11:22:29 -07:00
ef269c1b68
Merge pull request #725 from facebook/advancedAPI2
...
New Advanced API
2017-06-23 09:50:47 -07:00
cca1ed14db
Merge branch 'advancedAPI2' of github.com:facebook/zstd into advancedAPI2
2017-06-23 00:13:03 -07:00
32f546c763
zstreamtest : disabled multi-threading tests with --newapi --no-big-tests
...
--no-big-tests is typically used in combination with qemu-user-static
qemu-user-static allocated 4 GB of RAM upfront.
On 2 GB VM, this can degenerate into a crash.
It's not a problem as long as memory is not used.
But with multi-threading enabled, memory fragmentation kicks in,
so the amoung of RAM effectively touched increases,
and can pass beyond the 2 GB limit of the VM.
In single-threaded mode, there is no such issue :
memory requirement is smaller, and remains well-located,
so very little fragmentation is expected.
This modification should make `qemu-arm-static zstreamtests --newapi --no-big-tests`
work fine on Travis CI.
2017-06-23 00:09:02 -07:00
2e8cc6f12a
added sizeof for clarity
2017-06-22 15:52:33 -07:00
6d7f442120
changed makefile test
2017-06-22 15:44:01 -07:00
b325a2e4db
changed assignment
2017-06-22 15:36:28 -07:00
2935239544
added description for dictionary cli tests
2017-06-22 15:23:59 -07:00
97a8f89bec
made changes to cli test so it runs properly
2017-06-22 15:13:41 -07:00
829eb29033
added cli test for decodecorpus inside tests/Makefile. Also changed calculation of offset
2017-06-22 14:43:44 -07:00
028fb9d01b
added cli test case in Makefile
2017-06-22 13:53:39 -07:00
98751f69e7
should be updating seed whenever multiple files are generated
2017-06-22 10:23:36 -07:00
84cfa07d2d
changed format of command to --use-dict=#
2017-06-22 10:04:14 -07:00
f99c2c1a22
reduced --no-big-tests even more
...
to pass tests on qemu-aarch64
2017-06-21 23:35:58 -07:00
04094f37e9
fixed offset in this case os that it always goes past src start
2017-06-21 18:47:40 -07:00
49f8459d11
fixed minor cast warning
2017-06-21 18:43:39 -07:00
0950b3159a
more meaningful names for count variables
2017-06-21 18:30:27 -07:00
0b6eedeace
malloc samples instead of static allocation
2017-06-21 18:24:19 -07:00
4d3bdcf130
reduced CLevelMax for --no-big-tests
2017-06-21 18:20:02 -07:00
52a13abf3f
removed comment regarding windowSize requirement
2017-06-21 18:13:52 -07:00
32e36c2d9a
removed requirement that offset be less than windowSize
2017-06-21 18:00:47 -07:00
bfc2f00080
--no-big-tests for zstreamtest
...
Apply --no-big-tests for tsan tests
2017-06-21 17:57:14 -07:00
eeff2d0e3c
added const x2
2017-06-21 17:48:37 -07:00
04253e21bb
added const
2017-06-21 17:48:01 -07:00
2ceaec1aad
got rid of unnecessary casting to void*
2017-06-21 17:46:37 -07:00
ccae9ec807
added comment addressing use of bits in frame header descriptor
2017-06-21 17:43:21 -07:00
dce789281b
fixed : decompression of skippable frames in streaming mode
2017-06-21 15:53:42 -07:00
c16748b137
added test cases for new command --list
2017-06-21 12:09:53 -07:00
7bd1a2900e
added ZSTD_dictMode_e to control dictionary loading mode
2017-06-21 11:50:33 -07:00
db3606e249
added test for files with no frame content size
2017-06-20 17:43:36 -07:00
acf0df678c
fixed strange spacing
2017-06-20 15:41:43 -07:00
69bc4fab25
free the context object once done with decompression
2017-06-20 15:37:14 -07:00
ceb4f65620
allocate statically so we don't have to deal with freeing
2017-06-20 15:31:54 -07:00
a73c2a444a
added tests with null files, changed condition to check that the file is more than 0 bytes
2017-06-20 14:33:08 -07:00
58c19b4202
spacing matters for test scripts
2017-06-20 14:14:53 -07:00
ff8f83bd47
fixed fuzzer test
2017-06-20 12:17:32 -07:00
b07d0af2ba
added additional tests for error detection
2017-06-20 11:54:44 -07:00
43ffcdde40
const dictOffset
2017-06-20 11:27:39 -07:00
1e0d125e76
const value
2017-06-20 11:15:26 -07:00
0034129270
made changes to accommodate single file use case
2017-06-20 11:07:38 -07:00
6a99de2dd8
change test to list info on .zst files
2017-06-20 10:45:06 -07:00
36e14b1963
added description to advanced usage, changed to use-dict
2017-06-20 10:34:34 -07:00
15102069a1
removed testing code from generation code
2017-06-20 10:25:16 -07:00
ff5589c091
added comments for clarity
2017-06-20 10:17:38 -07:00
551fb7ca05
changed error condition
2017-06-20 10:14:03 -07:00
b8237c6c66
refactoring for readability
2017-06-20 10:12:13 -07:00
6f17de2132
changed computation for readability
2017-06-20 09:50:28 -07:00
4a4f9ba262
const value that doesn't change
2017-06-19 17:46:21 -07:00
3a6856edca
changed scope to avoid errors
2017-06-19 17:43:16 -07:00
7d2b5613b4
adhere to min dict size
2017-06-19 17:37:14 -07:00
ea2af1ffc4
changed dictionary to stay reasonable size during test
2017-06-19 17:34:50 -07:00
fdbb07283a
added test to runTestMode
2017-06-19 17:29:15 -07:00
bdec7b8ce7
fixed for loop declaration, fixed test case
2017-06-19 17:18:43 -07:00
23098b53e8
wrote a test for dictionary corpus
2017-06-19 17:13:00 -07:00
7a26390989
changing for loop declaration
2017-06-19 16:39:57 -07:00
147ef05f12
for loop declaration not allowed
2017-06-19 16:38:51 -07:00
791352ad0e
scope change
2017-06-19 16:32:38 -07:00
365e265a8e
changed scope so that warning disappears
2017-06-19 16:28:40 -07:00
f4f928595b
changed scope to get rid of mixed code/variables warning
2017-06-19 16:27:05 -07:00
7ea025cc7d
added in struct to keep functions more orderly
2017-06-19 16:23:00 -07:00
b91b810c64
made some small changes to dictInfo struct and initDictInfo
2017-06-19 16:05:05 -07:00
1ee4f6b36c
cleaning up code
2017-06-19 15:55:01 -07:00
2a52d72712
added comment to anotate decompressed error code
2017-06-19 14:21:35 -07:00
73ebb646d8
small stylistic changes
2017-06-19 14:20:35 -07:00
34f4e5cab2
changed if statement styling
2017-06-19 14:07:56 -07:00
1fc1a35dc4
added calls to free to match calls to malloc, made some stylistic changes, added init function for new struct
2017-06-19 14:02:27 -07:00
8f4fa43fe8
got files to decompress
2017-06-19 13:21:50 -07:00
688952062b
minor declaration statement warning fix
2017-06-19 11:53:42 -07:00
d7a3bffba9
new api : setting compression parameters is refused if a dictionary is already loaded
2017-06-19 11:53:01 -07:00
da4ddc1f76
fixed decompression so that it uses compressed version
2017-06-19 11:40:45 -07:00
33467add60
made changes so that HUF_writeCTable no longer fails
2017-06-19 11:27:50 -07:00
ed1d039127
newapi fuzzer tests : random flush orders in main loop
2017-06-19 11:07:33 -07:00
09ac51b56b
small style changes
2017-06-19 09:24:38 -07:00
f5deae8a67
new api fuzzer tests : frame parameters are randonly set
2017-06-18 23:41:38 -07:00
ea3630a889
new api fuzzer set : dictionary is randomly set
2017-06-18 23:31:55 -07:00
8dee0ec99e
new api fuzzer tests : compression parameters are randomly set
2017-06-18 23:25:15 -07:00
bd05b19713
removed macro CLAMP from decodecorpus
...
never used,
and would duplicate with CLAMP from zstd_compress.c
2017-06-16 18:16:27 -07:00
01743a36e7
fuzzer tests for new API
2017-06-16 17:56:41 -07:00
2f30433cd7
fixing general style
2017-06-15 18:06:32 -07:00
ddd1ab710c
added tests in playTest.sh
2017-06-15 16:53:03 -07:00
623b8fce85
add print statements for debugging and change malloc with updated size
2017-06-15 11:37:24 -07:00
52934fbfc8
added print statements for debugging, fixed first segfault
2017-06-15 09:47:29 -07:00
03a208e27c
changed samples to geometric distribution so that it is huffman compressible
2017-06-14 18:06:16 -07:00
d93207a79f
changed dictionary from using fixed amount of bytes for the header / entropy tables
2017-06-14 17:23:56 -07:00
664ed05ff6
changed randomly generated samples to randomly selected from the dictionary content
2017-06-14 16:42:51 -07:00
3b0e24c06b
added error message to catch case where size not given
2017-06-14 12:04:22 -07:00
aa95bc0a5f
code refactoring and adding some print statements for debugging
2017-06-14 10:54:47 -07:00
07cfc975bb
debugging error with finalizeDictionary() not writing dictionary properly
2017-06-13 17:27:02 -07:00
8dd621f788
changed makefile to include zdict functions, wrote out dictionary to path
2017-06-13 14:19:35 -07:00
f35f252e36
added code to generate dictionary using finalizeDictionary
2017-06-13 11:54:43 -07:00
37e1b1488b
removed previous double free crash, used new crash function
2017-06-09 13:57:18 -07:00
be9b0ae627
added error function for different compilation modes
2017-06-09 13:41:34 -07:00
11c3987baf
added code to extend the offset when a dictionary is detected
2017-06-09 09:48:56 -07:00
125ed59968
made sure dictionary ID was being written in the frame header of each file
2017-06-08 17:27:15 -07:00
233ee5334e
set the lower bits of frame header descriptor if dictionary is needed
2017-06-08 17:11:33 -07:00
c2d909e396
added code for generating dictionary/test files randomly. Still need to make sure dictionary ID matches
2017-06-08 17:06:30 -07:00
8c910d2097
updated ZSTDMT streaming API
...
ZSTDMT streaming API is now similar
and has same capabilites as single-thread streaming API.
It makes it easier to blend them together.
2017-06-03 01:15:02 -07:00
cdf7e82222
Added ZSTD_initStaticCDict()
2017-05-25 18:05:49 -07:00
57827f906f
added ZSTD_initStaticDDict()
2017-05-25 15:44:06 -07:00
25989e361c
updated ZSTD_estimate?DictSize() to pass parameter byReference
...
resulting ?Dict object is smaller when created byReference.
Seems better than a documentation note.
2017-05-25 15:07:37 -07:00
0fdc71c3dc
added ZSTD_initStaticDCtx()
2017-05-24 17:41:41 -07:00
c7fe262dc9
added ZSTD_initStaticCCtx()
...
makes it possible to statically or externally allocate CCtx.
static CCtx will only use provided memory area,
it will never resize nor malloc.
2017-05-23 13:20:41 -07:00
60a557e7fd
fixing symbol.c test
...
I believe it would be better to rely on fuzzer tests compiled with dll
2017-05-19 11:27:43 -07:00
fa3671eac7
changed ZSTD_BLOCKSIZE_ABSOLUTEMAX into ZSTD_BLOCKSIZE_MAX
...
Also :
change ZSTD_getBlockSizeMax() into ZSTD_getBlockSize()
created ZSTD_BLOCKSIZELOG_MAX
2017-05-19 10:51:30 -07:00
009d604e00
ZSTD_compress_generic() supports multiple successive frames
...
also : clarified streaming API implementation
2017-05-19 10:17:59 -07:00
0bdb575c31
Merge branch 'dev' into advancedAPI2
2017-05-16 16:32:29 -07:00
2d4d31c18a
removed gcc compilation flag -Wbad-function-cast
...
It makes it more difficult to directly cast the result of a function,
requiring to store the result in an intermediate variable.
It does not necessarily help readability,
and this restriction can be difficult to overcome in some constructions,
like some macros.
also : fixed minor Visual conversion warnings in datagencli.c
2017-05-16 11:34:38 -07:00
83d0c764dc
added several compilation flags
2017-05-15 17:15:46 -07:00
a5ffe3d370
pushed enum values for strategy by one (ZSTD_fast==1)
...
this makes it possible to use `0` to mean:
"do not change strategy"
2017-05-12 16:29:19 -07:00
5f2c7213c7
Merge remote-tracking branch 'upstream/dev' into btopt
...
* upstream/dev: (305 commits)
added test for ZSTD_estimateCStreamSize()
changed variable name, for clarity
fixed ZSTD_estimateCStreamSize()
shortened ZSTD_createCStream_Advanced()
fixed symbols test
added ZSTD_estimateDStreamSize()
changed name frameParams into frameHeader
regroup memory usage function declarations
separated ZSTD_estimateCStreamSize() from ZSTD_estimateCCtxSize()
bumped version number
added ZSTD_estimateCDictSize() and ZSTD_estimateDDictSize()
Updated ZSTD_freeCCtx()
updated ZSTD_estimateCCtxSize()
Updated ZSTD_sizeof_CCtx()
merged CCtx and CStream as a single same object
cli : -d and -t do not stop after a failed decompression
added dev branch CircleCI badge
added dev branch Appveyor badge
keep dev branch status only
creates a binary archive without the `programs` directory
...
2017-05-10 16:49:58 -07:00
30ab64e21d
added test for ZSTD_estimateCStreamSize()
2017-05-10 11:30:19 -07:00
461a9cc7c6
fixed symbols test
2017-05-09 16:53:09 -07:00
f16f4497ca
added ZSTD_estimateDStreamSize()
2017-05-09 16:18:17 -07:00
542c9dfcf8
changed name frameParams into frameHeader
...
ZSTD_frameParams => ZSTD_frameHeader
ZSTD_getFrameParams() -> ZSTD_getFrameHeader()
The new naming is more distinctive from ZSTD_frameParameters,
which is used during compression.
ZSTD_frameHeader is clearer in its intention to described frame header content.
It also implies we are decoding a ZSTD frame, hence we are at decoding stage.
2017-05-09 15:46:07 -07:00
fa8dadb294
separated ZSTD_estimateCStreamSize() from ZSTD_estimateCCtxSize()
...
for clarity
2017-05-08 18:24:16 -07:00
a1d6704d7f
added ZSTD_estimateCDictSize() and ZSTD_estimateDDictSize()
...
it complements ZSTD_estimateCCtxSize()
for the special case of ZSTD_initCStream_usingDict()
2017-05-08 17:51:49 -07:00
fc5145955a
updated ZSTD_estimateCCtxSize()
...
added a parameter streaming,
to estimate memory allocation size
when the CCtx is used for streaming (CStream).
Note : this function is not able to estimate
memory cost of a potential internal CDict
which can only happen when starting with ZSTD_initCStream_usingDict()
2017-05-08 17:07:59 -07:00
0be6fd3429
merged CCtx and CStream as a single same object
...
To be changed : ZSTD_sizeof_CCtx(), ZSTD_estimateCCtxSize()
2017-05-08 16:08:01 -07:00
01a1abfdb5
cli : -d and -t do not stop after a failed decompression
...
The problematic srcfile will be named on console/log,
but decompression/test will continue onto next file in the list.
2017-05-05 19:15:24 -07:00
f376d47c11
[CLI] Switch dictionary builder on CLI to cover
2017-05-02 11:18:27 -07:00
a51cab6e68
Merge pull request #678 from facebook/apiChange
...
Breaking API Change around CDict
2017-04-28 10:02:45 -07:00
36e79a1a1f
tests : -g is part of DEBUGFLAGS
2017-04-27 17:29:40 -07:00
c6915429f2
shortened Appveyor release tests
...
fuzzer supports time suffix `s` for "seconds"
2017-04-27 16:24:53 -07:00
Stella Lau
Nick Terrell
Yann Collet
Yann Collet
Paul Cruz
cyan4973