55b90ef010
Fix unit tests to agree with new changes
2020-11-16 11:36:37 -05:00
3d26615c84
Adjust unit tests to agree with new sequence generation API
2020-11-16 10:49:17 -05:00
2db8441245
Add RLE support
2020-11-16 10:49:17 -05:00
2bbdddf24e
Add test case to roundtrip using ZSTD_getSequences() and ZSTD_compressSequences()
2020-11-16 10:49:16 -05:00
9d936d61d2
Reduce number of memcpy() calls
2020-11-13 19:43:30 -05:00
1a8af0de73
Improve unit test
2020-11-12 11:09:09 -05:00
f62edf0fe9
Merge pull request #2381 from senhuang42/expand_sequence_extraction_api
...
Add enum to define ZSTD_Sequence type and update sequence extraction API
2020-11-06 13:00:31 -05:00
7d1dea070c
Update unit tests
2020-11-06 11:10:37 -05:00
51abd58208
Rename getSequences() to generateSequences()
2020-11-06 10:53:22 -05:00
eac309c71b
Add ZSTD_getDictID_fromCDict function to experimental section
2020-11-04 11:37:37 +00:00
c54a25b666
Revert compressibility change
2020-11-02 11:38:58 -05:00
d4d0346b40
Update name of enum, clarify documentation
2020-11-02 11:38:17 -05:00
9102f30dbf
Update unit test
2020-11-02 11:30:31 -05:00
3327932609
Update ZSTD_getSequences function signature
2020-11-02 10:17:59 -05:00
c37c714ef1
Merge pull request #2376 from senhuang42/clarify_sequence_extraction_api
...
Refine external ZSTD_Sequence API
2020-10-30 15:47:25 -04:00
ff93440fc6
Merge pull request #2375 from senhuang42/ldm_oss_fuzz_testcase
...
Add a test case for LDM + opt parser with small uncompressible block
2020-10-29 09:32:05 -04:00
7198ebb213
Un-mix declarations and code
2020-10-28 18:51:03 -04:00
60a52c29e6
Add check for allocation
2020-10-28 16:22:22 -04:00
599ff58e08
Merge pull request #2339 from terrelln/zstdmt-stability
...
Fix zstdmt stability issues and clean up the zstdmt code
2020-10-27 19:43:13 -07:00
169fc07aa1
Move test to appropriate location
2020-10-27 16:59:43 -04:00
db0b5d7d1e
Add test to fuzzer.c
2020-10-27 16:57:24 -04:00
17b700d78a
Merge pull request #2366 from senhuang42/enable_ldm_by_default
...
Enable LDM by default if window size >= 128MB and strategy uses opt parser
2020-10-27 14:59:28 -04:00
dc448563e9
Add test compatibility with last literals in sequences
2020-10-27 12:35:28 -04:00
81a2c02d8f
Move ldm no regression test to fuzzer longtests
2020-10-19 15:28:46 -04:00
df470e176b
Add unit test for no cctx requested params change
2020-10-19 10:52:41 -04:00
f5d5cd3b40
Merge pull request #2341 from senhuang42/ldm_optimized_for_opt_parser
...
Integrate long distance matches into optimal parser
2020-10-13 13:09:07 -07:00
c51a9e79b9
[zstdmt] Rip out the zstdmt API
...
This commit leaves only the functions used by zstd_compress.c. All other
functions have been removed from the API. The ZSTDMT unit tests in
fuzzer.c and zstreamtest.c have been rewritten to use the ZSTD API. And
the --mt zstreamtest tests have been ripped out.
2020-10-12 12:55:16 -07:00
d5c688e8ae
Fix ZSTD_adjustCParams_internal() to handle dictionary logic
...
Pass in the `ZSTD_cParamMode_e` to select how we define our cparams.
Based on the mode we either take the `dictSize` into account or we set
it to `0`. See the documentation for `ZSTD_cParamMode_e`.
Some of the modes currently share the same behavior. But they have
distinct modes because they are drastically different cases. E.g.
compression + reprocessing the dictionary and creating a cdict.
Additionally, when downsizing the hashLog and chainLog take the
(adjusted) dictionary size into account, since the size of the
dictionary gets added onto the window size.
Adds a simple test to ensure that we aren't downsizing too far.
2020-10-12 12:50:04 -07:00
7083f79008
[bug] Fix dictContentType when reprocessing cdict
...
Conditions to trigger:
* CDict is loaded as raw content.
* CDict starts with the zstd dictionary magic number.
* The CDict is reprocessed (not attached or copied).
* The new API is used (streaming or `ZSTD_compress2()`).
Bug: The dictionary is loaded as a zstd dictionary, not a raw content
dictionary, because the dict content type is set to `ZSTD_dct_auto`.
Fix: Pass in the dictionary content type from cdict creation to the call
to `ZSTD_compress_insertDictionary()`.
Test: Added a test case that exposes the bug, and fixed the raw
content tests to not modify the `dictBuffer`, which makes all future
tests with the `dictBuffer` raw content, which doesn't seem intentional.
2020-10-12 12:46:10 -07:00
e96ea5d147
Fix static analyze fuzzer.c error
2020-10-07 13:56:25 -04:00
b8bfc4e63d
Add cSize regression test to fuzzer.c
2020-10-07 13:56:25 -04:00
2e7d174130
Reset all decompression parameters in ZSTD_DCtx_reset()
...
* Reset all decompression parameters in `ZSTD_DCtx_reset()` when
resetting parameters.
* Add a test case.
2020-10-01 14:19:21 -07:00
9398acb245
Move Last Two Long Tests in fuzzer.c into Separate --long-tests Section
2020-09-17 13:31:10 -04:00
dec1a78d3e
minor fix casting for Visual
2020-09-14 11:46:23 -07:00
c91a0855f8
check endDirective in ZSTD_compressStream2()
...
fix #2297
also :
- `assert()` `endDirective` in `ZSTD_compressStream_internal()`, for debug mode
- add relevant tests
2020-09-14 10:56:08 -07:00
d6246d4a0f
Print More During Fuzzer Test to Avoid CI Killing it Due to Timeout
...
This is kind of hacky. And maybe this test doesn't need to be permanently as
exhaustive as it is now. But while we're actively developing the DDSS, we
should ensure it's compatible across many different modes.
2020-09-10 23:35:42 -04:00
6d3f816b3e
Test Fewer Dictionary Sizes
2020-09-10 22:30:52 -04:00
b6df3fd438
Fix Debug Logging in 32-bit Build
2020-09-10 22:10:02 -04:00
2cc2b40a1b
Test DDSS A Little More Thoroughly
2020-09-10 22:10:02 -04:00
b81f3a37f9
Easy: Fix Test
2020-09-10 18:51:52 -04:00
2cf6cfc55f
Add Fuzzer Test for the Various Dict Attachment Strategies
2020-09-10 18:51:52 -04:00
f82d9865b9
Merge pull request #2278 from senhuang42/ignore_checksum_advanced_param
...
New advanced decompression param to ignore checksums
2020-08-25 12:08:53 -07:00
614e446000
Merge pull request #2271 from terrelln/small-blocks
...
Small block optimizations
2020-08-24 18:54:33 -07:00
a030560d62
Add new DCtx param: validateChecksum and update unit tests
2020-08-24 17:28:00 -04:00
1302f8d676
[fix] Always return dstSize_tooSmall when it is the case
2020-08-24 13:38:13 -07:00
44c54a3e31
Addressing comments: more comments, cleanup, remove extra function, checksum logic
2020-08-24 16:14:19 -04:00
ffaa0df76d
Document change in CLI for --no-check during decompression in --help menu
2020-08-24 09:49:12 -04:00
20eb095882
Added unit test to fuzzer.c, changed definition param name
2020-08-22 13:26:33 -04:00
1b34b15e6b
Adding CLI capability to invoke decompression with no checksum
2020-08-21 17:49:30 -04:00
6a8dbdcd1f
Modify decompression loop to gnore checksums if flag is enabled
2020-08-21 16:46:46 -04:00
senhuang42
Luke Pitt
Nick Terrell
Yann Collet
W. Felix Handte
Yann Collet