facebook
/
zstd
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,197 Commits
2 Branches
0 Tags
28 MiB
Commit Graph

1439 Commits (0953645837d0f5c22aa3bef713de2c2e13ea1bc5)

Author SHA1 Message Date
Yann Collet d3f1a9b5bd fix partial-build test 
sometimes, the scope difference is solely determined by the list of source files,
not by the flags.
 2020-10-22 21:36:09 -07:00
Yann Collet 91a8cb9559 fix DEBUGLEVEL redefinition from tests/ 2020-10-22 00:20:40 -07:00
Yann Collet 494f7169ed fix directory creation for Windows' libzstd 2020-10-22 00:15:31 -07:00
Yann Collet ca75da8fa3 fix test 
DEBUGLEVEL redefinition
 2020-10-21 23:51:13 -07:00
Nick Terrell d6dae2000b
Merge pull request #2365 from senhuang42/move_opt_parser_test_to_long_tests 
Move ldm + opt parser no regression test to long tests
 2020-10-20 11:34:36 -04:00
senhuang42 81a2c02d8f Move ldm no regression test to fuzzer longtests 2020-10-19 15:28:46 -04:00
senhuang42 42d037bdba Add libregression build target, also fix make clean and .gitignore 2020-10-15 10:34:50 -04:00
Yann Collet f5d5cd3b40
Merge pull request #2341 from senhuang42/ldm_optimized_for_opt_parser 
Integrate long distance matches into optimal parser
 2020-10-13 13:09:07 -07:00
Nick Terrell d5c688e8ae Fix ZSTD_adjustCParams_internal() to handle dictionary logic 
Pass in the `ZSTD_cParamMode_e` to select how we define our cparams.
Based on the mode we either take the `dictSize` into account or we set
it to `0`. See the documentation for `ZSTD_cParamMode_e`.

Some of the modes currently share the same behavior. But they have
distinct modes because they are drastically different cases. E.g.
compression + reprocessing the dictionary and creating a cdict.

Additionally, when downsizing the hashLog and chainLog take the
(adjusted) dictionary size into account, since the size of the
dictionary gets added onto the window size.

Adds a simple test to ensure that we aren't downsizing too far.
 2020-10-12 12:50:04 -07:00
Nick Terrell 7083f79008 [bug] Fix dictContentType when reprocessing cdict 
Conditions to trigger:
* CDict is loaded as raw content.
* CDict starts with the zstd dictionary magic number.
* The CDict is reprocessed (not attached or copied).
* The new API is used (streaming or `ZSTD_compress2()`).

Bug: The dictionary is loaded as a zstd dictionary, not a raw content
dictionary, because the dict content type is set to `ZSTD_dct_auto`.

Fix: Pass in the dictionary content type from cdict creation to the call
to `ZSTD_compress_insertDictionary()`.

Test: Added a test case that exposes the bug, and fixed the raw
content tests to not modify the `dictBuffer`, which makes all future
tests with the `dictBuffer` raw content, which doesn't seem intentional.
 2020-10-12 12:46:10 -07:00
Yann Collet b951ad20a2
Merge pull request #2329 from senhuang42/prevent_summary_updates_when_using_stdout 
Prevent summary updates when using stdout
 2020-10-09 01:01:36 -07:00
Yann Collet c3ee284ca2
Merge pull request #2319 from facebook/fullbench_stream2 
update fullbench for compressStream2()
 2020-10-09 00:40:59 -07:00
senhuang42 e96ea5d147 Fix static analyze fuzzer.c error 2020-10-07 13:56:25 -04:00
senhuang42 b8bfc4e63d Add cSize regression test to fuzzer.c 2020-10-07 13:56:25 -04:00
senhuang42 429dec4f42 Add DEBUGLOG() calls in ldm helpers 2020-10-07 13:56:25 -04:00
senhuang42 cfd2aec1b7 Add unit tests into playTests.sh 2020-10-07 13:56:25 -04:00
senhuang42 7259b258d1 Add callsites to zstdcli.c and tests to playTests.sh 2020-10-07 13:47:38 -04:00
Nick Terrell 0057c4acf7
Merge pull request #2333 from terrelln/stable-dst 
Reset all decompression parameters in ZSTD_DCtx_reset()
 2020-10-01 18:56:11 -07:00
Nick Terrell 2e7d174130 Reset all decompression parameters in ZSTD_DCtx_reset() 
* Reset all decompression parameters in `ZSTD_DCtx_reset()` when
  resetting parameters.
* Add a test case.
 2020-10-01 14:19:21 -07:00
Yann Collet 83461ce963
Merge pull request #2322 from senhuang42/guard_against_stdin_for_warning_prompts 
Don't let warning messages consume input from stdin
 2020-09-30 08:26:50 -07:00
senhuang42 9f7212a48b Update unit tests 2020-09-24 16:44:33 -04:00
Yann Collet c6c0a57c53
Merge pull request #2315 from senhuang42/allow_zstd_suffix 
Support .zstd suffix only for decompression
 2020-09-24 09:44:48 -07:00
senhuang42 21cd640b93 Add unit tests to guard against bad stdin 2020-09-22 14:55:41 -04:00
senhuang42 7aa3da1cd7 Use IS_CONSOLE macro to detect that we're indeed using a console 2020-09-22 14:15:52 -04:00
Nick Terrell 973f2adeec [tests] Don't write to stdout 2020-09-22 00:40:27 -07:00
Yann Collet 5618e000bd update fullbench for compressStream2() 
makes it possible to measure scenarios such as #2314
 2020-09-21 07:19:20 -07:00
Felix Handte 200c960f1d
Merge pull request #2311 from felixhandte/ddss-fix-cparam-derivation 
Fix Compression Parameter Derivation Bugs Introduced by DDSS Changes
 2020-09-18 14:02:14 -04:00
senhuang42 07034952df Add -f to .zstd decompression CLI test 2020-09-18 13:01:45 -04:00
senhuang42 6b6cc80196 Support .zstd suffix only for decompression 2020-09-18 12:49:51 -04:00
W. Felix Handte 9398acb245 Move Last Two Long Tests in fuzzer.c into Separate --long-tests Section 2020-09-17 13:31:10 -04:00
W. Felix Handte f23a321781 Update Regression Test Results 2020-09-17 12:23:05 -04:00
Yann Collet e583e0be8c
Merge pull request #2299 from senhuang42/env_var_num_threads 
Allow environment variable to specify number of threads for compression
 2020-09-14 14:04:19 -07:00
Yann Collet dec1a78d3e minor fix casting for Visual 2020-09-14 11:46:23 -07:00
Yann Collet c91a0855f8 check endDirective in ZSTD_compressStream2() 
fix #2297
also :
- `assert()` `endDirective` in `ZSTD_compressStream_internal()`, for debug mode
- add relevant tests
 2020-09-14 10:56:08 -07:00
W. Felix Handte d6246d4a0f Print More During Fuzzer Test to Avoid CI Killing it Due to Timeout 
This is kind of hacky. And maybe this test doesn't need to be permanently as
exhaustive as it is now. But while we're actively developing the DDSS, we
should ensure it's compatible across many different modes.
 2020-09-10 23:35:42 -04:00
W. Felix Handte 6d3f816b3e Test Fewer Dictionary Sizes 2020-09-10 22:30:52 -04:00
W. Felix Handte b6df3fd438 Fix Debug Logging in 32-bit Build 2020-09-10 22:10:02 -04:00
W. Felix Handte 2cc2b40a1b Test DDSS A Little More Thoroughly 2020-09-10 22:10:02 -04:00
W. Felix Handte b81f3a37f9 Easy: Fix Test 2020-09-10 18:51:52 -04:00
W. Felix Handte 2cf6cfc55f Add Fuzzer Test for the Various Dict Attachment Strategies 2020-09-10 18:51:52 -04:00
Nick Terrell a90779397a [lib] Reduce zstd stack usage by 1KB 2020-09-09 14:35:39 -07:00
senhuang42 a71963c7b8 nbThreads instead of numThreads 2020-09-09 12:40:00 -04:00
senhuang42 0a170b20a8 Add ZSTD_NUMTHREADS tests to playTests.sh 2020-09-08 10:34:50 -04:00
senhuang42 3aec385a10 Fix merge conflicts 2020-08-26 15:43:38 -04:00
Yann Collet a8c66881e5
Merge pull request #2283 from senhuang42/progress_bars_for_multiple_files 
Refreshing progress bar for processing multiple files
 2020-08-26 11:54:50 -07:00
Nick Terrell cf83aceaf3
Merge pull request #2282 from terrelln/ncount-fix 
[bug] Fix FSE_readNCount()
 2020-08-26 10:31:07 -07:00
senhuang42 a73e131f10 Adjust playTests.sh refuse overwrite test to include -q 2020-08-26 11:40:05 -04:00
Nick Terrell ae163015b1 [fuzz] Fix stream_decompress timeouts 2020-08-25 17:13:09 -07:00
Nick Terrell 49eeb2d1fc [fuzz] Disable superblock expansion test 2020-08-25 17:13:06 -07:00
Nick Terrell 4193638996 [bug] Fix FSE_readNCount() 
* Fix bug introduced in PR #2271
* Fix long-standing bug that is impossible to trigger inside of zstd
* Add a fuzzer that makes sure the normalized count always round trips
  correctly
 2020-08-25 15:42:41 -07:00