W. Felix Handte
a232b3bb7c
Bump Split Log Support to ZSTD_dfast
2018-09-28 17:12:54 -07:00
W. Felix Handte
fe96e98f81
Support a Separate Hash Log in ZSTD_fast
2018-09-28 17:12:54 -07:00
W. Felix Handte
bc880ebe8f
Stop Passing in hashLog and stepSize to ZSTD_compressBlock_fast_generic
2018-09-28 17:12:54 -07:00
W. Felix Handte
b3107c7799
Temporary Commit to Retain Requested Hash and Chain Logs During Dict Attach
2018-09-28 17:12:54 -07:00
W. Felix Handte
34e0193129
Allow Setting Hash and Chain Logs on Contexts with Attached CDict
2018-09-28 17:12:54 -07:00
W. Felix Handte
eae8232f50
For Supported Strategies, Attach Dict Even When Params Don't Match
2018-09-28 17:12:54 -07:00
W. Felix Handte
01ff945eae
Split Attach and Copy Reset Strategies into Separate Implementation Functions
2018-09-28 17:12:54 -07:00
W. Felix Handte
a6d6bbeae1
Pull Attachment Decision into Separate Function
2018-09-28 17:12:54 -07:00
W. Felix Handte
b7fba599ae
And Then Avoid the Unused Parameter Warning
2018-09-28 17:12:54 -07:00
W. Felix Handte
1f188ae655
Move Asserts into Function to Avoid Unused Function Warning
2018-09-28 17:12:54 -07:00
W. Felix Handte
7212b5e5c2
Move Match State CParams Setting into resetCCtx and continueCCtx
2018-09-28 17:12:54 -07:00
W. Felix Handte
01e34d365b
Strengthen Assertion to Assert Equality
2018-09-28 17:12:53 -07:00
W. Felix Handte
50cc1cf4d5
Remove CParams Arg from ZSTD_ldm_blockCompress
2018-09-28 17:12:53 -07:00
W. Felix Handte
14764de49f
Stop Separately Passing CParams in ZSTD_lazy Internal Functions
2018-09-28 17:12:53 -07:00
W. Felix Handte
97149f22c3
Stop Separately Passing CParams in ZSTD_opt Internal Functions
2018-09-28 17:10:42 -07:00
W. Felix Handte
dcdf437fed
Also Remove CParams from Table Filling Functions' Args
2018-09-28 17:10:42 -07:00
W. Felix Handte
3483f89101
Also Assert Equivalency When Filling MatchState with Prefix
2018-09-28 17:10:42 -07:00
W. Felix Handte
6cb2454646
Remove CParams from Block Compressor Functions' Args
2018-09-28 17:10:42 -07:00
W. Felix Handte
03103269de
Assert ctx and ms cparams Equivalency
2018-09-28 17:10:42 -07:00
W. Felix Handte
4e3ecee9ed
Remove cParams from CDict
2018-09-28 17:10:42 -07:00
W. Felix Handte
76ef87ed9d
Add ZSTD_compressionParameters to ZSTD_matchState_t
2018-09-28 17:10:42 -07:00
Nick Terrell
a180ea07c4
Restore ZSTD_noCompressBlock() for clarity
2018-09-27 16:06:02 -07:00
Nick Terrell
f2d6db45cd
[zstd] Add -Wmissing-prototypes
2018-09-27 15:24:48 -07:00
Yann Collet
2a5cd8535a
Merge pull request #1342 from facebook/fixcatyd
...
fix : huge (>4GB) chain of blocks
2018-09-27 10:20:14 -07:00
Yann Collet
404a7bfed0
moved again overflow correction
...
cannot work from within ZSTD_compressBlock()
2018-09-26 18:06:53 -07:00
Yann Collet
0e2dbac18a
changed overflow correction place
...
keep one in compress_frameChunk(),
so that it's tested at every loop
in case some user simply some large mulit-GB input in a single invocation.
Add one in ZSTD_compressBlock(),
since compressBlock() explicitly skips frameChunk().
2018-09-26 15:35:38 -07:00
Yann Collet
f98c69d77c
fix : huge (>4GB) stream of blocks
...
experimental function ZSTD_compressBlock() is designed for very small data in mind,
for situation where saving the ~12 bytes of frame header can actually make a difference.
Some systems though may have to deal with small and large data entangled.
If it's larger than a block (> 128KB), compressBlock() cannot compress them in one round.
That's why it's possible to compress in multiple rounds.
This is a chain of compressed blocks.
Some users push this capability to the limit, encoding gigantic chain of blocks.
On crossing the 4GB limit, some internal overflow occurs.
This fix moves the overflow correction mechanism higher in the call chain,
so that it's applied also to gigantic chains of blocks.
Added a test case in fuzzer.c, which crashes before the fix, and pass now.
2018-09-26 14:24:28 -07:00
Yann Collet
04f47bbdd2
Merge branch 'dev' into adapt
2018-09-24 16:56:45 -07:00
Yann Collet
c484345a82
Merge branch 'mingw' into adapt
2018-09-21 16:00:46 -07:00
Yann Collet
bfff4f4809
ensure all writes to job->cSize are mutex protected
...
even when reporting errors,
using a macro for code brevity, as suggested by @terrelln,
2018-09-21 16:00:39 -07:00
Yann Collet
32b7cf1bcf
fixed tautological tests
...
involving ZSTD_TARGETLENGTH_MIN (== 0)
2018-09-21 15:04:43 -07:00
Yann Collet
c044345f8f
Merge branch 'mingw' into minclevel
2018-09-21 14:56:57 -07:00
Yann Collet
de6c75e4e5
Merge pull request #1318 from felixhandte/shadow-dict-matches
...
Don't Search Dictionary Context When Working Context Search Resulted in Mismatch
2018-09-21 12:15:33 -07:00
Yann Collet
a54c86cfc6
defined a minimum negative level
...
which can be probed using new function ZSTD_minCLevel().
Also : redefined ZSTD_TARGETLENGTH_MIN/MAX for consistency
used the opportunity to bump version number to v1.3.6
2018-09-20 16:52:03 -07:00
Yann Collet
7992942d66
fixed complex tsan issue
...
when job->consumed == job->src.size , compression job is presumed completed,
so it must be the very last action done in worker thread.
2018-09-20 13:47:31 -07:00
Yann Collet
6b07a66aec
fixed minor reporting discrepancy in MT mode
2018-09-19 16:30:55 -07:00
Yann Collet
ca02ebee07
removed static variables
...
so that --adapt can work on multiple input files too
2018-09-19 15:25:50 -07:00
Yann Collet
2f78228f65
Merge branch 'dev' into adapt
2018-09-19 12:43:42 -07:00
ko-zu
18b4a1da61
Fix clang build
...
Fix dixygen comment
Fix clang binary path
2018-09-16 10:27:02 +09:00
W. Felix Handte
b76c888497
ZSTD_dfast: Don't Search Dict Context When Mismatch Was Found
2018-09-14 15:24:25 -07:00
W. Felix Handte
b048af5999
ZSTD_fast: Don't Search Dict Context When Mismatch Was Found
2018-09-14 15:23:35 -07:00
Yann Collet
31ebb26945
Merge pull request #1301 from terrelln/lit-size
...
[zstd] Fix seqStore growth
2018-08-28 17:10:25 -07:00
Nick Terrell
5e580de6da
[zstd] Fix seqStore growth
...
We could undersize the literals buffer by up to 11 bytes,
due to a combination of 2 bugs:
* The literals buffer didn't have `WILDCOPY_OVERLENGTH` extra
space, like it is supposed to.
* We didn't check the literals buffer size in `ZSTD_sufficientBuff()`.
2018-08-28 13:24:44 -07:00
Yann Collet
b37a0a6bde
Merge pull request #1298 from facebook/bench
...
Refactored bench.c
2018-08-28 12:25:02 -07:00
Yann Collet
af23d39eb8
Merge pull request #1297 from felixhandte/check-offset-table
...
Fix Missing Offset Table Check
2018-08-24 17:36:44 -07:00
W. Felix Handte
37f17ee237
Mark Repeated Offset Table as Needing Check
2018-08-24 14:33:34 -07:00
Nick Terrell
e34e917655
Fix compiler warning
2018-08-23 17:48:06 -07:00
Nick Terrell
924944e471
[zstd] Reuse the ZSTD_CCtx more often with small data.
2018-08-23 17:48:06 -07:00
Yann Collet
2e45badff4
refactored bench.c
...
for clarity and safety, especially at interface level
2018-08-23 14:21:18 -07:00
Yann Collet
c71c4f23d7
fix "unused parameter" in single-thread mode
...
within newly added ZSD_toFlushNow()
2018-08-20 11:40:10 -07:00
