facebook/zstd
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,953 Commits 2 Branches 0 Tags
Commit Graph

1553 Commits

Author SHA1 Message Date
Yann Collet
c044345f8f Merge branch 'mingw' into minclevel 2018-09-21 14:56:57 -07:00
Yann Collet
de6c75e4e5
Merge pull request #1318 from felixhandte/shadow-dict-matches 
Don't Search Dictionary Context When Working Context Search Resulted in Mismatch
 2018-09-21 12:15:33 -07:00
Yann Collet
a54c86cfc6 defined a minimum negative level 
which can be probed using new function ZSTD_minCLevel().

Also : redefined ZSTD_TARGETLENGTH_MIN/MAX for consistency

used the opportunity to bump version number to v1.3.6
 2018-09-20 16:52:03 -07:00
Yann Collet
7992942d66 fixed complex tsan issue 
when job->consumed == job->src.size , compression job is presumed completed,
so it must be the very last action done in worker thread.
 2018-09-20 13:47:31 -07:00
Yann Collet
6b07a66aec fixed minor reporting discrepancy in MT mode 2018-09-19 16:30:55 -07:00
Yann Collet
ca02ebee07 removed static variables 
so that --adapt can work on multiple input files too
 2018-09-19 15:25:50 -07:00
Yann Collet
2f78228f65 Merge branch 'dev' into adapt 2018-09-19 12:43:42 -07:00
ko-zu
18b4a1da61 Fix clang build 
Fix dixygen comment
Fix clang binary path
 2018-09-16 10:27:02 +09:00
W. Felix Handte
b76c888497 ZSTD_dfast: Don't Search Dict Context When Mismatch Was Found 2018-09-14 15:24:25 -07:00
W. Felix Handte
b048af5999 ZSTD_fast: Don't Search Dict Context When Mismatch Was Found 2018-09-14 15:23:35 -07:00
Yann Collet
31ebb26945
Merge pull request #1301 from terrelln/lit-size 
[zstd] Fix seqStore growth
 2018-08-28 17:10:25 -07:00
Nick Terrell
5e580de6da [zstd] Fix seqStore growth 
We could undersize the literals buffer by up to 11 bytes,
due to a combination of 2 bugs:
* The literals buffer didn't have `WILDCOPY_OVERLENGTH` extra
  space, like it is supposed to.
* We didn't check the literals buffer size in `ZSTD_sufficientBuff()`.
 2018-08-28 13:24:44 -07:00
Yann Collet
b37a0a6bde
Merge pull request #1298 from facebook/bench 
Refactored bench.c
 2018-08-28 12:25:02 -07:00
Yann Collet
af23d39eb8
Merge pull request #1297 from felixhandte/check-offset-table 
Fix Missing Offset Table Check
 2018-08-24 17:36:44 -07:00
W. Felix Handte
37f17ee237 Mark Repeated Offset Table as Needing Check 2018-08-24 14:33:34 -07:00
Nick Terrell
e34e917655 Fix compiler warning 2018-08-23 17:48:06 -07:00
Nick Terrell
924944e471 [zstd] Reuse the ZSTD_CCtx more often with small data. 2018-08-23 17:48:06 -07:00
Yann Collet
2e45badff4 refactored bench.c 
for clarity and safety, especially at interface level
 2018-08-23 14:21:18 -07:00
Yann Collet
c71c4f23d7 fix "unused parameter" in single-thread mode 
within newly added ZSD_toFlushNow()
 2018-08-20 11:40:10 -07:00
Yann Collet
105677c6db created ZSTDMT_toFlushNow() 
tells in a non-blocking way if there is something ready to flush right now.
only works with multi-threading for the time being.

Useful to know if flush speed will be limited by lack of production.
 2018-08-17 18:11:54 -07:00
Yann Collet
1515f0bb0d fixed more issues detected by recent version of scan-build 
test run on Linux
 2018-08-16 15:20:25 -07:00
Yann Collet
3692c31598 Merge branch 'dev' into scanbuild 2018-08-15 13:50:49 -07:00
Yann Collet
6e66bbf5dd fixed several minor issues detected by scan-build 
only notable one :
writeNCount() resists better vs invalid distributions
(though it should never happen within zstd anyway)
 2018-08-14 16:55:35 -07:00
Yann Collet
3e4617ef54 frameProgression reports nbActiveWorkers and output flushed 2018-08-14 11:49:25 -07:00
Yann Collet
e7a49c6683 introduced command --adapt 2018-08-11 20:48:06 -07:00
Yann Collet
2dd76037be zstd cli can increase level when input is too slow 2018-08-09 15:51:30 -07:00
Yann Collet
79a35ac20d minor code comments improvements 2018-08-09 15:16:31 -07:00
W. Felix Handte
2ca7c69167 Fix CDict Attachment to Handle CDicts with Non-Zero Starts 
CDicts were previously guaranteed to be generated with `lowLimit=dictLimit=0`.
This is no longer true, and so the old length and index calculations are no
longer valid. This diff fixes them to handle non-zero start indices in CDicts.
 2018-08-07 18:14:14 -07:00
Yann Collet
5808027abf Merge branch 'dev' into fix1241 2018-08-03 16:08:33 -07:00
Nick Terrell
dc5a67cb7b Disallow tableLog == srcLog 2018-08-02 11:12:17 -07:00
cyan4973
aade1e5904 Merge branch 'dev' into fix1241 2018-07-30 16:30:35 +02:00
Nick Terrell
9889bca530 [FSE] Fix division by zero 
When the primary normalization method fails, and
`(1 << tableLog) == (maxSymbolValue + 1)`, and every symbol gets assigned
normalized weight 1 or -1 in the first loop, then the next division can
raise `SIGFPE`.
 2018-07-27 17:30:03 -07:00
Yann Collet
6e490a2f09
Merge pull request #1237 from terrelln/init-cstream-adv 
Set requestedParams in ZSTD_initCStream*()
 2018-07-18 16:33:30 +02:00
cyan4973
9597b438e9 fix #1241 
Ensure that first input position is valid for a match
even during first usage of context
by starting reference at 1
(avoiding the problematic 0).
 2018-07-17 18:52:57 +02:00
cyan4973
53e1f0504e zstdmt debug traces compatibles with mingw 
since mingw does not have `sys/times.h`,
remove this path when detecting mingw compilation.
 2018-07-17 14:39:44 +02:00
Nick Terrell
6d222c437c Set requestedParams in ZSTD_initCStream*() 
The correct parameters are used once, but once `ZSTD_resetCStream()` is
called the default parameters (level 3) are used. Fix this by setting
`requestedParams` in the `ZSTD_initCStream*()` functions.

The added tests both fail before this patch and pass after.
 2018-07-12 18:35:55 -07:00
Yann Collet
4489daec09 slightly adjusted default-distribution threshold 
depending on strategy.
fast favors faster compression and decompression speeds.
 2018-06-26 20:10:45 -07:00
Nick Terrell
b426bcc097
[zstdmt] Fix jobsize bugs (#1205) 
[zstdmt] Fix jobsize bugs

* `ZSTDMT_serialState_reset()` should use `targetSectionSize`, not `jobSize` when sizing the seqstore.
  Add an assert that checks that we sized the seqstore using the right job size.
* `ZSTDMT_compressionJob()` should check if `rawSeqStore.seq == NULL`.
* `ZSTDMT_initCStream_internal()` should not adjust `mtctx->params.jobSize` (clamping to MIN/MAX is okay).
 2018-06-25 15:21:08 -07:00
Yann Collet
3b53bfe4f3
Merge pull request #1200 from felixhandte/zstd-attach-dict-pref 
Add CCtx Param Controlling Dict Attachment Behavior
 2018-06-25 12:42:31 -07:00
Yann Collet
3934e010a2
Merge pull request #1197 from facebook/poolResize 
Thread Pool resize
 2018-06-22 14:20:07 -07:00
Yann Collet
fbd5dfc1b1 changed POOL_resize() return type to int 
return is now just en error code.
This guarantee that `ctx` remains valid after POOL_resize().
Gets rid of internal POOL_free() operation.
 2018-06-22 12:14:59 -07:00
Yann Collet
1d5648ca10
Merge pull request #1196 from felixhandte/zstd-btopt-in-place-dict 
ZSTD_btopt: Support Searching the Dictionary Context In-Place
 2018-06-22 11:53:23 -07:00
Yann Collet
f6242d30b7
Merge pull request #1202 from facebook/barelyCompressible 
Increase threshold detection of poorly compressible data
 2018-06-22 11:52:52 -07:00
Yann Collet
698fd00afb huf: increase threshold detection of poorly compressible data 2018-06-21 18:32:38 -07:00
W. Felix Handte
01bb1c1016 Add CCtx Param Controlling Dict Attachment Behavior 2018-06-21 17:29:25 -04:00
W. Felix Handte
3e91dc4d6a Add Repcode Bounds Check 2018-06-21 15:54:41 -04:00
W. Felix Handte
5bd3d4b7d2 Add Debug Log Statement 2018-06-21 15:54:07 -04:00
W. Felix Handte
3caba150c6 Fix dmsBtLow Test 2018-06-21 15:53:40 -04:00
W. Felix Handte
5da9bbc38e Conceivably Dedup ZSTD_noDict and ZSTD_dictMatchState _insertBt1 Impls 
By reverting to the bool extDict flag, we call ZSTD_insertBt1 with the same
const args in both non-extDict dictModes.
 2018-06-21 11:20:01 -04:00
W. Felix Handte
5d81f71e83 Consistency in Guarding DMS-Only Variable Initializations 2018-06-20 16:54:53 -04:00