31ebb26945
Merge pull request #1301 from terrelln/lit-size
...
[zstd] Fix seqStore growth
2018-08-28 17:10:25 -07:00
5e580de6da
[zstd] Fix seqStore growth
...
We could undersize the literals buffer by up to 11 bytes,
due to a combination of 2 bugs:
* The literals buffer didn't have `WILDCOPY_OVERLENGTH` extra
space, like it is supposed to.
* We didn't check the literals buffer size in `ZSTD_sufficientBuff()`.
2018-08-28 13:24:44 -07:00
b37a0a6bde
Merge pull request #1298 from facebook/bench
...
Refactored bench.c
2018-08-28 12:25:02 -07:00
af23d39eb8
Merge pull request #1297 from felixhandte/check-offset-table
...
Fix Missing Offset Table Check
2018-08-24 17:36:44 -07:00
37f17ee237
Mark Repeated Offset Table as Needing Check
2018-08-24 14:33:34 -07:00
e34e917655
Fix compiler warning
2018-08-23 17:48:06 -07:00
924944e471
[zstd] Reuse the ZSTD_CCtx more often with small data.
2018-08-23 17:48:06 -07:00
2e45badff4
refactored bench.c
...
for clarity and safety, especially at interface level
2018-08-23 14:21:18 -07:00
1515f0bb0d
fixed more issues detected by recent version of scan-build
...
test run on Linux
2018-08-16 15:20:25 -07:00
3692c31598
Merge branch 'dev' into scanbuild
2018-08-15 13:50:49 -07:00
6e66bbf5dd
fixed several minor issues detected by scan-build
...
only notable one :
writeNCount() resists better vs invalid distributions
(though it should never happen within zstd anyway)
2018-08-14 16:55:35 -07:00
79a35ac20d
minor code comments improvements
2018-08-09 15:16:31 -07:00
2ca7c69167
Fix CDict Attachment to Handle CDicts with Non-Zero Starts
...
CDicts were previously guaranteed to be generated with `lowLimit=dictLimit=0`.
This is no longer true, and so the old length and index calculations are no
longer valid. This diff fixes them to handle non-zero start indices in CDicts.
2018-08-07 18:14:14 -07:00
5808027abf
Merge branch 'dev' into fix1241
2018-08-03 16:08:33 -07:00
dc5a67cb7b
Disallow tableLog == srcLog
2018-08-02 11:12:17 -07:00
aade1e5904
Merge branch 'dev' into fix1241
2018-07-30 16:30:35 +02:00
9889bca530
[FSE] Fix division by zero
...
When the primary normalization method fails, and
`(1 << tableLog) == (maxSymbolValue + 1)`, and every symbol gets assigned
normalized weight 1 or -1 in the first loop, then the next division can
raise `SIGFPE`.
2018-07-27 17:30:03 -07:00
6e490a2f09
Merge pull request #1237 from terrelln/init-cstream-adv
...
Set requestedParams in ZSTD_initCStream*()
2018-07-18 16:33:30 +02:00
9597b438e9
fix #1241
...
Ensure that first input position is valid for a match
even during first usage of context
by starting reference at 1
(avoiding the problematic 0).
2018-07-17 18:52:57 +02:00
53e1f0504e
zstdmt debug traces compatibles with mingw
...
since mingw does not have `sys/times.h`,
remove this path when detecting mingw compilation.
2018-07-17 14:39:44 +02:00
6d222c437c
Set requestedParams in ZSTD_initCStream*()
...
The correct parameters are used once, but once `ZSTD_resetCStream()` is
called the default parameters (level 3) are used. Fix this by setting
`requestedParams` in the `ZSTD_initCStream*()` functions.
The added tests both fail before this patch and pass after.
2018-07-12 18:35:55 -07:00
4489daec09
slightly adjusted default-distribution threshold
...
depending on strategy.
fast favors faster compression and decompression speeds.
2018-06-26 20:10:45 -07:00
b426bcc097
[zstdmt] Fix jobsize bugs ( #1205 )
...
[zstdmt] Fix jobsize bugs
* `ZSTDMT_serialState_reset()` should use `targetSectionSize`, not `jobSize` when sizing the seqstore.
Add an assert that checks that we sized the seqstore using the right job size.
* `ZSTDMT_compressionJob()` should check if `rawSeqStore.seq == NULL`.
* `ZSTDMT_initCStream_internal()` should not adjust `mtctx->params.jobSize` (clamping to MIN/MAX is okay).
2018-06-25 15:21:08 -07:00
3b53bfe4f3
Merge pull request #1200 from felixhandte/zstd-attach-dict-pref
...
Add CCtx Param Controlling Dict Attachment Behavior
2018-06-25 12:42:31 -07:00
3934e010a2
Merge pull request #1197 from facebook/poolResize
...
Thread Pool resize
2018-06-22 14:20:07 -07:00
fbd5dfc1b1
changed POOL_resize() return type to int
...
return is now just en error code.
This guarantee that `ctx` remains valid after POOL_resize().
Gets rid of internal POOL_free() operation.
2018-06-22 12:14:59 -07:00
1d5648ca10
Merge pull request #1196 from felixhandte/zstd-btopt-in-place-dict
...
ZSTD_btopt: Support Searching the Dictionary Context In-Place
2018-06-22 11:53:23 -07:00
f6242d30b7
Merge pull request #1202 from facebook/barelyCompressible
...
Increase threshold detection of poorly compressible data
2018-06-22 11:52:52 -07:00
698fd00afb
huf: increase threshold detection of poorly compressible data
2018-06-21 18:32:38 -07:00
01bb1c1016
Add CCtx Param Controlling Dict Attachment Behavior
2018-06-21 17:29:25 -04:00
3e91dc4d6a
Add Repcode Bounds Check
2018-06-21 15:54:41 -04:00
5bd3d4b7d2
Add Debug Log Statement
2018-06-21 15:54:07 -04:00
3caba150c6
Fix `dmsBtLow` Test
2018-06-21 15:53:40 -04:00
5da9bbc38e
Conceivably Dedup ZSTD_noDict and ZSTD_dictMatchState _insertBt1 Impls
...
By reverting to the bool extDict flag, we call ZSTD_insertBt1 with the same
const args in both non-extDict dictModes.
2018-06-21 11:20:01 -04:00
5d81f71e83
Consistency in Guarding DMS-Only Variable Initializations
2018-06-20 16:54:53 -04:00
9c14eafe3d
Also Use `matchLow` for HC3 Match
2018-06-20 15:51:14 -04:00
0a6cf7cd1d
Minor Changes
2018-06-20 15:27:23 -04:00
ae1f3898a2
Remove Dead(!) HC3 DMS Lookup
2018-06-20 15:27:12 -04:00
93702a7a62
Merge pull request #1198 from facebook/msdebug
...
made Visual Studio compatible with DEBUGLEVEL >= 2
2018-06-20 12:26:31 -07:00
ae0b7ffa0a
made Visual Studio compatible with DEBUGLEVEL >= 2
2018-06-20 09:45:02 -07:00
166901dc72
reduced POOL_resize() restriction
...
It's not necessary to ensure that no job is ongoing.
The pool is only expanded, existing threads are preserved.
In case of error, the only option is to return NULL and terminate the thread pool anyway.
2018-06-19 18:07:18 -07:00
066fbbfe1c
make zstdmt resize its context
...
when nbThreads change.
Technically, it only expands.
But when instructed to use less threads,
the thread pool will limit nb of concurrent threads.
2018-06-19 17:28:56 -07:00
6768cf53fd
Merge pull request #1190 from terrelln/ldm-adjust
...
Adjust advanced parameters to source size
2018-06-19 14:40:56 -07:00
03c39c540b
Fix Incorrect Param
2018-06-19 15:36:33 -04:00
de639502aa
Update Dict Attachment Cut-Offs
2018-06-19 15:36:13 -04:00
f0a13bcd68
Make Sure Position 0 Gets Into the Tree
2018-06-19 15:10:06 -04:00
87fe4788a3
Fix Compression Ratio Regression #1
2018-06-19 13:01:21 -04:00
4bb79f9c55
Misc Changes
2018-06-19 13:01:21 -04:00
2091f34e9e
Find Proper Matches
2018-06-19 13:01:21 -04:00
64348a15f1
Misc Fixes
2018-06-19 13:01:21 -04:00
Yann Collet
Nick Terrell
W. Felix Handte
Yann Collet
cyan4973
Nick Terrell