Commit Graph

5328 Commits (b426bcc097355d82002d8bc7f531de7292384ceb)

Author SHA1 Message Date
W. Felix Handte 8b241da4df Properly Initialize Repcode Values 2018-05-25 13:13:28 -04:00
W. Felix Handte 7097a03749 Add Necessary Dict Variables 2018-05-25 13:13:28 -04:00
W. Felix Handte aacbbf4f9a Rename 'lowest' to 'localLowest' to Prepare to Introduce Dict Indices 2018-05-25 13:13:28 -04:00
W. Felix Handte c10d1b4011 Skeleton for In-Place Impl for ZSTD_dfast 2018-05-25 13:13:28 -04:00
Yann Collet f6ad59ab5c Merge branch 'dev' into staticDictCost 2018-05-24 16:21:02 -07:00
Yann Collet a7c75740b3
Merge pull request #1149 from terrelln/fuzz-py
Small fixes to fuzz.py
2018-05-24 15:56:09 -07:00
Nick Terrell fdd4d8510f Improve compiler detection to work on Mac 2018-05-24 14:21:12 -07:00
Nick Terrell ac852abb8b Define BIT_DEBUG for --debug 2018-05-24 14:21:12 -07:00
Nick Terrell 2a9975f77b Increase the maximum file size 2018-05-24 14:21:12 -07:00
Nick Terrell e712a3a0a3 Small fixes to fuzz.py 2018-05-24 14:21:12 -07:00
Yann Collet 47d2331ed8
Merge pull request #1150 from facebook/fracFse
fix corner case when requiring cost of an FSE symbol
2018-05-24 14:19:30 -07:00
Yann Collet b5ef32fea7 Merge branch 'dev' into fracFse 2018-05-24 14:09:49 -07:00
Yann Collet 776128d16f fix corner case when requiring cost of an FSE symbol
ensure that, when frequency[symbol]==0,
result is (tableLog + 1) bits
with both upper-bit and fractional-bit estimates.

Also : enable BIT_DEBUG in /tests
2018-05-24 13:59:11 -07:00
Yann Collet 08c5be5db3
Merge pull request #1117 from felixhandte/zstd-fast-in-place-dict
ZSTD_fast: Support Searching the Dictionary Context In-Place
2018-05-23 19:32:25 -07:00
Nick Terrell 06b70179da
Work around bug in zstd decoder (#1147)
Work around bug in zstd decoder

Pull request #1144 exercised a new path in the zstd decoder that proved to
be buggy. Avoid the extremely rare bug by emitting an uncompressed block.
2018-05-23 18:02:30 -07:00
Yann Collet 712318a244
Merge pull request #1146 from terrelln/fse-fix
[zstd] Fix decompression edge case
2018-05-23 16:41:42 -07:00
Nick Terrell f2d0924b87 Variable declarations 2018-05-23 14:58:58 -07:00
W. Felix Handte d9c7e67125 Assert that Dict and Current Window are Adjacent in Index Space 2018-05-23 17:53:03 -04:00
W. Felix Handte 298d24fa57 Make loadedDictEnd an Index, not the Dict Len 2018-05-23 17:53:03 -04:00
W. Felix Handte 7ef85e0618 Fixes in re Comments 2018-05-23 17:53:03 -04:00
W. Felix Handte 582b7f85ed Don't Attach Empty Dict Contents
In weird corner cases, they produce unexpected results...
2018-05-23 17:53:03 -04:00
W. Felix Handte 9c92223468 Avoid Undefined Behavior in Match Ptr Calculation 2018-05-23 17:53:03 -04:00
W. Felix Handte a44ab3b475 Remove Out-of-Date Comment 2018-05-23 17:53:03 -04:00
W. Felix Handte 95bdf20a87 Moar Renames 2018-05-23 17:53:03 -04:00
W. Felix Handte 7e0402e738 Also Attach Dict When Source Size is Unknown 2018-05-23 17:53:03 -04:00
W. Felix Handte 3ba70cc759 Clear the Dictionary When Sliding the Window 2018-05-23 17:53:03 -04:00
W. Felix Handte b05ae9b608 Refine ip Initialization to Avoid ARM Weirdness 2018-05-23 17:53:03 -04:00
W. Felix Handte 1a7b34ef28 Use New Index Invariant to Simplify Conditionals 2018-05-23 17:53:03 -04:00
W. Felix Handte 2d598e6fed Force Working Context Indices Greater than Dict Indices 2018-05-23 17:53:03 -04:00
W. Felix Handte d005e5daf4 Whitespace Fix 2018-05-23 17:53:03 -04:00
W. Felix Handte 154eb09419 Switch to Original Match Calc for noDict Repcode Check 2018-05-23 17:53:03 -04:00
W. Felix Handte 191fc74a51 Rename 'hasDict' to 'dictMode' 2018-05-23 17:53:03 -04:00
W. Felix Handte ae4fcf7816 Respond to PR Comments; Formatting/Style/Lint Fixes 2018-05-23 17:53:03 -04:00
W. Felix Handte ca26cecc7a Rename and Reformat 2018-05-23 17:53:03 -04:00
W. Felix Handte 66bc1ca641 Change Cut-Off to 8 KB 2018-05-23 17:53:03 -04:00
W. Felix Handte c31ee3c7f8 Fix Rep Code Initialization 2018-05-23 17:53:03 -04:00
W. Felix Handte b67196f30d Coalesce hasDictMatchState and extDict Checks into One Enum and Rename Stuff 2018-05-23 17:53:03 -04:00
W. Felix Handte 265c2869d1 Split Wrapper Functions to Cause Inlining 2018-05-23 17:53:03 -04:00
W. Felix Handte 6929964d65 Add bounds check in repcode tests 2018-05-23 17:53:03 -04:00
W. Felix Handte 70a537d1d7 Initial Repcode Check Support for Ext Dict Ctx 2018-05-23 17:53:03 -04:00
W. Felix Handte 8d24ff0353 Preliminary Support in ZSTD_compressBlock_fast_generic() for Ext Dict Ctx 2018-05-23 17:53:03 -04:00
W. Felix Handte d18a405779 Refer to the Dictionary Match State In-Place (Sometimes) 2018-05-23 17:53:03 -04:00
Nick Terrell c92dd11940 Error if reported size is too large in edge case 2018-05-23 14:47:20 -07:00
Nick Terrell a97e9a627a [zstd] Fix decompression edge case
This edge case is only possible with the new optimal encoding selector,
since before zstd would always choose `set_basic` for small numbers of
sequences.

Fix `FSE_readNCount()` to support buffers < 4 bytes.

Credit to OSS-Fuzz
2018-05-23 12:16:00 -07:00
Yann Collet 27dc078aa6
Merge pull request #1144 from terrelln/fse-entropy
Approximate FSE encoding costs for selection
2018-05-22 19:25:37 -07:00
Yann Collet 4a498f03dc
Merge pull request #1145 from terrelln/spec
Clarify what happens when Number_of_Sequences == 0
2018-05-22 16:21:40 -07:00
Nick Terrell 73f4c890cd Clarify what happens when Number_of_Sequences == 0 2018-05-22 16:12:33 -07:00
Nick Terrell e3959d5eba Fixes 2018-05-22 16:06:33 -07:00
Yann Collet 7a8b3496b4 Merge branch 'dev' into staticDictCost 2018-05-22 15:10:05 -07:00
Yann Collet a8ddf1d370 disable 2-passes strategy 2018-05-22 15:06:36 -07:00