5cb7615f1f
Add UNUSED_ATTR to ZSTD_storeSeq()
2019-09-20 21:37:13 -07:00
5dc0a1d659
HINT_INLINE ZSTD_storeSeq()
...
Clang on Mac wasn't inlining `ZSTD_storeSeq()` in level 1, which was
causing a 5% performance regression. This fixes it.
2019-09-20 16:39:27 -07:00
44c65da97e
Remove literals overread in ZSTD_storeSeq() for ~neutral perf
2019-09-20 12:23:25 -07:00
fde217df04
Fix bounds check in ZSTD_storeSeq()
2019-09-20 08:25:12 -07:00
67b1f5fc72
Fix too strict assert
2019-09-20 01:23:35 -07:00
ddab2a94e8
Pass iend into ZSTD_storeSeq() to allow ZSTD_wildcopy()
2019-09-20 00:56:20 -07:00
efd37a64ea
Optimize decompression and fix wildcopy overread
...
* Bump `WILDCOPY_OVERLENGTH` to 16 to fix the wildcopy overread.
* Optimize `ZSTD_wildcopy()` by removing unnecessary branches and
unrolling the loop.
* Extract `ZSTD_overlapCopy8()` into its own function.
* Add `ZSTD_safecopy()` for `ZSTD_execSequenceEnd()`. It is
optimized for single long sequences, since that is the important
case that can end up in `ZSTD_execSequenceEnd()`. Without this
optimization, decompressing a block with 1 long match goes
from 5.7 GB/s to 800 MB/s.
* Refactor `ZSTD_execSequenceEnd()`.
* Increase the literal copy shortcut to 16.
* Add a shortcut for offset >= 16.
* Simplify `ZSTD_execSequence()` by pushing more cases into
`ZSTD_execSequenceEnd()`.
* Delete `ZSTD_execSequenceLong()` since it is exactly the
same as `ZSTD_execSequence()`.
clang-8 seeds +17.5% on silesia and +21.8% on enwik8.
gcc-9 sees +12% on silesia and +15.5% on enwik8.
TODO: More detailed measurements, and on more datasets.
Crdit to OSS-Fuzz for finding the wildcopy overread.
2019-09-19 21:07:14 -07:00
bfff5b30a4
Merge pull request #1756 from mgrice/dev
...
Improvements in zstd decode performance
2019-09-18 11:35:50 -07:00
243200e5bf
minor refactor of ZSTD_fast
...
- reduced variables lifetime
- more accurate code comments
2019-09-17 14:02:57 -07:00
20c69077d1
Shrink Table Valid End During Alloc Alignment / Phase Change
2019-09-11 17:14:59 -04:00
51d90668ba
Add Assertions to Confirm that Workspace Pointers are Correctly Ordered
2019-09-11 17:14:59 -04:00
a10c191613
`__msan_poison()` Workspace When Preparing for Re-Use
2019-09-11 17:14:45 -04:00
7c57e2b9ca
Zero `h3size` When `h3log` is 0
...
This led to a nasty edgecase, where index reduction for modes that don't use
the h3 table would have a degenerate table (size 4) allocated and marked clean,
but which would not be re-indexed.
2019-09-11 13:14:26 -04:00
bc020eec92
Also Shrink Clean Table Area When Reducing Indices
2019-09-11 11:40:57 -04:00
1999b2ed9b
Update DEBUGLOG Statements
2019-09-11 11:21:00 -04:00
13e29a56de
Shrink Clean Table Area When Copying Table Contents into Context
...
The source matchState is potentially at a lower current index, which means
that any extra table space not overwritten by the copy may now contain
invalid indices. The simple solution is to unconditionally shrink the valid
table area to just the area overwritten.
2019-09-11 11:18:45 -04:00
edb3ad053e
Comments
2019-09-10 18:25:45 -04:00
f31ef28ff8
Only Reset Indexing in `ZSTD_resetCCtx_internal()` When Necessary
2019-09-10 18:25:45 -04:00
9968a53e91
Remove No-Longer-Used Continuation Functions
2019-09-10 18:25:45 -04:00
1b28e80416
Remove Fast Continue Path in `ZSTD_resetCCtx_internal()`
2019-09-10 18:25:45 -04:00
ad16eda5e4
`ZSTD_reset_matchState` Optionally Doesn't Restart Indexing
2019-09-10 18:25:45 -04:00
5b10bb5ec3
Rename `ZSTD_compResetPolicy_e` Values and Add Comment
2019-09-10 18:25:45 -04:00
0492b9a9ec
Accept `ZSTD_indexResetPolicy_e` Param in `ZSTD_reset_matchState()`
2019-09-10 18:25:45 -04:00
14c5471d5e
Introduce `ZSTD_indexResetPolicy_e` Enum
2019-09-10 18:25:45 -04:00
17b6da2e0f
Track Usable Table Space in Compression Workspace
2019-09-10 18:25:37 -04:00
22bd158e0f
Merge pull request #1712 from felixhandte/workspace-efficiency-2
...
Allocate Internal Buffers via Workspace Abstraction
2019-09-10 15:20:29 -07:00
a9d373f093
Remove Empty lib/compress/zstd_cwksp.c
2019-09-10 16:03:13 -04:00
41416f0927
Merge pull request #1773 from bimbashrestha/rle_first_block_decompression_fix
...
Removing redundant condition in decompression, making first block rle…
2019-09-10 11:17:29 -07:00
81208fd7c2
Forward Declare `ZSTD_cwksp_available_space` to Fix Build
2019-09-09 19:10:09 -04:00
91bf1babd1
Inline Workspace Functions
2019-09-09 18:53:53 -04:00
0db3ffe7ee
Forward resetCCtx Errors when Using CDict
2019-09-09 16:47:19 -04:00
eb6f69d978
Fix sizeof_CCtx and sizeof_CDict Calculations for Statically Init'ed Objects
2019-09-09 16:45:17 -04:00
e3703825a8
Fix workspaceTooSmall Calculation
2019-09-09 15:12:14 -04:00
0a65a67901
Shorten `&zc->workspace` -> `ws` in `ZSTD_resetCCtx_internal()`
2019-09-09 14:59:09 -04:00
1120e4d962
Clean Up TODOs and Comments pt. II
2019-09-09 14:04:39 -04:00
c60e1c3be5
Nit
2019-09-09 13:34:08 -04:00
7d7b665c90
Pull Phase Advance Logic Out into Internal Function
2019-09-09 13:34:08 -04:00
8549ae9f1d
Hide Workspace Movement Behind Helper Function
2019-09-09 13:34:08 -04:00
2405c03bcd
Fix DEBUGLOG Statement Levels
2019-09-09 13:34:08 -04:00
7100d24221
Fix Rescale Continue Special Case
2019-09-09 13:34:08 -04:00
7321e4c9f3
Remove Unused noRealloc CRP Value
2019-09-09 13:34:08 -04:00
901bba4ca6
Re-Implement Workspace Shrinking when Oversized
2019-09-09 13:34:08 -04:00
881bcd80ca
Cleanup from Move
2019-09-09 13:34:08 -04:00
b511a84adc
Move Workspace Functions to Their Own File
2019-09-09 13:34:08 -04:00
077a2d7dc9
Rename
2019-09-09 13:34:08 -04:00
ebd162194f
Clean Up TODOs and Comments
2019-09-09 13:34:08 -04:00
2abe0145b1
Improve Comments a Bit
2019-09-09 13:34:08 -04:00
7a2416a863
Allocate CDict in Workspace (Rather than in Separate Allocation)
2019-09-09 13:34:08 -04:00
65057cf009
Rewrite ZSTD_initStaticCCtx to Alloc CCtx in Workspace
2019-09-09 13:34:08 -04:00
58b69ab15c
Only the CCtx Itself Needs to be Cleared during Static CCtx Init
2019-09-09 13:34:08 -04:00
Nick Terrell
Nick Terrell
Yann Collet
Yann Collet
W. Felix Handte