4a61aaf368
Remove redundant comment
2019-11-08 13:57:26 -05:00
c787b351ea
Use ZSTD Error codes, improve explanation of ZSTD_loadCEntropy() and ZSTD_loadDEntropy()
2019-11-08 13:57:26 -05:00
04fb42b4f3
Integrated refactor into getDictHeaderSize, now passes tests
2019-11-08 13:57:26 -05:00
0bcaf6db08
First working pass at refactor of loadZstdDictionary()
2019-11-08 13:57:26 -05:00
4b141b63e0
Revert "Move decompress symbols into zstd_internal.h, remove dependency"
...
This reverts commit a152b4c67a5266f611db4a2eac4a79003852a795.
2019-11-08 13:57:26 -05:00
84404cff6e
Move decompress symbols into zstd_internal.h, remove dependency
2019-11-08 13:57:26 -05:00
341e0641ed
Checks malloc() for failure, returns 0 if so
2019-11-08 13:57:26 -05:00
97b7f712f3
Change to heap allocation, remove implicit type conversion
2019-11-08 13:57:25 -05:00
3c36a7f13a
Add ZDICT_getHeaderSize()
2019-11-08 13:57:08 -05:00
8c474f9845
Fix parameter selection and adjustment with srcSize == 0
2019-11-07 08:58:43 -08:00
5688447758
Merge pull request #1873 from felixhandte/make-overlap-log-multithread-only
...
Fix #1861 : Restrict overlapLog Parameter When Not Built With Multithreading
2019-11-06 16:56:37 -05:00
ba4613602f
Merge pull request #1843 from moozzyk/issue-1637
...
Take ZSTD_parameters as a const pointer
2019-11-06 16:56:14 -05:00
c13f81905a
Fix #1861 : Restrict overlapLog Parameter When Not Built With Multithreading
...
This parameter is unused in single-threaded compression. We should make it
behave like the other multithread-only parameters, for which we only accept
zero when we are not built with multithreading.
2019-11-06 16:05:02 -05:00
13bb7500e8
Fix frame argument to compression
2019-11-05 16:15:55 -05:00
f2932fb5eb
Fix more merge conflicts
2019-11-05 15:54:05 -05:00
7ce891870c
Fix merge conflicts
2019-11-05 15:51:25 -05:00
3fb5b106da
Replacing some literals with constants
2019-11-05 10:26:57 -08:00
60205fec02
Fix 2 bugs in dictionary loading
...
* Silently skip dictionaries less than 8 bytes, unless using `ZSTD_dct_fullDict`.
This changes the compressor, which silently skips dictionaries <= 8 bytes.
* Allow repcodes that are equal to the dictionary content size, since it is in bounds.
2019-11-01 16:52:07 -07:00
b9ede1c8c2
Make sure contentsize is known
2019-10-30 16:03:58 -04:00
9c1860861e
Fix assert in ZSTD_safecopy
...
In the case that `op >= oend_w` it is possible that `diff < 8` because
the two buffers could be adjacent.
Credit to OSS-Fuzz, which found the bug. It isn't reproducible because
it depends on the memory layout.
2019-10-28 17:51:17 -07:00
01ec595b85
Merge pull request #1851 from felixhandte/pkg-config-prefix-fix
...
In pkg-config File, Derive Lib and Include Dir from Prefix at Use-Time
2019-10-28 14:24:56 -04:00
74065da4c5
updated API inline doc and manual
...
regarding ZSTD_CDict created without a dictBuffer.
2019-10-28 11:15:41 -07:00
74bd76c3ff
In pkg-config File, Derive Lib and Include Dir from Prefix at Use-Time
...
Addresses #1794 . Instead of deriving the lib dir and include dir at
build-time, let's do it like everyone else does at pkg-config run-time.
This has the disadvantage that we can no longer override LIBDIR and
INCLUDEDIR in the Makefile and have that reflected in the .pc file.
2019-10-25 15:07:31 -04:00
c2140e9db0
Merge pull request #1845 from facebook/zbuff
...
improve deprecation warning macro
2019-10-25 09:59:00 -07:00
a9a216a846
Merge pull request #1824 from senhuang42/new_path_for_cdict
...
Avoid using CDict params when input is large.
2019-10-23 12:04:40 -07:00
63e435dda1
improve deprecation warning macro
...
fix #1488
although, curiously enough, I was never able to reproduce the issue
(according to the bug report, it should be present while using gcc 4.8).
2019-10-23 11:59:32 -07:00
eda7946a36
Take ZSTD_parameters as a const pointer
...
Fixes : #1637
2019-10-22 23:21:54 -07:00
f966cd080a
added documentation on DYNAMIC_BMI2 build macro
2019-10-22 17:43:09 -07:00
5d5c895b18
fix initCStream_advanced() for fast strategies
...
Compression ratio of fast strategies (levels 1 & 2)
was seriously reduced, due to accidental disabling of Literals compression.
Credit to @QrczakMK, which perfectly described the issue, and implementation details,
making the fix straightforward.
Example : initCStream with level 1 on synthetic sample P50 :
Before : 5,273,976 bytes
After : 3,154,678 bytes
ZSTD_compress (for comparison) : 3,154,550
Fix #1787 .
To follow : refactor the test which was supposed to catch this issue (and failed)
2019-10-22 15:01:38 -07:00
111b0c53b0
update documentation on deprecated functions
...
mostly : note that these functions will soon generate deprecation warnings
2019-10-22 13:51:18 -07:00
b1ec94e63c
Fix ZSTD_f_zstd1_magicless for small data
...
* Fix `ZSTD_FRAMEHEADERSIZE_PREFIX` and `ZSTD_FRAMEHEADERSIZE_MIN` to
take a `format` parameter, so it is impossible to get the wrong size.
* Fix the places that called `ZSTD_FRAMEHEADERSIZE_PREFIX` without
taking the format into account, which is now impossible by design.
* Call `ZSTD_frameHeaderSize_internal()` with `dctx->format`.
* The added tests catch both bugs in `ZSTD_decompressFrame()`.
Fixes #1813 .
2019-10-21 21:16:17 -07:00
c2e1e54f24
((x or y) or z) == (x or y or z), remove brackets
2019-10-21 19:16:50 -04:00
59c81aa31b
Line up comments :)
2019-10-21 19:12:15 -04:00
dbda8c318a
Trailing comma
2019-10-21 19:10:13 -04:00
0c00455ea6
Merge branch 'dev' of github.com:senhuang42/zstd into new_path_for_cdict
2019-10-21 19:06:51 -04:00
5b2f4ac1a8
merge
2019-10-21 19:02:52 -04:00
2ab484a5f9
Fix bad merge
2019-10-21 18:55:17 -04:00
919d1d8e93
Merge pull request #1831 from terrelln/zstdmt-bad-memset
...
[zstdmt] Don't memset the jobDescription
2019-10-21 15:53:57 -07:00
b6c3459d50
merge
2019-10-21 18:46:17 -04:00
6cf04c0344
Merge pull request #1834 from facebook/winFix
...
Windows fixes
2019-10-21 13:45:17 -07:00
676f89902a
Added multiplier, renamed new enum to something more useful
2019-10-21 15:36:12 -04:00
1f3a51fb52
Updated forceAttachDict param bounds
2019-10-21 15:36:12 -04:00
8f69c47643
Add enum to decision process
2019-10-21 15:36:12 -04:00
e4de8b098a
Added support for forcing new CDict behavior and updated enum
2019-10-21 15:36:12 -04:00
9294f4826b
Changed to int from BYTE
2019-10-21 15:36:12 -04:00
f0fccc8847
Changed to int from BYTE
2019-10-21 15:36:12 -04:00
bb2df8c499
Trailing whitespace
2019-10-21 15:36:12 -04:00
cf51501d2f
Fix test
2019-10-21 15:36:12 -04:00
ea3cb6988f
Cast to BYTE to appease appveyor
2019-10-21 15:36:12 -04:00
a727a85a7e
merge conflicts round 2
2019-10-21 15:36:12 -04:00
053a35fd64
formatting
2019-10-21 15:35:33 -04:00
3fa4daaa55
Fix error
2019-10-21 15:35:33 -04:00
3328348c63
Add compressionlevel to cdict
2019-10-21 15:32:39 -04:00
cf725630a6
Merge pull request #1795 from felixhandte/workspace-asan
...
Add Poisoned Redzones to the Workspace When Compiling with ASAN
2019-10-21 12:15:17 -04:00
e8aa3e486d
Updated forceAttachDict param bounds
2019-10-20 22:01:08 -04:00
6d297265f9
Add enum to decision process
2019-10-20 19:02:47 -04:00
1daa898c93
Added support for forcing new CDict behavior and updated enum
2019-10-20 14:03:09 -04:00
0bc39bc3a0
[zstdmt] Don't memset the jobDescription
2019-10-18 15:05:51 -07:00
243824551f
[threading] Add debug utilities
2019-10-18 15:05:34 -07:00
1795133c45
refactored FIO_compressMultipleFilenames() prototype
...
for consistency
2019-10-17 15:32:03 -07:00
6446ffb277
Merge pull request #1827 from facebook/dm_Dct
...
updated erroneous comments using ZSTD_dm_*
2019-10-17 10:30:58 -07:00
19741c7d99
Merge pull request #1815 from facebook/zlibwrap
...
make zlibWrapper strict ISO-C90 compatible
2019-10-16 16:45:15 -07:00
6323966e53
updated erroneous comments using ZSTD_dm_*
...
instead of the current ZSTD_dct_*,
reported by @nigeltao (#1822 )
2019-10-16 16:14:04 -07:00
2d5201b0ab
removed wildcopy8()
...
which is no longer used,
noticed by @davidbolvansky
2019-10-16 14:51:33 -07:00
4455f00cb8
Changed to int from BYTE
2019-10-16 15:06:02 -04:00
4f7d26b0ee
Changed to int from BYTE
2019-10-16 15:05:29 -04:00
cf00ea367a
Trailing whitespace
2019-10-16 10:31:27 -04:00
8cb2174446
Fix test
2019-10-16 10:29:31 -04:00
5e901b6f32
Cast to BYTE to appease appveyor
2019-10-15 13:58:44 -04:00
5c010c9d2d
merge conflicts round 2
2019-10-15 13:10:05 -04:00
a06b51879c
merge conflict
2019-10-15 12:58:50 -04:00
23dac23a49
formatting
2019-10-15 12:44:48 -04:00
0c8df5c928
Fix error
2019-10-15 12:28:23 -04:00
a65eb39f9d
Add compressionlevel to cdict
2019-10-15 10:22:06 -04:00
fb77afc626
Merge pull request #1760 from bimbashrestha/extract_sequences_api
...
Adding api for extracting sequences from seqstore
2019-10-10 13:11:18 -07:00
ede31da2ea
Fix CCtx Size Estimation
2019-10-10 15:02:08 -04:00
bd6a20b8a0
Expand Default Redzone Size
2019-10-10 13:45:55 -04:00
2c80a9f8ac
Check if CCtx in Workspace after Null Check
2019-10-10 13:40:16 -04:00
b6987acbbf
Declare the ASAN Functions We Need, Don't Include the Header
2019-10-10 13:40:16 -04:00
0ffae7e440
Stop Allocating Extra Space for Table Redzones
2019-10-10 13:40:16 -04:00
a07037b784
Don't Try to Redzone the Tables
2019-10-10 13:40:16 -04:00
0cc481ef66
Fix Workspace Size Calculation
2019-10-10 13:40:16 -04:00
b6c0a02a17
Fix ZSTD_sizeof_matchState() Calculation
2019-10-10 13:40:16 -04:00
8cffd6ed08
Avoid ASAN Failure in ZSTD_cwksp_free()
2019-10-10 13:40:16 -04:00
ef0b5707c5
Refactor Freeing CCtxes / CDicts Inside Workspaces
2019-10-10 13:40:16 -04:00
143b296cf6
Surround Workspace Allocs with Dead Zone
2019-10-10 13:40:16 -04:00
19a0955ec9
Add `ZSTD_cwksp_alloc_size()` to Help Calculate Needed Workspace Size
2019-10-10 13:40:16 -04:00
da88c35d41
Stop Assuming Tables are Adjacent
2019-10-10 13:40:16 -04:00
35c30d6ca7
Poison Unused Workspace Memory
2019-10-10 13:40:16 -04:00
edb6d884a5
Detect Whether We're Being Compiled with ASAN
2019-10-10 13:40:16 -04:00
dc1fb684bf
Remove Unused MEM_SKIP_MSAN Macro
2019-10-10 13:40:16 -04:00
36528b96c4
Manually moving instead of memcpy on decoder and using genBuffer()
2019-10-03 09:26:51 -07:00
61ec4c2e7f
Cleaning sequence parsing logic
2019-10-03 06:42:40 -07:00
cb18fffe65
enforce C90 compatibility for zlibWrapper
2019-09-24 17:50:58 -07:00
ad2a2785f7
bump version number to v1.4.4
...
so that future reports on `dev` branch use this number instead
2019-09-24 15:15:33 -07:00
c04245b257
Replacing assert with memory_allocation error code throw
2019-09-23 15:42:16 -07:00
be0bebd24e
Adding test and null check for malloc
2019-09-23 15:08:18 -07:00
1ab1a40c9c
Fixed one more place
2019-09-23 21:32:56 +02:00
1f7228c040
Use clz ^ 31 instead of 31 - clz; better codegen for GCC
2019-09-23 21:23:09 +02:00
7451c6578c
Merge pull request #1804 from terrelln/wild-and-fast
...
Optimize (de)compression and fix wildcopy overread
2019-09-21 17:04:36 -07:00
5cb7615f1f
Add UNUSED_ATTR to ZSTD_storeSeq()
2019-09-20 21:37:13 -07:00
5dc0a1d659
HINT_INLINE ZSTD_storeSeq()
...
Clang on Mac wasn't inlining `ZSTD_storeSeq()` in level 1, which was
causing a 5% performance regression. This fixes it.
2019-09-20 16:39:27 -07:00
f3c4fd17e3
Passing in dummy dst buffer of compressbound(srcSize)
2019-09-20 15:50:58 -07:00
c047fcf7bf
Merge pull request #1806 from felixhandte/estimate-cctx-doc
...
Update Comment on `ZSTD_estimateCCtxSize()`
2019-09-20 15:36:00 -04:00
44c65da97e
Remove literals overread in ZSTD_storeSeq() for ~neutral perf
2019-09-20 12:23:25 -07:00
f7d9b36835
Update Comment on `ZSTD_estimateCCtxSize()`
2019-09-20 14:11:29 -04:00
fde217df04
Fix bounds check in ZSTD_storeSeq()
2019-09-20 08:25:12 -07:00
67b1f5fc72
Fix too strict assert
2019-09-20 01:23:35 -07:00
ddab2a94e8
Pass iend into ZSTD_storeSeq() to allow ZSTD_wildcopy()
2019-09-20 00:56:20 -07:00
cdad7fa512
Widen ZSTD_wildcopy to 32 bytes
2019-09-20 00:52:15 -07:00
efd37a64ea
Optimize decompression and fix wildcopy overread
...
* Bump `WILDCOPY_OVERLENGTH` to 16 to fix the wildcopy overread.
* Optimize `ZSTD_wildcopy()` by removing unnecessary branches and
unrolling the loop.
* Extract `ZSTD_overlapCopy8()` into its own function.
* Add `ZSTD_safecopy()` for `ZSTD_execSequenceEnd()`. It is
optimized for single long sequences, since that is the important
case that can end up in `ZSTD_execSequenceEnd()`. Without this
optimization, decompressing a block with 1 long match goes
from 5.7 GB/s to 800 MB/s.
* Refactor `ZSTD_execSequenceEnd()`.
* Increase the literal copy shortcut to 16.
* Add a shortcut for offset >= 16.
* Simplify `ZSTD_execSequence()` by pushing more cases into
`ZSTD_execSequenceEnd()`.
* Delete `ZSTD_execSequenceLong()` since it is exactly the
same as `ZSTD_execSequence()`.
clang-8 seeds +17.5% on silesia and +21.8% on enwik8.
gcc-9 sees +12% on silesia and +15.5% on enwik8.
TODO: More detailed measurements, and on more datasets.
Crdit to OSS-Fuzz for finding the wildcopy overread.
2019-09-19 21:07:14 -07:00
ae6d0e64ae
Addressing comments
2019-09-19 15:25:20 -07:00
3cac061db5
Merge pull request #1802 from bimbashrestha/rle_block_bound_fix_pt2
...
Adding 4 blocks to FSE_BLOCKBOUND() in lib/common (different from las…
2019-09-18 16:32:37 -07:00
6e9f6813bb
adding bit container size
2019-09-18 13:49:45 -07:00
f9b6abb896
Adding 4 blocks to FSE_BLOCKBOUND() in lib/common (different from last week)
2019-09-18 13:29:05 -07:00
bfff5b30a4
Merge pull request #1756 from mgrice/dev
...
Improvements in zstd decode performance
2019-09-18 11:35:50 -07:00
243200e5bf
minor refactor of ZSTD_fast
...
- reduced variables lifetime
- more accurate code comments
2019-09-17 14:02:57 -07:00
76fea3fb99
Resolving appveyor test failure implicit conversion
2019-09-16 14:02:23 -07:00
a874435478
Merge branch 'dev' into extract_sequences_api
2019-09-16 13:29:59 -07:00
2164a130f3
Merge pull request #1780 from felixhandte/workspace-efficiency-3
...
Avoid Clearing Tables Even When Changing CParams
2019-09-16 14:37:05 -04:00
72ea79cacd
Don't Include `sanitizer/msan_interface.h`, Since Not All Platforms Provide It
...
Instead, explicitly declare the functions we use.
2019-09-16 12:08:03 -04:00
bff6072e3a
Bailing early when collecting sequences and documentation
2019-09-16 08:26:21 -07:00
fbeaf6989e
[libzstd] Improve advanced API docs
2019-09-15 12:41:24 -07:00
09b1844d9b
Merge pull request #1784 from bimbashrestha/fse_block_bound_err
...
Rearranging assert and allowing 4 extra for FSE_BLOCKBOUND()
2019-09-12 19:09:27 -07:00
fe9af338ed
Added assert to BIT_flushBits()
2019-09-12 15:35:27 -07:00
43da5bf27e
Rearranging assert and allowing 4 extra for FSE_BLOCKBOUND()
2019-09-12 14:43:50 -07:00
20c69077d1
Shrink Table Valid End During Alloc Alignment / Phase Change
2019-09-11 17:14:59 -04:00
51d90668ba
Add Assertions to Confirm that Workspace Pointers are Correctly Ordered
2019-09-11 17:14:59 -04:00
a10c191613
`__msan_poison()` Workspace When Preparing for Re-Use
2019-09-11 17:14:45 -04:00
7c57e2b9ca
Zero `h3size` When `h3log` is 0
...
This led to a nasty edgecase, where index reduction for modes that don't use
the h3 table would have a degenerate table (size 4) allocated and marked clean,
but which would not be re-indexed.
2019-09-11 13:14:26 -04:00
bc020eec92
Also Shrink Clean Table Area When Reducing Indices
2019-09-11 11:40:57 -04:00
1999b2ed9b
Update DEBUGLOG Statements
2019-09-11 11:21:00 -04:00
13e29a56de
Shrink Clean Table Area When Copying Table Contents into Context
...
The source matchState is potentially at a lower current index, which means
that any extra table space not overwritten by the copy may now contain
invalid indices. The simple solution is to unconditionally shrink the valid
table area to just the area overwritten.
2019-09-11 11:18:45 -04:00
edb3ad053e
Comments
2019-09-10 18:25:45 -04:00
f31ef28ff8
Only Reset Indexing in `ZSTD_resetCCtx_internal()` When Necessary
2019-09-10 18:25:45 -04:00
9968a53e91
Remove No-Longer-Used Continuation Functions
2019-09-10 18:25:45 -04:00
1b28e80416
Remove Fast Continue Path in `ZSTD_resetCCtx_internal()`
2019-09-10 18:25:45 -04:00
ad16eda5e4
`ZSTD_reset_matchState` Optionally Doesn't Restart Indexing
2019-09-10 18:25:45 -04:00
5b10bb5ec3
Rename `ZSTD_compResetPolicy_e` Values and Add Comment
2019-09-10 18:25:45 -04:00
0492b9a9ec
Accept `ZSTD_indexResetPolicy_e` Param in `ZSTD_reset_matchState()`
2019-09-10 18:25:45 -04:00
14c5471d5e
Introduce `ZSTD_indexResetPolicy_e` Enum
2019-09-10 18:25:45 -04:00
17b6da2e0f
Track Usable Table Space in Compression Workspace
2019-09-10 18:25:37 -04:00
22bd158e0f
Merge pull request #1712 from felixhandte/workspace-efficiency-2
...
Allocate Internal Buffers via Workspace Abstraction
2019-09-10 15:20:29 -07:00
1407919d13
Addressing comments on parsing
2019-09-10 15:10:50 -07:00
47199480da
Cleaning up parsing per suggestion
2019-09-10 13:18:59 -07:00
a9d373f093
Remove Empty lib/compress/zstd_cwksp.c
2019-09-10 16:03:13 -04:00
5ba495b622
Merge pull request #1775 from facebook/edufix
...
fix educational decoder
2019-09-10 12:12:08 -07:00
41416f0927
Merge pull request #1773 from bimbashrestha/rle_first_block_decompression_fix
...
Removing redundant condition in decompression, making first block rle…
2019-09-10 11:17:29 -07:00
e3c5825918
Fizing litLength == 0 case
2019-09-10 10:38:13 -07:00
9e7bb55e14
Addressing comments
2019-09-09 20:04:46 -07:00
81208fd7c2
Forward Declare `ZSTD_cwksp_available_space` to Fix Build
2019-09-09 19:10:09 -04:00
91bf1babd1
Inline Workspace Functions
2019-09-09 18:53:53 -04:00
0db3ffe7ee
Forward resetCCtx Errors when Using CDict
2019-09-09 16:47:19 -04:00
eb6f69d978
Fix sizeof_CCtx and sizeof_CDict Calculations for Statically Init'ed Objects
2019-09-09 16:45:17 -04:00
e3703825a8
Fix workspaceTooSmall Calculation
2019-09-09 15:12:14 -04:00
0a65a67901
Shorten `&zc->workspace` -> `ws` in `ZSTD_resetCCtx_internal()`
2019-09-09 14:59:09 -04:00
1120e4d962
Clean Up TODOs and Comments pt. II
2019-09-09 14:04:39 -04:00
c60e1c3be5
Nit
2019-09-09 13:34:08 -04:00
7d7b665c90
Pull Phase Advance Logic Out into Internal Function
2019-09-09 13:34:08 -04:00
8549ae9f1d
Hide Workspace Movement Behind Helper Function
2019-09-09 13:34:08 -04:00
2405c03bcd
Fix DEBUGLOG Statement Levels
2019-09-09 13:34:08 -04:00
7100d24221
Fix Rescale Continue Special Case
2019-09-09 13:34:08 -04:00
7321e4c9f3
Remove Unused noRealloc CRP Value
2019-09-09 13:34:08 -04:00
901bba4ca6
Re-Implement Workspace Shrinking when Oversized
2019-09-09 13:34:08 -04:00
881bcd80ca
Cleanup from Move
2019-09-09 13:34:08 -04:00
b511a84adc
Move Workspace Functions to Their Own File
2019-09-09 13:34:08 -04:00
077a2d7dc9
Rename
2019-09-09 13:34:08 -04:00
ebd162194f
Clean Up TODOs and Comments
2019-09-09 13:34:08 -04:00
2abe0145b1
Improve Comments a Bit
2019-09-09 13:34:08 -04:00
7a2416a863
Allocate CDict in Workspace (Rather than in Separate Allocation)
2019-09-09 13:34:08 -04:00
65057cf009
Rewrite ZSTD_initStaticCCtx to Alloc CCtx in Workspace
2019-09-09 13:34:08 -04:00
58b69ab15c
Only the CCtx Itself Needs to be Cleared during Static CCtx Init
2019-09-09 13:34:08 -04:00
88c2fcd0ee
Align Alloc Pointer When Transitioning from Buffers to Aligned Allocs
2019-09-09 13:34:08 -04:00
e936b73889
Remove Overly-Restrictive Assert
2019-09-09 13:34:08 -04:00
75d574368b
When Loading Dict By Copy, Always Put it in the Workspace
2019-09-09 13:34:08 -04:00
e69b67e33a
Alloc Tables Separately
2019-09-09 13:34:08 -04:00
6177354b36
Begin Introducing Phases
2019-09-09 13:34:08 -04:00
786f2266bb
TMP
2019-09-09 13:34:08 -04:00
c25283cf00
Disambiguate 'workspace' and 'entropyWorkspace'
2019-09-09 13:34:08 -04:00
ccaac852e8
Normalize Case 'workSpace' -> 'workspace'
2019-09-09 13:27:18 -04:00
44e122053b
Mentioning cli only in the comment as suggested
2019-09-06 14:48:41 -07:00
2b0a271ed2
fix eductional decoder
...
fix #1774
also :
- fix minor compilation warnings
- make sure the `test` is run during CI tests
2019-09-06 14:30:13 -07:00
a917cd597d
Put back omission for first rle block and updated comment as suggested
2019-09-06 13:44:25 -07:00
d687d603e4
Removing redundant condition in decompression, making first block rles valid to deocmpress
2019-09-06 10:46:19 -07:00
9816560649
Fixing assert and DEBUGLOG due to ZSTD_CCtx_params parameter change to const pointer
2019-09-05 15:47:17 +05:30
771645471f
Passing ZSTD_CCtx_params by const pointer
2019-09-05 15:28:30 +05:30
5f8b0f6890
Changing api to get sequences across all blocks
2019-08-30 09:18:44 -07:00
5198347382
Merge pull request #1744 from bimbashrestha/dev
...
Generate RLE blocks in the encoder
2019-08-29 15:19:10 -07:00
623b90f85d
Fixing ci-circle test complaints
2019-08-29 13:09:42 -07:00
5d89771529
fix warning: always_inline function might not be inlinable
2019-08-29 12:32:15 -07:00
ece465644b
Adding api for extracting sequences from seqstore
2019-08-29 12:29:39 -07:00
b830599582
Improvements in zstd decode performance
...
Summary: The idea behind wildcopy is that it can be cheaper to copy more bytes (say 8) than it is to copy less (say, 3). This change takes that further by exploiting some properties:
1. it's almost always OK to copy 16 bytes instead of 8, which means fewer copy instructions, and fewer branches
2. A 16 byte chunk size means that ~90% of wildcopy invocations will have a trip count of 1, so branch prediction will be improved.
Speedup on Xeon E5-2680v4 is in the range of 3-5%.
Measured wildcopy length distributions on silesia.tar:
level <=8 <=16 <=24 >24
1 78.05% 11.49% 3.52% 6.94%
3 82.14% 8.99% 2.44% 6.43%
6 85.81% 6.51% 2.92% 4.76%
8 83.02% 7.31% 3.64% 6.03%
10 84.13% 6.67% 3.29% 5.91%
15 77.58% 7.55% 5.21% 9.66%
16 80.07% 7.20% 3.98% 8.75%
Test Plan: benchmark silesia, make check
2019-08-29 12:25:56 -07:00
c3e3c8bf32
Undoing the last commit (that was an accident)
2019-08-29 12:05:47 -07:00
4a1ca5e0a8
Adding method for extracting sequences.
2019-08-29 11:55:12 -07:00
e5704bbfdf
Added test for multiple blocks of zeros and fixed nit about comments
2019-08-28 08:32:34 -07:00
e9c0fc12d2
Merge pull request #1748 from terrelln/cover-deadlock
...
[dictBuilder] Fix deadlock in *COVER error case
2019-08-27 10:17:28 -07:00
0932de54bc
[dictBuilder] Fix deadlock in *COVER error case
...
The COVER and FASTCOVER dictionary builders can deadlock when
dictionary construction errors, likely because there are too few
samples, or too few distinct dmers. The deadlock only occurs when
there are errors.
Fixes #1746 .
2019-08-26 18:19:29 -07:00
96201d9774
Added bool to cctx and fixed some comment nits
2019-08-26 15:30:41 -07:00
991cbc9024
Fixing mixed declaration compiler complaint
2019-08-26 15:00:50 -07:00
ce264ce53b
Forbiding emission of RLE when its the first block
2019-08-26 14:54:29 -07:00
Sen Huang
Nick Terrell
Felix Handte
Bimba Shrestha
Yann Collet
Yann Collet
moozzyk
Nick Terrell
Dávid Bolvanský
Bimba Shrestha
Varun S Nair
mgrice
bimbashrestha