facebook/zstd
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,192 Commits 2 Branches 0 Tags
Commit Graph

3518 Commits

Author SHA1 Message Date
W. Felix Handte
c25283cf00 Disambiguate 'workspace' and 'entropyWorkspace' 2019-09-09 13:34:08 -04:00
W. Felix Handte
ccaac852e8 Normalize Case 'workSpace' -> 'workspace' 2019-09-09 13:27:18 -04:00
Bimba Shrestha
44e122053b Mentioning cli only in the comment as suggested 2019-09-06 14:48:41 -07:00
Yann Collet
2b0a271ed2 fix eductional decoder 
fix #1774
also :
- fix minor compilation warnings
- make sure the `test` is run during CI tests
 2019-09-06 14:30:13 -07:00
Bimba Shrestha
a917cd597d Put back omission for first rle block and updated comment as suggested 2019-09-06 13:44:25 -07:00
Bimba Shrestha
d687d603e4 Removing redundant condition in decompression, making first block rles valid to deocmpress 2019-09-06 10:46:19 -07:00
Varun S Nair
9816560649 Fixing assert and DEBUGLOG due to ZSTD_CCtx_params parameter change to const pointer 2019-09-05 15:47:17 +05:30
Varun S Nair
771645471f Passing ZSTD_CCtx_params by const pointer 2019-09-05 15:28:30 +05:30
Bimba Shrestha
5f8b0f6890 Changing api to get sequences across all blocks 2019-08-30 09:18:44 -07:00
Yann Collet
5198347382
Merge pull request #1744 from bimbashrestha/dev 
Generate RLE blocks in the encoder
 2019-08-29 15:19:10 -07:00
Bimba Shrestha
623b90f85d Fixing ci-circle test complaints 2019-08-29 13:09:42 -07:00
mgrice
5d89771529 fix warning: always_inline function might not be inlinable 2019-08-29 12:32:15 -07:00
Bimba Shrestha
ece465644b Adding api for extracting sequences from seqstore 2019-08-29 12:29:39 -07:00
mgrice
b830599582 Improvements in zstd decode performance 
Summary: The idea behind wildcopy is that it can be cheaper to copy more bytes (say 8) than it is to copy less (say, 3).  This change takes that further by exploiting some properties:
1. it's almost always OK to copy 16 bytes instead of 8, which means fewer copy instructions, and fewer branches
2. A 16 byte chunk size means that ~90% of wildcopy invocations will have a trip count of 1, so branch prediction will be improved.

Speedup on Xeon E5-2680v4 is in the range of 3-5%.

Measured wildcopy length distributions on silesia.tar:

level	<=8	<=16	<=24	>24
1	78.05%	11.49%	3.52%	6.94%
3	82.14%	8.99%	2.44%	6.43%
6	85.81%	6.51%	2.92%	4.76%
8	83.02%	7.31%	3.64%	6.03%
10	84.13%	6.67%	3.29%	5.91%
15	77.58%	7.55%	5.21%	9.66%
16	80.07%	7.20%	3.98%	8.75%

Test Plan: benchmark silesia, make check
 2019-08-29 12:25:56 -07:00
Bimba Shrestha
c3e3c8bf32 Undoing the last commit (that was an accident) 2019-08-29 12:05:47 -07:00
bimbashrestha
4a1ca5e0a8 Adding method for extracting sequences. 2019-08-29 11:55:12 -07:00
bimbashrestha
e5704bbfdf Added test for multiple blocks of zeros and fixed nit about comments 2019-08-28 08:32:34 -07:00
Nick Terrell
e9c0fc12d2
Merge pull request #1748 from terrelln/cover-deadlock 
[dictBuilder] Fix deadlock in *COVER error case
 2019-08-27 10:17:28 -07:00
Nick Terrell
0932de54bc [dictBuilder] Fix deadlock in *COVER error case 
The COVER and FASTCOVER dictionary builders can deadlock when
dictionary construction errors, likely because there are too few
samples, or too few distinct dmers. The deadlock only occurs when
there are errors.

Fixes #1746.
 2019-08-26 18:19:29 -07:00
bimbashrestha
96201d9774 Added bool to cctx and fixed some comment nits 2019-08-26 15:30:41 -07:00
bimbashrestha
991cbc9024 Fixing mixed declaration compiler complaint 2019-08-26 15:00:50 -07:00
bimbashrestha
ce264ce53b Forbiding emission of RLE when its the first block 2019-08-26 14:54:29 -07:00
bimbashrestha
33b6446ca7 Removing accidental method call 2019-08-26 14:34:43 -07:00
bimbashrestha
7b041b552e Removing assert for rle that doesn't always hold 2019-08-26 12:26:53 -07:00
bimbashrestha
1f2bf77f2a Using typedef U32 instead of int 2019-08-26 09:00:22 -07:00
bimbashrestha
ba46932492 Removing implicit conversion from const void* to const BYTE* and added constant for threshold 2019-08-26 08:51:34 -07:00
Carl Woffenden
c690f22e96 Merge branch 'dev' into amalgamate 2019-08-23 23:05:02 +02:00
Carl Woffenden
5144e66095 Revert "Merge remote-tracking branch 'origin/master' into dev" 
This reverts commit 0df29a4e5ff118ec7aa870ea9c032e9f99870e43, reversing
changes made to 69c875a0cc771098f9ae9bb7a0a3e3ff6eb3f5c1.
 2019-08-23 23:04:21 +02:00
Carl Woffenden
0fcaa675e0 Merge remote-tracking branch 'upstream/dev' into dev 2019-08-23 23:03:52 +02:00
Carl Woffenden
0df29a4e5f Merge remote-tracking branch 'origin/master' into dev 2019-08-23 22:57:06 +02:00
bimbashrestha
0e3ba02cf1 Fixing more test falure errors 2019-08-22 13:54:41 -07:00
bimbashrestha
4faf3a5911 Fixing ci-circle test failure issues 2019-08-22 13:46:15 -07:00
bimbashrestha
cba5350f88 Moving RLE logic to inside ZSTD_compressBlock_internal and adding assert 2019-08-22 12:12:44 -07:00
Nick Magerko
493f95c7df Fix merge conflicts 2019-08-22 11:51:41 -07:00
bimbashrestha
4c90d862e3 Generate RLE blocks in the encoder 2019-08-22 11:27:20 -07:00
Nick Terrell
54ad33448c
Merge pull request #1737 from terrelln/legacy-fix 
[legacy] Fix buffer overflow in v0.2 and v0.4 raw literals decompression
 2019-08-21 10:10:24 -07:00
Carl Woffenden
901ea61f83 Tweaks to create a single-file decoder 
The CHECK_F macros differ slightly (but eventually do the same thing). Older GCC needs to fallback on the old-style pragma optimisation flags.
 2019-08-21 17:49:17 +02:00
Yann Collet
38b6428fcd
Merge pull request #1725 from emaste/dev 
remove extraneous doubled ;s
 2019-08-21 05:19:30 -07:00
Yann Collet
fe0877c664
Merge pull request #1721 from facebook/seq127 
fixed very minor inefficiency (nbSeq==127)
 2019-08-21 05:19:12 -07:00
Yann Collet
757ab66879
Merge pull request #1713 from cemeyer/fix_gcc4_build 
Fix the build on GCC 4.x after 812e8f2a1
 2019-08-21 05:17:42 -07:00
Nick Terrell
07f22d465d [legacy] Fix buffer overflow in v0.2 and v0.4 raw literals decompression 
Extends the fix in PR#1722 to v0.2 and v0.4. These aren't built into
zstd by default, and v0.5 onward are not affected.

I only add the `srcSize > BLOCKSIZE` check to v0.4 because the comments
say that it must hold, but the equivalent comment isn't present in v0.2.

Credit to OSS-Fuzz.
 2019-08-20 17:13:04 -07:00
Nick Magerko
de6a6c7364 Fix ZSTD_SRCSIZEHINT_MIN typo 2019-08-20 13:07:51 -07:00
Nick Magerko
c7a24d7a14 Define ZSTD_SRCSIZEHINT_MIN as 0 2019-08-20 13:06:15 -07:00
Nick Magerko
2d39b43906 Use int for srcSizeHint when sensible 2019-08-19 16:49:25 -07:00
Nick Magerko
09894dc2eb Add mention of regression with poor size hints 2019-08-19 13:41:36 -07:00
Nick Magerko
fee8fbcddf Make upper bound INT_MAX 2019-08-19 12:58:54 -07:00
Nick Magerko
edf2abf106 Fix fall-through case 2019-08-19 12:32:43 -07:00
Nick Magerko
dffbac5f89 Add --size-hint=# option 2019-08-19 11:38:49 -07:00
Ed Maste
b81d7cc6a0 remove extraneous doubled ;s 2019-08-15 21:17:06 -04:00
W. Felix Handte
a42bbb4e05 Fix Buffer Overflow in Legacy (v0.3) Raw Literals Decompression 2019-08-15 14:28:30 -04:00