facebook/zstd
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,308 Commits 2 Branches 0 Tags
Commit Graph

9308 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann Collet
abc694f53e update man pages 2021-12-20 14:24:06 -08:00
Yann Collet
1219b7f3e3 updated changelog for v1.5.1 2021-12-20 12:28:27 -08:00
Yann Collet
24960134d9
Merge pull request #2940 from facebook/revert-2885-limit-level-32bit-systems 
Revert "Limit `ZSTD_maxCLevel` to 21 for 32-bit binaries."
 2021-12-20 12:27:28 -08:00
Yann Collet
a9e43b37d0
Revert "Limit ZSTD_maxCLevel to 21 for 32-bit binaries." 2021-12-20 11:43:14 -08:00
Yann Collet
85bd1e663b
Merge pull request #2934 from facebook/lazy_rebalance 
rebalance lazy compression levels
 2021-12-20 11:37:44 -08:00
Yann Collet
2624652a32 Merge branch 'dev' into lazy_rebalance 2021-12-20 08:28:21 -08:00
Yann Collet
80a28f2078 update regression results 2021-12-20 07:54:57 -08:00
Yann Collet
9fc7f38e12 added target update_regressionResults 
to automate the creation of updated tests/regression/results.csv
 2021-12-16 21:46:14 -08:00
Yann Collet
01adddc3e0 update regression results 2021-12-16 20:43:23 -08:00
Yann Collet
7c7b9244d6 update regression results 2021-12-16 16:07:54 -08:00
Yann Collet
47466d2192 Merge branch 'dev' into lazy_rebalance 2021-12-16 14:46:21 -08:00
Yann Collet
9a32492730 updated regression results.csv 2021-12-16 14:39:30 -08:00
Yann Collet
925649b2d0 fixed version number for fast modes 2021-12-16 11:52:43 -08:00
Yann Collet
f829c32258 forgot the chainlog is effectively a "fake" value with rowHash 
the only value which makes sense is `hashlog-1`
as it mimics the real memory usage.
 2021-12-16 11:37:40 -08:00
Yann Collet
5c96146db3
Merge pull request #2931 from facebook/fix_rowlog 
fixed incorrect rowlog initialization
 2021-12-16 09:48:12 -08:00
Yann Collet
5c63096269 updated benchmark for v1.5.1 
answers #2764
 2021-12-16 04:46:21 -08:00
Yann Collet
8a724f91cb minor changelog update 2021-12-16 01:07:15 -08:00
Yann Collet
7a347769dc minor changelog update 2021-12-15 21:45:30 -08:00
Yann Collet
db1b408a2f rebalance lazy compression levels 2021-12-15 21:33:31 -08:00
Yann Collet
c8d6067615 fixed incorrect rowlog initialization 
the variable has only very limited usage,
being only used once at the beginning of the block for prefetching only,
hence the error had no impact on compression ratio.
 2021-12-15 14:37:05 -08:00
Elliot Gorokhovsky
c5f1e826ca
Merge pull request #2925 from embg/dict_training_sample_limit_size 
Allow user to specify memory limit for dictionary training
 2021-12-15 15:58:17 -05:00
Yann Collet
622fee6140 minor changelog update, for clarity 2021-12-15 11:56:11 -08:00
Yann Collet
eaf786242d
Merge pull request #2929 from facebook/sse_row_lazy 
simplify SSE implementation of row_lazy match finder
 2021-12-15 11:47:15 -08:00
Felix Handte
d5933a4009
Merge pull request #2930 from nolange/reduce_tables_to_8bit 
Reduce tables to 8bit
 2021-12-15 12:43:25 -05:00
Norbert Lange
2fbb1d10c1 Reduce bit tables to 8bit 
This saves some 1.7Kb in rodata section (x86_64, zstd tool),
while assembler code stays the same except
the type of a few load/extend instructions.

Should not have negative performance implications.
 2021-12-14 23:47:57 +01:00
Norbert Lange
99923dfc1a Add typedefs for 8bit (un)signed 
To make code more expressive, add U8 and S8 typedefs
 2021-12-14 23:47:57 +01:00
binhdvo
64205b7832
Fix performance degradation with -m32 (#2926) 2021-12-14 15:53:50 -05:00
Elliot Gorokhovsky
71c0c07c19 Allow user to specify memory limit for dictionary training 2021-12-14 14:29:01 -05:00
Felix Handte
5e2fede604
Merge pull request #2921 from felixhandte/neg-lvl-stagger-step 
Stagger Stepping in Negative Levels
 2021-12-14 14:13:57 -05:00
Yann Collet
05430b25a8 roll SSE implementation of row_lazy match finder 
mostly for maintenance convenience.

Performance wise, there is very little change,
slightly faster for slog 3 & 4,
neutral or very slightly negative for slot 5 & 6.
 2021-12-14 10:44:23 -08:00
W. Felix Handte
450fca9704 Update Regression Tests w/ New Sizes 2021-12-13 17:29:32 -05:00
W. Felix Handte
82a49c88f9 Increment Step by 1 not 2 
I couldn't find a good way to spread `ip0` and `ip1` apart when we accelerate
due to incompressible inputs. (The methods I tried slowed things down quite a
bit.)

Since we aren't splaying ip0 and ip1 apart (which would be like `0_1_2_3_`, as
opposed to the `01__23__` we were actually doing), it's a big ambitious to
increment `step` by 2. Instead, let's increment it by 1, which has the benefit
sliiightly improving compression. Speed remains pretty much unchanged.
 2021-12-13 16:59:33 -05:00
Nick Terrell
3e2a70b6fb
Merge pull request #2905 from 15596858998/dev_1205 
add test case
 2021-12-13 13:45:23 -08:00
W. Felix Handte
6ca5f42402 Rewrite step to Track Increment Between Pairs of Positions 
The position updates are rewritten from `ip[N] = ip[N-1] + step` to be
`ip[N] = ip[N-2] + step`. This lets us only deal with the asymmetric spacing
of gaps at setup and then we only have to keep a single `step` variable.

This seems to work quite well on GCC and Clang!
 2021-12-13 14:48:26 -05:00
W. Felix Handte
b8434cb754 Allow Templating ZSTD_fast Matchfinders on Acceleration (Lvl < -1) 2021-12-13 14:46:57 -05:00
Felix Handte
65404fe14a
Merge pull request #2923 from IAL32/patch-1 
typo: Small spelling mistake in example
 2021-12-13 13:15:21 -05:00
zx123123
c69d13eb99
Update playTests.sh 2021-12-13 08:58:42 +08:00
Adrian Castro
e0f9dc0dde
typo: Small spelling mistake in example 
Just a couple of characters:
`main` -> `may`
 2021-12-11 12:02:23 +01:00
Yann Collet
252ef866fb
Merge pull request #2922 from facebook/x32 
x32 compatibility
 2021-12-11 00:12:10 -08:00
Yann Collet
e1ab2200ff fixed x32 compatibility 2021-12-10 21:02:17 -08:00
Yann Collet
c94cda283c added x32 compatibility test 2021-12-10 20:56:20 -08:00
W. Felix Handte
ace6a7e746 Decompose step into Two Variables 
This avoids an additional addition, at the cost of an additional variable.
 2021-12-10 16:44:23 -05:00
W. Felix Handte
22501cd283 Stagger Application of stepSize in ZSTD_fast 
This replicates the behavior of @terrelln's `ZSTD_fast` implementation. That
is, it always looks at adjacent pairs of positions, and only applies the
acceleration every other position. This produces a more fine-grained
acceleration.
 2021-12-10 16:44:23 -05:00
Yann Collet
4cc5e2818a complete changelog with #2885 2021-12-09 09:53:45 -08:00
Yann Collet
c077b530a0
Merge pull request #2917 from facebook/change151 
Update changelog for v1.5.1
 2021-12-09 08:45:34 -08:00
Felix Handte
0c26d98c0d
Merge pull request #2910 from felixhandte/reject-irregular-dicts 
Reject Irregular Dictionary Files
 2021-12-09 11:44:37 -05:00
Yann Collet
3d738307b4 Update changelog for v1.5.1 2021-12-08 16:55:38 -08:00
Yann Collet
57383d2317
Merge pull request #2914 from facebook/xxhash081 
updated xxHash to latest v0.8.1
 2021-12-08 16:48:46 -08:00
Yann Collet
3ce265fea8 remove offending static assert lines 
no idea why visual + clang-cl + appveyor don't like them,
I've not been able to reproduce the issue locally,
but these static assert are very unlikely to deliver a useful signal,
I can't imagine a situation where they will be wrong,
and if they are, then a ton of other things will be broken way before reaching that point.
 2021-12-08 15:05:17 -08:00
Nick Terrell
8b40095b3f
Merge pull request #2916 from terrelln/issue-2906 
Remove possible NULL pointer addition
 2021-12-08 16:51:10 -05:00