facebook/zstd
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,073 Commits 2 Branches 0 Tags
Commit Graph

9073 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Terrell
a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files 
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
 2021-03-30 10:30:43 -07:00
sen
84ccb81e7c
Merge pull request #2561 from senhuang42/longlength_enum 
Add enum for representing long length ID
 2021-03-26 15:55:12 -04:00
Sen Huang
b1a43455f8 Add enum for representing long length ID 2021-03-26 10:41:09 -07:00
sen
ab216bc2c5
Merge pull request #2559 from senhuang42/add_dict_regression_tests_backup 
Add different dict modes to compression ratio regression test, update results.csv
 2021-03-25 19:26:06 -04:00
Sen Huang
bbbd578f45 Update results.csv 2021-03-25 11:16:37 -07:00
sen
4fe2e7ae14
Merge pull request #2558 from senhuang42/msan_block_splitter_fix 
Fix block splitter minor MSAN warning.
 2021-03-25 13:51:43 -04:00
Sen Huang
f27e326456 Restrict dictmode regression tests only to advanced API, fix some compiler warnings 2021-03-25 10:39:08 -07:00
Sen Huang
1cadf86b39 Add tests to regression tests for dict 2021-03-25 10:39:08 -07:00
sen
b0407b9f0e
Merge pull request #2555 from senhuang42/default_clevel_func 
Add ZSTD_defaultCLevel() function to public API
 2021-03-25 13:07:28 -04:00
Sen Huang
ef4e26bda5 Add clang msan fuzz test to github actions 2021-03-25 09:11:43 -07:00
Sen Huang
2a907bf4aa Move lastCountSize into a returned struct, fix MSAN error 2021-03-25 09:11:15 -07:00
Sen Huang
e398744a35 Add ZSTD_defaultCLevel() function to public API 2021-03-25 08:04:00 -07:00
Nick Terrell
f8ac0ea7ef
Merge pull request #2539 from terrelln/linux-kernel-fixes 
Fixes for the next linux kernel patch version
 2021-03-24 10:34:29 -07:00
sen
bf542c8a8d
Merge pull request #2447 from senhuang42/block_splitter_v2 
Recursive block splitting
 2021-03-24 12:27:22 -04:00
Yann Collet
8889ca36fc
Merge pull request #2554 from facebook/fix2553 
fix man page typo
 2021-03-24 08:55:33 -07:00
Sen Huang
5b566ebe08 Rename *compressSequences*() functions for clarity 2021-03-24 08:21:29 -07:00
Sen Huang
0ef1f935b7 Add a fallback in case the total blocksize of split blocks exceeds raw block size 2021-03-24 08:21:29 -07:00
Sen Huang
c90e81a692 Enable block splitter by default when applicable 2021-03-24 08:21:29 -07:00
Sen Huang
e34332834a Clean up various functions, add debuglogging for estimate vs. actual sizes 2021-03-24 08:21:29 -07:00
Sen Huang
41c3eae6d9 Fix various fuzzer failures: repcode history, superblocks 2021-03-24 08:21:29 -07:00
senhuang42
0633bf17c3 Change 1.3.4 bugfix to be cross-compatible with superblocks and normal compression 2021-03-24 08:21:29 -07:00
senhuang42
eb1ee8686d Refactor buildSequencesStatistics() to avoid pointer increment for superblocks 2021-03-24 08:21:29 -07:00
senhuang42
e2bb215117 Add unit tests and fuzzer param 2021-03-24 08:21:09 -07:00
senhuang42
de52de1347 Add recursive block split algorithm 2021-03-24 08:21:09 -07:00
senhuang42
f06f6626ed Update function names for consistency 2021-03-24 08:20:54 -07:00
senhuang42
c56d6e49e8 Add block splitter to experimental params 2021-03-24 08:20:54 -07:00
senhuang42
2949a95224 Refactor block compression logic into single function 2021-03-24 08:20:54 -07:00
senhuang42
c05c090cc2 Centralize entropy statistics calculations to zstd_compress.c 2021-03-24 08:20:29 -07:00
Yann Collet
0f99a0d987 fix man page typo 
fix #2553 detected by @jwilk
 2021-03-24 05:55:04 -07:00
sen
c48889f097
Merge pull request #2538 from senhuang42/monotonicity_test 
Add memory monotonicity test over srcSize
 2021-03-22 16:54:34 -04:00
Nick Terrell
ebc2dfa821
Merge pull request #2524 from terrelln/huf-stack-reduction 
[huf] Reduce stack usage of HUF_readDTableX2 by ~972 bytes
 2021-03-22 12:37:54 -07:00
Nick Terrell
634bfd339f [FSE] Clean up workspace using dynamically sized struct 2021-03-22 11:07:07 -07:00
Yann Collet
8b97931ae1
Merge pull request #2550 from facebook/vvv_srcSize 
fix #2549
 2021-03-22 10:44:44 -07:00
Sen Huang
dff4a0e867 Make ZSTD_estimateCCtxSize_internal() loop through all srcSize parameter sets as well 2021-03-21 16:15:31 -07:00
Yann Collet
9fb4a42c7b fix #2549 2021-03-20 17:29:41 -07:00
Yann Collet
f5434663ea Refactor prefetching for the decoding loop 
Following #2545,
I noticed that one field in `seq_t` is optional,
and only used in combination with prefetching.
(This may have contributed to static analyzer failure to detect correct initialization).

I then wondered if it would be possible to rewrite the code
so that this optional part is handled directly by the prefetching code
rather than delegated as an option into `ZSTD_decodeSequence()`.

This resulted into this refactoring exercise
where the prefetching responsibility is better isolated into its own function
and `ZSTD_decodeSequence()` is streamlined to contain strictly Sequence decoding operations.
Incidently, due to better code locality,
it reduces the need to send information around,
leading to simplified interface, and smaller state structures.
 2021-03-19 15:48:17 -07:00
Yann Collet
515807182f
Merge pull request #2548 from SupervisedThinking/build_fix 
meson: fix build by adding missing files
 2021-03-19 15:20:14 -07:00
SupervisedThinking
edf2b1176d meson: fix build by adding missing files 
fixes https://github.com/facebook/zstd/issues/2519
 2021-03-19 19:52:45 +01:00
sen
eace4abc25
Merge pull request #2540 from senhuang42/fix_dds_supported 
Fix dedicated dict search isSupported() requirements.
 2021-03-17 23:25:53 -04:00
Nick Terrell
756bd59322 [huf][fse] Clean up workspaces 
* Move `counting` to a struct in `FSE_decompress_wksp_body()`
* Fix error code in `FSE_decompress_wksp_body()`
* Rename a variable in `HUF_ReadDTableX2_Workspace`
 2021-03-17 16:50:37 -07:00
ihsinme
a5bf09d764
simple fix for using bit operator. 
good day.
It seems to me that the developer intended to use a logical operator.
so I suggest a simple fix.
 2021-03-17 11:37:42 +03:00
Sen Huang
77ae664ba6 Fix ZSTD_dedicatedDictSearch_isSupported() requirements 2021-03-16 17:36:05 -07:00
senhuang42
386111adec Add a nbSeq argument to compressSequences() 
Refactor ZSTD_compressBlock_internal() to do the block header write within and add nbSeq argument to compressSequences()
 2021-03-16 14:04:22 -07:00
senhuang42
98764493cf Move block header write into compressBlock_internal() 2021-03-16 14:04:22 -07:00
Nick Terrell
ea288e0d8e [lib] Bump zstd version number 2021-03-16 11:47:27 -07:00
Nick Terrell
cd1551d261 [lib][tracing] Add ZSTD_NO_TRACE macro 
When defined, it disables tracing, and avoids including the header.
 2021-03-16 11:47:27 -07:00
Nick Terrell
7222614a19 [contrib][freestanding] Remove tracing support 
Remove tracing support from `freestanding.py` to keep things simple.
 2021-03-16 11:47:27 -07:00
Nick Terrell
e4b914e663 [contrib][linux] Expose zstd headers to avoid duplication 
Expose the zstd headers in `include/linux` to avoid struct duplication.
This makes the member names not follow Kernel style guidelines, and
exposes the zstd symbols. But, the LMKL reviewers are okay with that.
 2021-03-16 11:47:22 -07:00
Nick Terrell
49a9e070f5 [contrib][linux-kernel] Update test include stubs 
Update the test include stubs so they are able to run the current zstd
version in the kernel, so I can compare stack usage.
 2021-03-16 11:40:24 -07:00
Nick Terrell
d2dd35ae65 [contrib][linux-kernel] Fix unaligned.h 
Fix the `unaligned.h` shim in the tests that was causing corruption in
the tests. Note that this is a problem with the test shim, not the
kernel code.
 2021-03-16 11:40:24 -07:00