facebook/zstd
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,487 Commits 2 Branches 0 Tags
Commit Graph

7487 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Terrell
e0d6daabac Fix Appveyor failure 2019-11-19 11:12:26 -08:00
Bimba Shrestha
8f0c2d04c8 Going back to original flow but removing else return 2019-11-19 10:03:07 -08:00
Bimba Shrestha
b032a6e1d9 Updated some more outdated bounds 2019-11-19 09:55:13 -08:00
W. Felix Handte
722149cf2b Easy: Update Comment on ZSTD_initDStream() 2019-11-19 01:57:15 -05:00
W. Felix Handte
b5fb2e7c12 Add Tool to Diagnose Whether Corrupt Blobs are Plausibly Bit-Flips 
I spend an increasing amount of my time looking at "Corrupted block detected"
failures in decompression. Not infrequently, I suspect that it is the result
of hardware failure, and that the blob has become bit-flipped or otherwise
corrupted somewhere along the line.

For that reason I was motivated to write a little tool to inspect blobs that
fail to decompress, to try modifying them, and then check whether they
decompress successfully. This seems like potentially a generally useful tool,
so I figured it might be worth putting in `contrib/`.
 2019-11-19 01:53:36 -05:00
Nick Terrell
6a7f65117e
Merge pull request #1866 from legrosbuffle/dev 
Optimized loop bounds to allow the compiler to unroll the loop.
 2019-11-18 16:16:30 -08:00
Nick Terrell
a839d6852c
Merge pull request #1888 from senhuang42/superblocks_fixed 
RLE test and re-enable RLE in main compression loop
 2019-11-18 16:09:33 -08:00
Nick Terrell
76e661fa04
Merge pull request #1889 from vtorri/dev 
shared library: rename import library with .dll.a extension
 2019-11-18 16:07:28 -08:00
Bimba Shrestha
80586f5e80 Reversing condition order and forwarding error 2019-11-18 13:53:55 -08:00
Sen Huang
bc3e21578d No margin on RLE test size check 2019-11-18 16:39:16 -05:00
Bimba Shrestha
dade64428f Output regular uncompressed block when compressSequences fails 2019-11-18 08:43:14 -08:00
Bimba Shrestha
2d5d961a60 Typo in comment 2019-11-15 19:00:53 -08:00
Bimba Shrestha
dba767c0bb Leaving room for checksum 2019-11-15 18:44:51 -08:00
Sen Huang
db8efbfe7d Updated comment to reflect actual compression behavior 2019-11-15 16:11:14 -05:00
Vincent Torri
6b5c10b48c shared library: rename import library with .dll.a extension 
mort of open source project are using this extension for the import library.
The Win32 linker is supporting this extension, see
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/Using_ld_the_GNU_Linker/win32.html
section "direct linking to a dll"
 2019-11-15 19:46:06 +01:00
Vincent Torri
a2d9dbd53c Merge remote-tracking branch 'upstream/master' into dev 2019-11-15 19:37:15 +01:00
Sen Huang
75c34684c0 Modified existing RLE test to take compressed size into account 2019-11-15 12:26:48 -05:00
Clement Courbet
b3c9fc27b4 Optimized loop bounds to allow the compiler to unroll the loop. 
This has no measurable impact on large files but improves small file
decompression by ~1-2% for 10kB, benchmarked with:

head -c 10000 silesia.tar > /tmp/test
make CC=/usr/local/bin/clang-9 BUILD_STATIC=1 && ./lzbench -ezstd -t1,5 /tmp/test
 2019-11-15 08:27:05 +01:00
Sen Huang
d9646dcbb5 Fixed main compression logic changes 2019-11-14 19:39:09 -05:00
Yann Collet
4b1ac69f19
Merge pull request #1868 from senhuang42/superblocks_fixed 
Superblocks rebased for merge
 2019-11-14 13:31:34 -08:00
John Ericson
10552eaffe Fix MinGW cross install from Unix 
Needed a bunch of `$(EXT)`
 2019-11-14 14:19:07 -05:00
John Ericson
0ede342acc Fix MinGW cross build from Unix 
generate_res.bat seems rather pointless, am I missing something? I just
inlined it into the Makefile.
 2019-11-14 14:19:06 -05:00
Bimba Shrestha
982830b081 Fixing hashLog error 2^26=64Mib 2019-11-14 10:48:55 -08:00
Sen Huang
c26d32c91c Change superblock #include to be last 2019-11-14 13:12:17 -05:00
Yann Collet
d67742bc5d
Merge pull request #1858 from senhuang42/dictionary_header_size 
Method to get dictionary header size
 2019-11-14 09:44:07 -08:00
Yann Collet
2ba0167a7d
Merge pull request #1884 from facebook/specMBS 
Specs. clarifications for Block_Maximum_Size
 2019-11-14 09:32:07 -08:00
Yann Collet
098b36e9ab clarifications for Block_Maximum_Size 
as a follow up of #1882
 2019-11-13 09:50:15 -08:00
John Ericson
106278e7e5 Use lowercase windows.h 
Helps for cross builds, doesn't matter on Windows itself.
 2019-11-12 19:03:01 -05:00
Yann Collet
7d85f354fd
Merge pull request #1875 from ryandesign/c89 
Change c90 to c89
 2019-11-11 11:00:10 -08:00
Nick Terrell
cabbb1e99e
Merge pull request #1874 from terrelln/adjust-fix 
Fix parameter selection and adjustment with srcSize == 0
 2019-11-08 10:57:59 -08:00
Sen Huang
c85d10d0ea Remove mixed declarations 2019-11-08 13:57:26 -05:00
Sen Huang
d9c475f3b3 Fix static analyze error, use proper bounds for dictEnd 2019-11-08 13:57:26 -05:00
Sen Huang
d06b90692b Move asserts to loadZstdDictionary() 2019-11-08 13:57:26 -05:00
Sen Huang
b39149e156 Expose ZSTD_reset_compressedBlockState() to shared API 2019-11-08 13:57:26 -05:00
Sen Huang
6ce335371b Add error forwarding to loadCEntropy(), make check for dictSize >= 8 from bad merge 2019-11-08 13:57:26 -05:00
Sen Huang
4a61aaf368 Remove redundant comment 2019-11-08 13:57:26 -05:00
Sen Huang
c787b351ea Use ZSTD Error codes, improve explanation of ZSTD_loadCEntropy() and ZSTD_loadDEntropy() 2019-11-08 13:57:26 -05:00
Sen Huang
04fb42b4f3 Integrated refactor into getDictHeaderSize, now passes tests 2019-11-08 13:57:26 -05:00
Sen Huang
0bcaf6db08 First working pass at refactor of loadZstdDictionary() 2019-11-08 13:57:26 -05:00
Sen Huang
4b141b63e0 Revert "Move decompress symbols into zstd_internal.h, remove dependency" 
This reverts commit a152b4c67a5266f611db4a2eac4a79003852a795.
 2019-11-08 13:57:26 -05:00
Sen Huang
84404cff6e Move decompress symbols into zstd_internal.h, remove dependency 2019-11-08 13:57:26 -05:00
Sen Huang
341e0641ed Checks malloc() for failure, returns 0 if so 2019-11-08 13:57:26 -05:00
Sen Huang
97b7f712f3 Change to heap allocation, remove implicit type conversion 2019-11-08 13:57:25 -05:00
Sen Huang
e1edc554a3 Added 2 unit tests: one for sanity, one for correctnesson fixed dict 2019-11-08 13:57:25 -05:00
Sen Huang
3c36a7f13a Add ZDICT_getHeaderSize() 2019-11-08 13:57:08 -05:00
Ryan Schmidt
6ddb783c3e Change c90 to c89 
c89 and c90 are the same language. Old compilers like gcc 4.2.1 don't
know the c90 alias.
 2019-11-07 11:04:31 -06:00
Nick Terrell
8c474f9845 Fix parameter selection and adjustment with srcSize == 0 2019-11-07 08:58:43 -08:00
Yann Collet
2ed5344e84 added target zstd-dictBuilder 
check that the dictBuilder can be compiled
with only lib/common and lib/compress as dependencies.
 2019-11-06 15:46:58 -08:00
Yann Collet
9df49dc50a Visual compiler bug work-around 2019-11-06 15:23:44 -08:00
Yann Collet
be34969e5c minor man page fix 2019-11-06 15:07:35 -08:00