Commit Graph

1092 Commits (cdff19c4b33186315902c4e2991c182a0a21f579)

Author SHA1 Message Date
Yann Collet d1733f7417 fixed crc bug in rare timing conditions within bench.c 2016-08-21 01:04:46 +02:00
Yann Collet 8baf78a291 minor coding style 2016-08-20 13:04:20 +02:00
Yann Collet 1bee2d5e08 slight decompression speed improvement 2016-08-20 02:59:04 +02:00
Yann Collet 0cfe2ec2fd sync fse version 2016-08-20 00:26:26 +02:00
Yann Collet af1960396b sync huff0 2016-08-19 19:38:19 +02:00
Yann Collet 7be46bf8f2 promoted streaming API to stable
(except _advanced() variants)
2016-08-19 18:39:36 +02:00
Yann Collet fdba57d513 update version number 2016-08-19 18:32:30 +02:00
Yann Collet da3fbcb302 Added ZDICT_getDictID() 2016-08-19 14:23:58 +02:00
Yann Collet a5dbf9f629 Merge pull request #297 from borzunov/dev
Export functions related to dictionary compression from DLL
2016-08-18 15:05:01 +02:00
Yann Collet 49d105cfcf better warning and error messages in case of dictionary training failure (#292) 2016-08-18 15:02:11 +02:00
Alexander Borzunov 0f6f17a14f Rename ZSTDLIB_API to ZDICTLIB_API in zdict.h 2016-08-18 16:47:06 +05:00
Alexander Borzunov 1f48382b1a Export functions related to dictionary compression from DLL 2016-08-18 16:12:49 +05:00
Yann Collet e80d15304a Merge pull request #296 from inikep/Other
Other
2016-08-18 11:48:48 +02:00
inikep a7bb322a93 removed never referenced functions 2016-08-18 10:30:21 +02:00
Yann Collet 18442c1482 minor refactoring 2016-08-18 01:40:32 +02:00
Yann Collet c411902230 fixed g++ conversion warning 2016-08-17 01:50:54 +02:00
Yann Collet 53e17fbd5e updated streaming API 2016-08-17 01:39:22 +02:00
Yann Collet 655393cc72 updated doc for streaming API 2016-08-16 15:11:28 +02:00
Yann Collet 104e5b072d added : streaming decompression API 2016-08-16 15:11:28 +02:00
Yann Collet 5a0c8e2439 new streaming API (compression) 2016-08-16 15:11:27 +02:00
Yann Collet ba92046031 Merge branch 'dev' of github.com:Cyan4973/zstd into dev 2016-08-11 22:10:04 +02:00
Yann Collet e9b414d825 fixed msan warning (#281) 2016-08-11 22:09:09 +02:00
inikep 5f49eba512 added usage of rep[0]-1 for the optimal parser 2016-08-10 15:01:53 +02:00
inikep 98e08cbe34 fixed: tree not updated after finding very long rep matches 2016-08-10 15:00:30 +02:00
inikep 038d1497c9 fixed compilation with Visual Studio 2005 2016-08-10 14:30:10 +02:00
inikep 48849f86f0 fixed compilation with Intel Compiler with Windows 2016-08-10 14:26:35 +02:00
Yann Collet 1ea5622a32 updated xxhash 2016-08-10 09:40:08 +02:00
Yann Collet 666398e7ed added : xxhash namespace enforced from xxhash.h.
added : xxhash namespace test.
removed : -DXXH_NAMESPACE
2016-08-10 08:16:51 +02:00
Yann Collet 8ded0b84aa update xxhash to v0.6.2 2016-08-10 07:40:40 +02:00
Yann Collet 280f9a8754 minor comment 2016-08-08 00:44:00 +02:00
Yann Collet e0b4a2d40f fixed dictionary generation, reported by Bartosz Taudul 2016-08-03 03:36:03 +02:00
Yann Collet ae40b18d55 bumped library number 2016-08-03 01:59:21 +02:00
Yann Collet 0763905f44 ZSTD_compress_usingCDict() correctly provides original size by default in frame header
Fixed dictionary examples
2016-08-03 01:57:57 +02:00
Yann Collet bf2bc112bb bench : controlled display update when loading lot of files 2016-08-02 23:48:13 +02:00
Yann Collet 346efccc35 fixed doc typo 2016-08-02 14:26:00 +02:00
Yann Collet f116e87f59 fixed analyzer warning 2016-08-01 19:15:18 +02:00
Yann Collet 9ba929f1d4 Merge branch 'dev' of github.com:Cyan4973/zstd into dev 2016-08-01 02:26:59 +02:00
Yann Collet 3ca750372d updated doc (#269) 2016-08-01 02:26:20 +02:00
Yann Collet c55eb18c11 Merge pull request #267 from inikep/dev08
fixed ZSTD_compressBlock_opt_extDict_generic
2016-07-31 22:00:16 +02:00
inikep 056df510aa fixed ZSTD_compressBlock_opt_extDict_generic 2016-07-31 20:08:53 +02:00
Yann Collet 917fe188f1 Implemented repOffset "minus 1" on ll==0 2016-07-31 04:01:57 +02:00
Yann Collet 2a2ba3691c Merge pull request #266 from jrmarino/master
Enable build on FreeBSD ports (includes DragonFly BSD) [dev branch]
2016-07-31 02:21:26 +02:00
jrmarino 0d07ec0c0c Enable build on FreeBSD ports (includes DragonFly BSD)
Zstd has been introduced to FreeBSD ports
(http://www.freshports.org/archivers/zstd/) which DragonFly BSD also
uses.  FreeBSD and DragonFly use the install targets (albeit modified in
some cases) so they must be added to the associated Makefile filters.
2016-07-30 19:11:15 -05:00
Yann Collet 8cebfd1d26 fix attempt on test-zstd-speed 2016-07-31 01:59:23 +02:00
Yann Collet 66f69e58d2 restore decompression speed on fizzle 2016-07-30 15:32:47 +02:00
Yann Collet 3b2bd1d11c zstd_opt uses same tables as zstd_compress 2016-07-30 13:21:41 +02:00
Yann Collet f714f59c16 fixed visual warning 2016-07-30 12:05:28 +02:00
Yann Collet 761f8dbbd2 back to normal table cell copy 2016-07-30 11:43:53 +02:00
Yann Collet 3c6b808870 minor decompression speed gains 2016-07-30 03:20:47 +02:00
Yann Collet 70a9ff4af3 fixed too large selectivity level, reported by Ilona Papava 2016-07-30 01:09:14 +02:00
Yann Collet c0ce4f1211 slightly improved compression speed 2016-07-30 00:55:13 +02:00
Yann Collet ed57d8530a new seqStore 2016-07-29 21:22:17 +02:00
Yann Collet 6b615d32cd Updated API comments, following suggestions by Bryan O'Sullivan 2016-07-29 19:40:37 +02:00
Yann Collet c00d30fbe4 Merge pull request #264 from inikep/dev08
Dev08
2016-07-29 17:42:30 +02:00
inikep 6b68ba2079 zstd_opt.h: fixed checking of rep codes (2) 2016-07-29 16:45:39 +02:00
inikep 59b86fc141 zstd_opt.h: fixed checking of rep codes 2016-07-29 11:00:33 +02:00
Yann Collet 6a82f0f8bf minor comments 2016-07-29 00:55:45 +02:00
Yann Collet ffa7d0ac1e clarified comment 2016-07-28 21:01:17 +02:00
Yann Collet 4c5bbf64f9 fixed : frame concatenation without checksum 2016-07-28 20:30:25 +02:00
Yann Collet 60ba31c570 zbuff uses ZSTD_compressEnd() 2016-07-28 19:55:09 +02:00
Yann Collet 16e73033ad introduced stage zbf_end 2016-07-28 16:32:34 +02:00
Yann Collet 62470b4bab Changed ZSTD_compressEnd() 2016-07-28 15:29:08 +02:00
Yann Collet e7bf9156d1 Clarified API comments, from suggestions by ‎Bryan O'Sullivan‎ 2016-07-28 05:00:57 +02:00
Yann Collet d469a98c01 Clarified API comments, from suggestions by ‎Bryan O'Sullivan‎ 2016-07-28 04:55:03 +02:00
Yann Collet 19c1002e46 applied ZSTD_compressContinueThenEnd() 2016-07-28 01:25:46 +02:00
Yann Collet 5b56739b63 created ZSTD_compressContinueThenEnd() 2016-07-28 01:17:22 +02:00
Yann Collet c991cc1828 new frame end, 32-bits checksums 2016-07-28 00:55:43 +02:00
Yann Collet d4180cad9c minor code refactoring 2016-07-27 21:21:36 +02:00
Yann Collet 731ef16fc1 minor code style refactoring 2016-07-27 21:05:12 +02:00
Yann Collet 4b9ca0a6b5 minor example variation 2016-07-27 19:53:19 +02:00
Yann Collet 4110534886 ZSTD_maxCLevel() is promoted to "stable" API (#254, by @FrancescAlted) 2016-07-27 15:09:11 +02:00
Yann Collet 55a8bea0b5 fixed dictionary generation 2016-07-27 14:48:47 +02:00
Yann Collet c154d9d6a2 better support for large dictionaries (> 128 KB) 2016-07-27 14:37:00 +02:00
Yann Collet 07626dfa51 improved dictbuilder notifications on selectivity 2016-07-27 13:28:46 +02:00
Yann Collet f796f7ab45 removed fastscan mode 2016-07-27 12:53:54 +02:00
Yann Collet dd25a27702 added tutorial warning messages for dictBuilder 2016-07-27 12:43:09 +02:00
inikep 003c7a8568 optimal parser: removed ZSTD_REP_INIT 2016-07-27 11:07:13 +02:00
Yann Collet 04cdd8660d Merge pull request #262 from ebiggers/misc_updates
Miscellaneous updates
2016-07-27 01:25:45 +02:00
Eric Biggers 0a55e7a0bb ZSTD_decompressFrame(): use remainingSize instead of iend - ip
Same behavior, but no need to have redundant variables.
2016-07-26 13:22:27 -07:00
Eric Biggers aa6c70bf60 ZSTD_decompressFrame(): pass up error code from ZSTD_decodeFrameHeader() 2016-07-26 13:22:27 -07:00
Eric Biggers e4d0265ea9 Replace remaining references to "direct mode" with "single segment mode" 2016-07-26 13:22:27 -07:00
Yann Collet d50f9db3ea Improved speed on clang and gcc -O2, thanks to @ebiggers ! (#263) 2016-07-26 21:30:35 +02:00
Yann Collet 7adc2328a3 fixed --test on zero-length files, reported by @amnilsson 2016-07-26 15:49:24 +02:00
inikep 4178f5c289 fixed gcc warning: always_inline function might not be inlinable 2016-07-25 21:17:45 +02:00
inikep fca90f8f60 legacy decoder for v0.7 format 2016-07-25 17:49:08 +02:00
Yann Collet cbc5e9dc19 fixes oob read 2016-07-24 18:02:04 +02:00
Yann Collet 38b75ddeb2 removed special case all-1 huffman distribution 2016-07-24 15:35:59 +02:00
Yann Collet 7ed5e33b89 minor comment changes 2016-07-24 14:26:11 +02:00
Yann Collet 10b9c13d07 fixed doc on cLevel default, reported by Oliver Lange 2016-07-24 01:21:53 +02:00
Yann Collet f8e7b5363f unified encoding types 2016-07-23 16:31:49 +02:00
Yann Collet 571a59034a changed enccoding type order : raw, rle, compressed, repeat-stats 2016-07-23 15:52:05 +02:00
Yann Collet c2e1a68d81 changed streamNb order to 1-4-4-4 2016-07-22 17:30:52 +02:00
Yann Collet 772d912c2f more complete support for literals repeat mode 2016-07-22 15:04:25 +02:00
Yann Collet 9f2d82d4a4 fixed : big-endian decoding 2016-07-22 14:37:10 +02:00
Yann Collet 32faf6c8e7 fixed conversion warnings 2016-07-22 14:37:09 +02:00
Yann Collet 5e45a5fbb3 force loop-align to 32 for zstd_decompress 2016-07-22 14:37:09 +02:00
Yann Collet 5288ac0cb7 changed filed order 2016-07-22 14:37:09 +02:00
Yann Collet 198e6aac44 Literals header fields use little endian convention 2016-07-22 14:37:09 +02:00
Yann Collet 6fa05a2371 cBlockSize uses little-endian convention 2016-07-22 14:37:09 +02:00
Yann Collet 7bf72bbf5e update header to v0.8 2016-07-22 14:37:09 +02:00
Yann Collet 5894ea8d01 updated cLevels 2016-07-22 14:36:46 +02:00
Yann Collet d5c5a77990 minor comments clarifications 2016-07-20 13:35:14 +02:00
Yann Collet 572b817be3 Merge pull request #253 from gymdis/heapmode_off_legacy_fix
Fix compile issue with ZSTD_LEGACY_SUPPORT=1 and ZSTD_HEAPMODE=0
2016-07-19 13:52:03 +02:00
Christopher Bergqvist 780a9fa857 Fix compile issue with ZSTD_LEGACY_SUPPORT=1 and ZSTD_HEAPMODE=0 2016-07-19 13:25:38 +02:00
Yann Collet cf05b9d477 ZSTD_getBlockSizeMax() 2016-07-18 16:52:10 +02:00
Yann Collet 16aa38b0e0 minor doc clarifications 2016-07-18 03:52:47 +02:00
Yann Collet 85f3919960 moved `zstd.h` to `/lib` 2016-07-17 20:42:21 +02:00
Yann Collet 9375590462 update version to v0.7.5 2016-07-17 16:44:18 +02:00
Yann Collet e557fd5e92 minor compression level corrections 2016-07-17 16:21:37 +02:00
Yann Collet d54b2d23b4 minor static assert for 32/64 bits system. Suggested by @ebiggers 2016-07-17 15:53:18 +02:00
Yann Collet 972e5806ee fixed : premature frame end on zero-sized raw block - reported by @ebiggers 2016-07-17 15:39:24 +02:00
luben karavelov 10f999f856 Add legacy support for the low-level streaming API 2016-07-17 01:03:26 +02:00
Yann Collet 6cacd34d44 minor formatting changes 2016-07-15 17:58:13 +02:00
Yann Collet f6ff53cd4e implemented dictID reserved ranges 2016-07-15 17:03:38 +02:00
Yann Collet 98c8884999 added target zstd in root Makefile 2016-07-15 16:12:38 +02:00
Yann Collet 961b6a0e34 ZSTD_compressBlock() limits block size depending on windowLog parameter 2016-07-15 11:58:49 +02:00
Yann Collet 227cc39e15 improved efficiency for large messages with small dictionaries 2016-07-15 11:27:09 +02:00
Yann Collet ea2ecdc315 fixed issue with small dictionary 2016-07-14 23:27:31 +02:00
Yann Collet e9ed5cdc94 fixed minor coverity warning 2016-07-14 21:02:57 +02:00
Yann Collet b23e1ce319 removed debugging traces 2016-07-14 17:46:38 +02:00
Yann Collet 17508f1a16 fixed a few minor coverity warnings 2016-07-14 17:18:20 +02:00
Yann Collet 8847238cac simplified ZSTD_estimateCCtxSize() 2016-07-14 17:05:38 +02:00
Yann Collet 69c2cdb45c fixed conversion warning 2016-07-14 16:52:45 +02:00
Yann Collet 5e80dd3261 fixed minor coverity warnings 2016-07-13 19:21:57 +02:00
Yann Collet 3c174f4da9 fixed minor coverity warning 2016-07-13 17:25:53 +02:00
Yann Collet 2b1a3638e6 changed macro name to ZSTDCLI_CLEVEL_DEFAULT 2016-07-13 15:16:00 +02:00
Yann Collet 3c242e79d3 updated compression levels table 2016-07-13 14:56:24 +02:00
Yann Collet fbc69f8649 changed for #245 2016-07-13 13:52:58 +02:00
Yann Collet eed2081e55 fixed conversion warning 2016-07-12 15:11:40 +02:00
Yann Collet a43a854cdb updated paramgrill 2016-07-12 13:42:10 +02:00
Yann Collet 73d74a05b9 fixed dfast strategy 2016-07-12 13:03:48 +02:00
Yann Collet 45dc35628c first version of doubleFast 2016-07-12 09:47:31 +02:00
Yann Collet d158c35e9f added ZSTD_estimateDCtxSize() 2016-07-11 13:46:25 +02:00
Yann Collet 8e0ee681b8 added ZSTD_sizeofDCtx() 2016-07-11 13:09:52 +02:00
Yann Collet 3ae543ce75 added ZSTD_estimateCCtxSize() 2016-07-11 03:12:17 +02:00
Yann Collet 25c506601c promote ZSTD_getDecompressedSize() to stable API 2016-07-10 01:46:18 +02:00
Yann Collet 3b6ae77e15 comment clarification 2016-07-08 23:42:22 +02:00
Yann Collet 722e14bb65 fixed compilation error in decompression module 2016-07-08 19:22:16 +02:00
Yann Collet bd10607063 updated spec 2016-07-08 19:16:57 +02:00
Yann Collet c5fb5b7fcd support offset > 128 MB 2016-07-08 13:13:37 +02:00
Yann Collet ed3845d3fa introduced ZSTD_WINDOWLOG_MAX_32 (#239), suggested by @GregSlazinski 2016-07-08 12:57:10 +02:00
Yann Collet 26f681451f updated doc 2016-07-08 11:45:08 +02:00
Yann Collet 19c27d27f1 simplified legacy functions, no longer need magic number 2016-07-07 14:40:13 +02:00
Yann Collet e72efeb0a1 removed "error_public.h" dependency from "zstd.h" 2016-07-07 14:17:40 +02:00
Yann Collet 974f52fc5d Added "dictionary decompression" example 2016-07-07 14:08:00 +02:00
Yann Collet e09d38e921 removed `mem.h` dependency from `zbuff.h` (experimental section) 2016-07-07 13:17:37 +02:00
Yann Collet f323bf7d32 added : ZSTD_getDecompressedSize() 2016-07-07 13:14:21 +02:00
Yann Collet 52c04fe58f removed `mem.h` dependency from `zstd.h` (experimental section) 2016-07-07 11:53:18 +02:00
Yann Collet f246cf5423 ZSTD_decompress_usingDDict() compatible with Legacy mode 2016-07-06 20:32:27 +02:00
Yann Collet 29652e2618 sample set limitation closer to 2 GB 2016-07-06 16:25:46 +02:00
Yann Collet 99b045b70a dictBuilder protection vs huge sample sets (>2 GB) 2016-07-06 16:12:38 +02:00
Yann Collet 445d49d898 fixed conversion warning 2016-07-06 13:27:22 +02:00
Yann Collet a295b3170f fixed conversion warning 2016-07-06 13:13:12 +02:00
Yann Collet 517e1ba623 fixed dictBuilder issue with HC levels. Reported by Bartosz Taudul. 2016-07-06 12:35:09 +02:00
Yann Collet fe07eaa972 simplified ZSTD_decodeSequence() 2016-07-06 02:25:44 +02:00
Yann Collet 9ca73364e6 updated spec 2016-07-05 10:53:38 +02:00
Yann Collet f9cac7a734 Added GNU separator `--`, to specifies that all following arguments are necessary file names (and not commands). Suggested by @chipturner (#230) 2016-07-04 18:18:24 +02:00
Yann Collet 23f05ccc6b updated specifications 2016-07-04 16:13:11 +02:00
Yann Collet d916c908e0 updated doc 2016-07-04 00:42:58 +02:00
Yann Collet 698cb63305 Updated specifications 2016-07-03 18:49:35 +02:00
Yann Collet d57dffbe76 ZSTD_storeSeq takes an U32 as offset type 2016-07-03 01:48:26 +02:00
Yann Collet 302ff036f6 simplified repcodes for lazy_extDict 2016-07-03 01:28:16 +02:00
Yann Collet 9634f67107 fix lazy parser 2016-07-03 01:23:58 +02:00
Yann Collet 92d75667e4 fix for fast mode 2016-07-03 01:10:53 +02:00
Yann Collet 5e734ad09b revert fix 2016-07-02 23:55:34 +02:00
Yann Collet 0d5bf8f06f fixed risk of segfault on very large files (multiple GB) 2016-07-02 21:39:47 +02:00
Yann Collet 2fa9904844 update specification and comments 2016-07-01 20:55:28 +02:00
Yann Collet c093208ab8 fix : potential leak (#229) 2016-06-30 14:07:30 +02:00
Yann Collet 6c6e1751f6 use ZSTD_getParams() to simplify code 2016-06-27 15:28:45 +02:00
Yann Collet 3d2cd7f816 Introduced ZSTD_getParams()
bench now uses ZSTD_createCDict_advanced()
2016-06-27 15:12:26 +02:00
Yann Collet 529d9c7dee updated version to v0.7.2 2016-06-27 10:03:10 +02:00
Yann Collet d4f4e58ee1 fixed ZSTD_decompressBlock() using multiple blocks 2016-06-27 01:31:35 +02:00
Yann Collet 63b5e7a2ea Improved comments 2016-06-26 17:42:15 +02:00
Yann Collet 3755eb8fea fixed strict-aliasing warning on gcc6 2016-06-22 13:15:53 +02:00
Yann Collet 23042929da Fixed : dictBuilder fails if first sample is too small 2016-06-22 11:05:34 +02:00
Yann Collet 391a128794 fix : segfault in command line during automatic overwrite protection mode 2016-06-21 17:06:25 +02:00
Yann Collet bda68c253b refactored ZBUFF_compressEnd() for better maintainability 2016-06-21 15:18:11 +02:00
Yann Collet aa29226b7c fix : ZBUFF_compressEnd() gives right amount remaining to flush, including future epilogue 2016-06-21 14:04:57 +02:00
Yann Collet f15c1cb00c Fixed : ZBUFF_compressEnd() called multiple times with too small dst buffer (#206) 2016-06-21 13:11:48 +02:00
Yann Collet a49e066b26 clarified comments on `ZSTD_compressContinue()` 2016-06-21 11:54:03 +02:00
Yann Collet d4f38d0dcd updated library to v0.7.1 2016-06-21 10:15:43 +02:00
Yann Collet 22d76322ce minor refactor 2016-06-21 08:01:51 +02:00
Yann Collet a436a529bc minor : fast_extDict does no longer skip first byte 2016-06-20 23:34:04 +02:00
Yann Collet 4623d11571 new correction, less extreme replacement value 2016-06-20 19:15:37 +02:00
Yann Collet 5477cc25f7 fixed corruption error related to inter-blocks rep-offset 2016-06-20 18:31:25 +02:00
Yann Collet e4811ba761 Modified : ZSTD_createDDict() accepts dictionary < 8 bytes in pure content mode (reported by @chipturner) 2016-06-19 23:06:54 +02:00
Yann Collet 06d9a73b48 minor refactor, using `WILDCOPY_OVERLENGTH` macro instead of hard-coded 8 2016-06-19 14:27:21 +02:00
Yann Collet 19cab46f2f Joined `seqStore` initialization at dispatch point 2016-06-17 12:54:52 +02:00
Yann Collet 510cff3570 minor comment change 2016-06-16 16:39:55 +02:00
Yann Collet 4948f270b3 make room for reserved "information bit" in frame header 2016-06-16 15:38:51 +02:00
Yann Collet 23ba41533a Fixed zstd_opt encoding error with repeat-offsets 2016-06-16 13:20:46 +02:00
Yann Collet 80d033fb43 fixed ptr arithmetic warning 2016-06-16 01:41:50 +02:00
Yann Collet ad39b7a718 zdict stores standard rep-offset.
It can use custom ones, but the proper formula and impact on statistics is not done yet.
2016-06-16 01:14:41 +02:00
Yann Collet 736d419289 strengthened dict loading on decompresson side 2016-06-16 01:05:04 +02:00
Yann Collet 8e36a9c169 decoder restores repOffsets from dictionary 2016-06-16 01:05:04 +02:00
Yann Collet 52a0622beb RepsCodes are saved into Dict
(uncomplete : need decompression to regenerate them)
2016-06-16 01:05:04 +02:00
Yann Collet efd0b4993a fixed fuzzer error (inter-block repeated offsets) 2016-06-16 00:53:56 +02:00
Yann Collet 9b998e4d08 Fixed decompression of literals in dictionary mode 2016-06-15 23:11:20 +02:00
Yann Collet d059092897 fixed conversion warnings 2016-06-14 15:34:24 +02:00
Yann Collet 45c03c564f fixed corruption with inter-blocks repeated offsets 2016-06-14 13:46:11 +02:00