Commit Graph

50 Commits (31533bacce265904ecc376206ebf52f4cbb2e0b5)

Author SHA1 Message Date
Yann Collet e42afbc6fa Comply with suggested comments by @terrelln
created FSE_CTABLE_SIZE() and FSE_DTABLE_SIZE()
2017-04-26 11:39:35 -07:00
Yann Collet 4f818182b8 clarified frame parameters for ZSTD_compress*_usingCDict()
created ZSTD_compressBegin_usingCDict_internal(),
which gives direct control to frame Parameters.
ZSTD_resetCStream_internal() now points into it.
2017-04-17 18:29:06 -07:00
Yann Collet 4bcc69b761 solves warnings when compiling with global XXH_STATIC_LINKING_ONLY
XXH_STATIC_LINKING_ONLY protection macro is intended to be triggered just before the include.
The main idea is to keep this setting local :
user module shall explicitly understand and accept the static linking restriction
which becomes transparent when triggering the macro at project level.
Global definition also triggers redefinition warnings for user modules which do locally define the macro.

This new version compiles lib and cli without warning when the macro is set globally.
That's not a scenario to be recommended, since it trades a local effect for a global one,
but it was easy enough to provide from zstd side.
2017-03-01 11:33:25 -08:00
Yann Collet 32dfae6f98 fixed Multi-threaded compression
MT compression generates a single frame.
Multi-threading operates by breaking the frames into independent sections.
But from a decoder perspective, there is no difference :
it's just a suite of blocks.

Problem is, decoder preserves repCodes from previous block to start decoding next block.
This is also valid between sections, since they are no different than changing block.

Previous version would incorrectly initialize repcodes to their default value at the beginning of each section.
When using them, there was a mismatch between encoder (default values) and decoder (values from previous block).

This change ensures that repcodes won't be used at the beginning of a new section.
It works by setting them to 0.
This only works with regular (single segment) variants : extDict variants will fail !
Fortunately, sections beyond the 1st one belong to this category.

To be checked : btopt strategy.
This change was only validated from fast to btlazy2 strategies.
2017-01-19 10:32:55 -08:00
Nick Terrell 064a143520 Fix execSequence wildcopy undefined behavior
execSequence relied on pointer overflow to handle cases where
`sequence.matchLength < 8`.  Instead of passing an `size_t` to
wildcopy, pass a `ptrdiff_t`.
2016-12-12 19:01:23 -08:00
Przemyslaw Skibinski 9ca65af810 zstd_opt.h: improved price function 2016-11-23 17:22:54 +01:00
Przemyslaw Skibinski 179555c1d1 working fullbench-dll 2016-11-15 18:05:46 +01:00
Nick Terrell eb7873a048 ZSTD_execSequence() accepts match in last 7 bytes
The zstd reference compressor will not emit a match in the last 7
bytes of a block.  The decompressor will also not accept a match
in the last 7 bytes.  This patch makes the decompressor accept a
match in the last 7 bytes.
2016-10-25 21:24:15 -07:00
Yann Collet 51f4d566c2 small decompression speed boost for very small data 2016-09-22 15:57:28 +02:00
Yann Collet 95d07d7447 introduced CHECK_E 2016-09-06 16:38:51 +02:00
Yann Collet 3e21ec5b01 introduced CHECK_F 2016-09-06 15:36:19 +02:00
Yann Collet 5c956d593c FORCE_INLINE common definition 2016-09-06 15:05:19 +02:00
Yann Collet 4ded9e591c added boilerplate 2016-08-30 11:06:28 -07:00
Yann Collet 23b6e05d8e ZSTD_malloc() and ZSTD_free(), to simplify customMem 2016-08-28 21:05:43 -07:00
inikep e416e30019 remove unnecessary comments 2016-08-24 17:32:09 +02:00
inikep 4e90f6c1e0 removed ZSTD_LOG_ENCODE and ZSTD_LOG_BLOCK 2016-08-24 17:24:11 +02:00
inikep 83388e109f removed ZSTD_LOG_PARSER 2016-08-24 17:22:20 +02:00
inikep 8a36f8527c removed stats in debug mode 2016-08-24 17:19:12 +02:00
inikep 5f49eba512 added usage of rep[0]-1 for the optimal parser 2016-08-10 15:01:53 +02:00
Yann Collet 280f9a8754 minor comment 2016-08-08 00:44:00 +02:00
Yann Collet 8cebfd1d26 fix attempt on test-zstd-speed 2016-07-31 01:59:23 +02:00
Yann Collet c0ce4f1211 slightly improved compression speed 2016-07-30 00:55:13 +02:00
Yann Collet ed57d8530a new seqStore 2016-07-29 21:22:17 +02:00
Yann Collet c00d30fbe4 Merge pull request #264 from inikep/dev08
Dev08
2016-07-29 17:42:30 +02:00
Yann Collet c991cc1828 new frame end, 32-bits checksums 2016-07-28 00:55:43 +02:00
Yann Collet c154d9d6a2 better support for large dictionaries (> 128 KB) 2016-07-27 14:37:00 +02:00
inikep 003c7a8568 optimal parser: removed ZSTD_REP_INIT 2016-07-27 11:07:13 +02:00
Yann Collet f8e7b5363f unified encoding types 2016-07-23 16:31:49 +02:00
Yann Collet 571a59034a changed enccoding type order : raw, rle, compressed, repeat-stats 2016-07-23 15:52:05 +02:00
Yann Collet 2fa9904844 update specification and comments 2016-07-01 20:55:28 +02:00
Yann Collet 63b5e7a2ea Improved comments 2016-06-26 17:42:15 +02:00
Yann Collet 52a0622beb RepsCodes are saved into Dict
(uncomplete : need decompression to regenerate them)
2016-06-16 01:05:04 +02:00
Yann Collet 4266c0a2fd adding inter-blocks rep-offsets 2016-06-14 01:49:25 +02:00
Yann Collet 0974f681a4 completed `.gitignore` 2016-06-10 14:44:16 +02:00
Yann Collet 9dd12742f3 `litBlockType_t` is an `enum` 2016-06-10 00:12:26 +02:00
Yann Collet 673f0d7cdc new frame format, allowing custom window size 2016-06-06 00:26:38 +02:00
Yann Collet 49bb0041af removed `ZSTD_highbit()` from `zstd_internal.h`, as it is only used by `zstd_compress.c` 2016-06-04 20:17:38 +02:00
Yann Collet d3b7f8d21f Merged `zstd_static.h` into `zstd.h` . Now requires `ZSTD_STATIC_LINKING_ONLY` macro 2016-06-04 19:47:02 +02:00
inikep 2a74609b90 zlibWrapper: ZWRAP_createCCtx and ZWRAP_freeCCtx use custom memory allocation functions 2016-06-03 14:53:51 +02:00
inikep 3763c77f6b defaultCustomNULL replaced with defaultCustomMem 2016-06-03 13:28:20 +02:00
inikep db2f540414 added defaultCustomNULL 2016-06-03 12:56:56 +02:00
inikep ff9114aee3 zlibWrapper: added support for custom memory allocation functions 2016-06-02 16:52:36 +02:00
inikep c4807f4d2f default custom allocation functions moved to zstd_internal.h 2016-06-02 15:11:39 +02:00
inikep a6b942018d Merge remote-tracking branch 'refs/remotes/origin/dev' into dev070
# Conflicts:
#	.travis.yml
#	Makefile
#	lib/common/zstd_static.h
#	programs/Makefile
#	projects/VS2008/zstd/zstd.vcproj
#	projects/VS2008/zstdlib/zstdlib.vcproj
#	projects/cmake/lib/CMakeLists.txt
#	projects/cmake/programs/CMakeLists.txt
2016-06-01 00:07:09 +02:00
inikep f772bf54a5 support for skippable frames 2016-05-31 12:43:46 +02:00
Yann Collet c46fb924df added dictionary ID (incomplete) 2016-05-29 05:01:04 +02:00
inikep fb5df613cf zstd_stats.h included only in debug mode 2016-05-24 15:36:37 +02:00
inikep 35b891c5b0 zstd_stats.h updated to v0.6 2016-05-20 19:42:20 +02:00
Yann Collet ddb8ebd5b3 Stricter gnu90 tests 2016-05-05 04:59:53 +02:00
inikep 23a0889301 separation of lib/ into common/, compress/, decompress/, dictBuilder/, legacy/ 2016-04-22 12:43:18 +02:00