Commit Graph

431 Commits (2f7bbd65399669d9ee39a0cc7cb19d13f3001a3a)

Author SHA1 Message Date
Sean Purcell 042ba122ae Change g_displayLevel to int and fix DISPLAYUPDATE flush 2017-03-23 11:21:59 -07:00
Yann Collet 35c147ab6a Merge pull request #605 from iburinoc/term
Fix zstd not erroring on compressing to terminal without forceStdout
2017-03-17 14:02:48 -07:00
Yann Collet 0a189b63fe fix minor details in man page 2017-03-16 16:33:53 -07:00
Sean Purcell b8e52d3c83 Fix zstd not erroring on compressing to terminal without forceStdout 2017-03-16 16:06:19 -07:00
Nick Terrell aa8bcf360f Add xz and lzma support.
Finish feature started by @inikep.

* Add xz and lzma compression and decompression support to target `xzstd`.
* Fix bug in gzip decompression that silently accepted truncated files.
* Add gzip frame composition tests.
* Add xz/lzma compatibility tests.
* Add xz/lzma frame composition tests.
2017-03-13 18:45:41 -07:00
Yann Collet c1c040eae1 added gzip tests
also : made sure zstd --format=gzip -V
would fail if gzip compatibility is not supported
2017-03-01 16:49:20 -08:00
Przemyslaw Skibinski 684858e7b7 fix memory leaks 2017-02-21 18:17:24 +01:00
Yann Collet c09d16ba8c preset behavior for gzip, gunzip and gzcat
when zstd is called through a link named gzip, gunzip or gzcat,
provides the same behavior as the related program.
gzip compresses using --format=gz
both gzip and gunzip enable --rm by default
2017-02-14 10:45:19 -08:00
Przemyslaw Skibinski 970419535f fixed function name (2) 2017-02-14 09:47:29 +01:00
Przemyslaw Skibinski 4f9eaa7bb3 fixed gcc warnings 2017-02-08 18:08:09 +01:00
Przemyslaw Skibinski cb56306a50 .gz suffix for gzip compressed files 2017-02-08 17:37:14 +01:00
Przemyslaw Skibinski cfd4dc299a add "--format=gzip" option 2017-02-08 15:17:55 +01:00
Przemyslaw Skibinski 40580ff669 added description of "--priority=rt" 2017-02-08 13:49:06 +01:00
Przemyslaw Skibinski 0665a359aa "--rt-prio" renamed to "--priority=rt" 2017-02-07 20:12:59 +01:00
Przemyslaw Skibinski d05014c739 added the "--rt-prio" option 2017-02-07 16:48:01 +01:00
Yann Collet 3dc85bae66 minor : fixed zstd-frugal
fixed a minor unused variable warning when compiling zstd-frugal target
2017-02-02 16:19:09 -08:00
Yann Collet 48bed91606 Merge pull request #527 from facebook/zstdmt
zstdmt refinements
2017-01-31 16:36:46 -08:00
Yann Collet 8d8513fb64 fixed C constant restrictions 2017-01-30 14:37:08 -08:00
Yann Collet 6ccd37c8d4 cli : added advanced parameter overlapLog
as a hidden (undocumented) parameter for now
2017-01-30 13:07:24 -08:00
Yann Collet 6be2337c26 added command --block-size=
for Multi-threading only.
alias : -B#
2017-01-30 11:17:26 -08:00
Nick Terrell 2fe9126591 Add multithread support to COVER 2017-01-27 11:56:02 -08:00
Yann Collet 8dafb1acf5 CLI : automatically set overlap size to max (windowSize) for max compression level 2017-01-25 17:01:13 -08:00
Yann Collet 512cbe8c10 zstdmt cli and API allow selection of section sizes
By default, section sizes are 4x window size.
This new setting allow manual selection of section sizes.
The larger they are, the (slightly) better the compression ratio,
but also the higher the memory allocation cost,
and eventually the lesser the nb of possible threads,
since each section is compressed by a single thread.

It also introduces a prototype to set generic parameters,
ZSTDMT_setMTCtxParameter()

The idea is that it's possible to add enums
to extend the list of parameters that can be set this way.
This is more long-term oriented than a fixed-size struct.
Consider it as a test.
2017-01-24 17:08:53 -08:00
Yann Collet d7e3cb58c5 Resolved merge conflict dev+zstdmt 2017-01-20 16:44:50 -08:00
Yann Collet 500014af49 zstd cli can now compress using multi-threading
added : command -T#
added : ZSTD_resetCStream() (zstdmt_compress)
added : FIO_setNbThreads()  (fileio)
2017-01-19 17:04:28 -08:00
Yann Collet f22adae984 fixed minor warning (unused variable) in fuzzer 2017-01-19 13:46:30 -08:00
Nick Terrell 3a1fefcf00 Simplify COVER parameters 2017-01-02 17:51:38 -08:00
Nick Terrell df8415c502 Add COVER to the zstd cli 2017-01-02 14:43:08 -08:00
Yann Collet 747452677d fixed cmake tests 2017-01-02 02:05:45 +01:00
Yann Collet ab7a579180 added -T command , to set nb of threads 2016-12-28 16:11:09 +01:00
Przemyslaw Skibinski 7a8a03c20d util.h: restore BSD license for Facebook Open-Source 2016-12-21 15:08:44 +01:00
Przemyslaw Skibinski 97a258d71d updated comments 2016-12-21 14:00:41 +01:00
Przemyslaw Skibinski e679741b18 _CRT_SECURE_NO_WARNINGS moved to util.h 2016-12-21 13:47:11 +01:00
Przemyslaw Skibinski 2f6ccee6af platform.h: removed Compiler Options 2016-12-21 13:23:34 +01:00
Przemyslaw Skibinski 20b089e53d simplified zstdcli.c 2016-12-21 09:19:15 +01:00
Przemyslaw Skibinski f8046b8e72 Merge remote-tracking branch 'refs/remotes/facebook/dev' into v112
# Conflicts:
#	appveyor.yml
2016-12-19 08:20:26 +01:00
Przemyslaw Skibinski 0b37205098 util.h: minor improvement 2016-12-16 17:12:23 +01:00
Przemyslaw Skibinski b0f3663edc imporved support for POSIX-type OSes 2016-12-16 15:41:18 +01:00
Przemyslaw Skibinski 3cdfe266cf use PLATFORM_POSIX_VERSION 2016-12-16 15:00:50 +01:00
Przemyslaw Skibinski b866e72826 tools use platform.h 2016-12-16 14:24:01 +01:00
Przemyslaw Skibinski c71e552b2e fixed "strategy" in advanced compression parameters 2016-12-13 20:04:32 +01:00
Przemyslaw Skibinski 98ef0f98df fixed conversion warning 2016-12-13 14:52:21 +01:00
Przemyslaw Skibinski 8349d675e0 fileio.c: support advanced compression parameters 2016-12-13 13:24:59 +01:00
Przemyslaw Skibinski 897b8bb5eb bench.c: support advanced compression parameters 2016-12-13 13:03:41 +01:00
Przemyslaw Skibinski 334b8962b0 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2016-12-13 12:19:19 +01:00
Przemyslaw Skibinski 671f28d1e5 added parseCompressionParameters 2016-12-13 12:18:07 +01:00
Dimitry Andric 83cc2fb083 Enable using isatty() and nanosleep() on *BSD 2016-12-12 19:24:51 +01:00
Yann Collet 2a826e592b compatibility with zstd-frugal (noBench mode) 2016-12-06 17:56:20 -08:00
Yann Collet d946501d2c decode benchmark - single file (hidden option) 2016-12-06 16:49:23 -08:00
Yann Collet 743b33f57e fix zstdcat 2016-12-02 15:18:57 -08:00
Yann Collet 52afb3993e zbuff API now generates deprecation warnings 2016-11-16 08:50:54 -08:00
Przemyslaw Skibinski 38b590ad69 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
# Conflicts:
#	lib/Makefile
2016-11-04 10:10:54 +01:00
Yann Collet 407a11f63e fixed Visual compatibility 2016-11-03 15:52:01 -07:00
Przemyslaw Skibinski 26306fcacf BMK_SetNbIterations renamed to BMK_SetNbSeconds 2016-11-03 11:38:01 +01:00
Yann Collet b9550d6d43 fixed benchmark mode typo .
added benchmark mode tests
2016-10-28 14:43:24 -07:00
Yann Collet 22de81e87e updated man page 2016-10-28 13:58:31 -07:00
Yann Collet 33fdd099bb fix command line interpretation 2016-10-17 17:48:48 -07:00
Yann Collet 70077bc9bb refactor for long commands 2016-10-14 14:41:17 -07:00
Yann Collet d7b120ab5c added long commands --memory= and --memlimit-decompress= 2016-10-14 14:22:32 -07:00
Yann Collet 1122349ac2 added long comment --memlimit= 2016-10-14 14:07:11 -07:00
Yann Collet d4cda27b63 new command -M#, to limit memory usage during decompression (#403) 2016-10-14 13:32:20 -07:00
Nick Terrell 3e82f2b61a Remove references to GPLv2 license
License headers added to Makefiles were taken from `zstd/Makefile`.
2016-10-11 15:28:40 -07:00
inikep 23cec875cd fixed Makefile targets: zstd-small, zstd-decompress, zstd-compress 2016-10-05 11:56:22 +02:00
Yann Collet 993060e0f2 cli : better adaptation to small files 2016-09-21 16:46:08 +02:00
Yann Collet 714464f05d fixed : cli : forgotten mandatory argument 2016-09-21 16:05:03 +02:00
Yann Collet 27b5ac666e Implemented "command must be followed by argument" protection
suggested by @terrelln (#375)
2016-09-21 14:31:22 +02:00
Yann Collet 0977f7ece6 minor refactor for clarity 2016-09-21 12:24:43 +02:00
Yann Collet 60038948e6 added -- command in help 2016-09-16 18:52:52 +02:00
Yann Collet 43eeea4725 fileio uses ZSTD_resetCStream() 2016-09-15 15:38:44 +02:00
Yann Collet c4cc9bf973 -r generates an error on systems which do not support it 2016-09-13 17:50:08 +02:00
Yann Collet 60181e3aaf zstd cli correctly detects console on Mac OS-X 2016-09-01 17:14:02 -07:00
Yann Collet c843142ad0 zstd -d writes to stdout when input is stdin 2016-09-01 15:05:57 -07:00
Yann Collet 4ded9e591c added boilerplate 2016-08-30 11:06:28 -07:00
Yann Collet 5f53b0335e fixed continuation context 2016-08-28 10:00:49 -07:00
Yann Collet 87c18b2ebd fixed multiple minor warnings for XCode 2016-08-26 01:43:47 +02:00
inikep 5a54870047 fixed Intel Compiler warnings with Visual Studio
http://encode.ru/threads/2119-Zstandard?p=49504&viewfull=1#post49504
2016-08-18 09:00:25 +02:00
David Lam ac43e9582e Add more descriptive message to -v option in zstd CLI 2016-08-12 17:47:18 -07:00
Yann Collet fe0590f63f new --ultra policy 2016-08-12 18:04:15 +02:00
Yann Collet 415251c63b minor style change 2016-08-01 14:26:49 +02:00
Yann Collet a9febe81ae changed bench behavior for slow compression levels 2016-08-01 13:40:52 +02:00
Yann Collet 4110534886 ZSTD_maxCLevel() is promoted to "stable" API (#254, by @FrancescAlted) 2016-07-27 15:09:11 +02:00
Yann Collet b402490546 fixed #260, reported by @amnilsson 2016-07-26 00:49:47 +02:00
Yann Collet 10b9c13d07 fixed doc on cLevel default, reported by Oliver Lange 2016-07-24 01:21:53 +02:00
Yann Collet 06dd74114a minor : long tests display progression status 2016-07-18 04:06:43 +02:00
Yann Collet 988bcf360a -v and --verbose increase display level by 1 2016-07-15 19:43:30 +02:00
Yann Collet 5e80dd3261 fixed minor coverity warnings 2016-07-13 19:21:57 +02:00
Yann Collet 2b1a3638e6 changed macro name to ZSTDCLI_CLEVEL_DEFAULT 2016-07-13 15:16:00 +02:00
Yann Collet 2cac5b30b9 changed default compression level to 3 (can be modified with macro ZSTDCLI_DEFAULT_CLEVEL) 2016-07-13 14:15:08 +02:00
Yann Collet f9cac7a734 Added GNU separator `--`, to specifies that all following arguments are necessary file names (and not commands). Suggested by @chipturner (#230) 2016-07-04 18:18:24 +02:00
Yann Collet 92c986b4e8 fixed cmake error (missing errno) 2016-07-04 01:37:30 +02:00
Yann Collet d916c908e0 updated doc 2016-07-04 00:42:58 +02:00
Yann Collet f4b7b747c7 `-c` does no longer modify displayLevel when `-v` set. 2016-07-02 22:57:28 +02:00
Yann Collet a7a5d7736c Updated doc 2016-07-02 00:37:32 +02:00
Yann Collet c98f8e7b57 Modified : zstd CLI enables checksum by default 2016-06-20 16:31:24 +02:00
Yann Collet b09b12ce10 Added command `--rm` : remove source file after successful de/compression 2016-06-09 23:01:19 +02:00
Yann Collet f3a1a84c9a fixed cmake project 2016-06-05 01:05:01 +02:00
Yann Collet d3b7f8d21f Merged `zstd_static.h` into `zstd.h` . Now requires `ZSTD_STATIC_LINKING_ONLY` macro 2016-06-04 19:47:02 +02:00
Yann Collet 441f5d61fd dictionary size objective can be described in Megabytes 2016-06-03 15:15:47 +02:00
Yann Collet 1dd7961557 minor code refactoring 2016-06-03 15:14:09 +02:00
Yann Collet 2c4acdabdc Changed command name to `--check` , `-C`, following `xz` convention 2016-06-02 17:05:50 +02:00
Yann Collet 87cfbe3e5f checksum generation can be selected from command line (`--checksum`) 2016-06-01 19:22:15 +02:00
inikep a6b942018d Merge remote-tracking branch 'refs/remotes/origin/dev' into dev070
# Conflicts:
#	.travis.yml
#	Makefile
#	lib/common/zstd_static.h
#	programs/Makefile
#	projects/VS2008/zstd/zstd.vcproj
#	projects/VS2008/zstdlib/zstdlib.vcproj
#	projects/cmake/lib/CMakeLists.txt
#	projects/cmake/programs/CMakeLists.txt
2016-06-01 00:07:09 +02:00
Yann Collet 6381e99fb2 Added : ability to create compressed frames without dictID 2016-05-31 02:29:45 +02:00
Yann Collet 290aaa7521 Added : ability to manually select the dictionary ID of a newly created dictionary 2016-05-30 21:18:52 +02:00
inikep 93fc13ebc8 playTests.sh working with Linux, Windows, OS X 2016-05-30 10:17:55 +02:00
Yann Collet fcf2087a15 restored zstdcli.c 2016-05-29 05:16:05 +02:00
Yann Collet c46fb924df added dictionary ID (incomplete) 2016-05-29 05:01:04 +02:00
inikep bff23b72c5 fixed Visual Studio warning 2016-05-25 17:40:58 +02:00
inikep 957823f56f zstdcli: -r (operate recursively on directories) works with dictBuilder and compression 2016-05-25 15:30:55 +02:00
inikep 4f1acdd158 usage of fileno() only on POSIX and Windows 2016-05-25 12:41:41 +02:00
inikep 02c244bf78 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-05-24 17:15:04 +02:00
Yann Collet de95f96462 Pass-Through mode support (using `-df`), for compatibility with gzip 2016-05-23 19:46:47 +02:00
Yann Collet 32990b5dae Added tests for Sparse mode support
Fixed : complex cli arg case involving a mix of `stdin` and `-o`
2016-05-23 17:48:57 +02:00
Yann Collet 75424d1139 Added : Sparse write support
--[no-]sparse command
2016-05-23 16:56:56 +02:00
Yann Collet de4d4da52d changed version to 0.6.2
removed make-controlled version (now used from zstd.h)
removed std=c99 by default
added : c90/gno90/c99/gnu99/c11 tests
2016-05-20 12:20:12 +02:00
inikep 60af95d971 fixed compression with -c with multiple files on Windows 2016-05-19 10:29:49 +02:00
inikep 4dbf7f4a3b dynamic memory allocation in UTIL_createFileList 2016-05-11 14:11:00 +02:00
inikep 3733797fcd bench.c: experimental -r (operate recursively on directories) for Windows and _POSIX_C_SOURCE >= 200112L 2016-05-10 14:22:55 +02:00
Yann Collet 6f9c056662 Fixed BMI behavior on x86 and x32 targets, reported by @tycho (#178) 2016-05-01 10:26:30 +02:00
inikep db3964382c introduced ZSTD_NODECOMPRESS to link only compressor 2016-04-22 18:22:30 +02:00
inikep 3c7c3527d0 introduced ZSTD_NOCOMPRESS to generate decompressor only 2016-04-22 13:59:05 +02:00
inikep 4699b3be03 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-04-21 12:30:58 +02:00
inikep 38654988f3 minor speed improvements 2
bench.c: block size has to be bigger than 32 bytes
zstdcli.c: support for e.g. -B16k -B16m
2016-04-21 12:18:47 +02:00
Yann Collet e02808ffc5 bumped lib version to 0.6.1
added ZSTD_VERSION_STRING
2016-04-20 22:46:16 +02:00
Yann Collet 7928f6f024 Fixed : cli error on stdin + stdout, as detected by Alexey Spiridonov
Added : relevant test case
2016-04-10 20:40:26 +02:00
inikep 5553442376 fixed compilation with ZSTD_NOBENCH 2016-03-25 13:35:33 +01:00
inikep eaba91a161 added ZSTD_compressBegin_targetSrcSize 2016-03-23 20:30:26 +01:00
inikep 2872b6f01f Merge remote-tracking branch 'refs/remotes/Cyan4973/dev060' into dev
# Conflicts:
#	lib/zstd_compress.c
#	programs/bench.c
2016-03-22 14:38:34 +01:00
Yann Collet 5a854af006 Fixed #153, reported by @thatsafunnyname 2016-03-22 00:22:50 +01:00
inikep 5fdd0b46d3 added additionalParam 2016-03-14 19:51:11 +01:00
inikep e9554b73cc -r# : test all compression levels from -bX to # 2016-03-14 18:10:30 +01:00
inikep 4e26bb69c7 bench.c can generate a log file with average values 2016-03-14 12:48:51 +01:00
Yann Collet 8a1d1a6a6c implemented `--ultra` mode protection 2016-03-10 21:02:25 +01:00
Yann Collet 6aa896adc6 zstd cli : control maximum compression level 2016-02-16 16:33:48 +01:00
Yann Collet 2d08b098ef Updated man zstd
Added `-t` | `--test` to zstd cli
Added a few long commands, compatible with gzip
2016-02-16 14:42:08 +01:00
Yann Collet accfd80c37 Added : option `-t` to test compressed files integrity 2016-02-15 19:33:16 +01:00
Yann Collet e8bd73df80 Fixed : error when compressing one file 2016-02-13 03:09:48 +01:00
Yann Collet 6f3acbac0d notificationLevel into ZDICT_param_t 2016-02-12 20:19:48 +01:00
Yann Collet 1f1f239ed9 zstd cli can decompress multiple files with no specific suffix condition into stdout 2016-02-12 18:33:26 +01:00
Yann Collet f062436fa6 cli : added ability to compress multiple files into stdout (-c) 2016-02-12 15:56:46 +01:00
Yann Collet c8da2c9f8f updated tests 2016-02-12 02:56:27 +01:00
Yann Collet 71eafdd23f Integrated dictionary builder into zstd cli 2016-02-12 02:31:57 +01:00
Yann Collet 4488661678 minor cosmetics 2016-02-11 04:17:50 +01:00
Yann Collet 521b0108dd Added PowerPC tests with Qemu user more
Shortened tests for emulated platforms
removed date from CLI for reproducibility
2016-02-08 01:27:59 +01:00
Yann Collet 45ff4309fd Updated documentation 2016-02-05 15:24:57 +01:00
Yann Collet 324a3e27c3 fixed visual warning 2015-12-18 03:19:27 +01:00
Yann Collet 6a4583542a fixed asan warning 2015-12-18 02:51:14 +01:00
Yann Collet 17d188fa38 robust file list 2015-12-18 02:14:46 +01:00
Yann Collet 31683c0b28 benchmark can use dictionary 2015-12-18 01:26:48 +01:00
Yann Collet deb078b9b1 added : -m : decompress multiple files 2015-12-17 20:30:14 +01:00
Yann Collet 4f13703b2f added : -m : compress multiple files in a single command 2015-12-17 02:23:58 +01:00
Yann Collet 7a3ab588c7 minor fixes 2015-12-15 11:25:12 +01:00
Yann Collet f6f3d7526a external dictionary capability added to command line 2015-12-13 13:35:21 +01:00
Yann Collet 6c3e2e7855 fixed : high compression modes for Windows 32 bits 2015-12-11 10:44:07 +01:00
Yann Collet e47c4e5f8e strengthened bufferless streaming decompression 2015-12-05 09:23:53 +01:00
Yann Collet 28e7cefea9 new zstd-noBench & zstd-frugal builds 2015-12-03 12:11:30 +01:00
Yann Collet 6a2f03295c update version to 0.4.2 2015-12-02 15:05:22 +01:00
Yann Collet d062f1379c fixed benchmark autogen 2015-12-01 01:31:17 +01:00
Yann Collet a5c737411a complex level selector 2015-11-22 14:16:56 +01:00
Yann Collet 7f6e91fa98 fixed library compilation 2015-11-11 14:39:50 +01:00
Yann Collet 43e0cd512a small blocks params 2015-11-09 16:38:17 +01:00
Yann Collet f48e35c206 minor bt improvements 2015-11-07 01:13:31 +01:00
Yann Collet f12c1304a8 integrate fast mode 2015-11-05 18:16:59 +01:00
Yann Collet 50c5cdb44c fixed issues reported by Maciej Adamczyk 2015-11-04 20:35:33 +01:00
Yann Collet 3564487614 lazydeep 2015-11-02 16:14:46 +01:00
Yann Collet be2010ea1b lazy strategy 2015-10-31 12:57:14 +01:00
Yann Collet 44fe99116e fixed hc alloc 2015-10-29 22:02:40 +01:00
Yann Collet c776c46b46 range bench 2015-10-29 19:10:54 +01:00
Yann Collet 2acb5d3d48 HC modes can be selected from command line 2015-10-29 16:49:43 +01:00
Yann Collet f3eca25322 quick first zstdhc version 2015-10-22 15:31:46 +01:00
Yann Collet 1c00dc3e24 Added block size to bench 2015-10-21 08:22:25 +01:00
Yann Collet 50b6f941e8 added : --verbose command 2015-08-26 10:32:17 +01:00
Yann Collet f44b2b02b4 Zstd cli : --version and --help write to stdout 2015-08-25 23:32:45 +01:00
Yann Collet 17867ced03 Fixed minor clang warnings 2015-07-07 00:14:27 -08:00
Yann Collet effbd9811b Merge pull request #30 from jforberg/dev
Add `unzstd` program for decompression.
2015-07-05 15:02:41 -07:00
Yann Collet b5e06dc7c3 Fixed a few minor visual analyzer warnings 2015-07-04 23:20:56 -08:00
Yann Collet 94f998b1fc Fixed a few visual analyzer warnings 2015-07-04 23:10:40 -08:00
Yann Collet 213089c078 Updated FSE
Added clang and g++ tests
2015-06-18 07:43:16 -08:00
Johan Förberg 273d049a7c Add `unzstd` program for decompression. 2015-03-24 20:15:56 +01:00
Yann Collet e1e6f7d0b1 Removed minor warnings under minGW 2015-01-25 15:50:24 +01:00
Yann Collet 4856a00164 Initial release 2015-01-24 01:58:16 +01:00