5022a18d51
improved #232 fix
2017-01-25 13:11:26 +01:00
64fa2dbc5e
Fixed https://github.com/facebook/zstd/issues/232
2017-01-25 13:02:33 +01:00
512cbe8c10
zstdmt cli and API allow selection of section sizes
...
By default, section sizes are 4x window size.
This new setting allow manual selection of section sizes.
The larger they are, the (slightly) better the compression ratio,
but also the higher the memory allocation cost,
and eventually the lesser the nb of possible threads,
since each section is compressed by a single thread.
It also introduces a prototype to set generic parameters,
ZSTDMT_setMTCtxParameter()
The idea is that it's possible to add enums
to extend the list of parameters that can be set this way.
This is more long-term oriented than a fixed-size struct.
Consider it as a test.
2017-01-24 17:08:53 -08:00
500014af49
zstd cli can now compress using multi-threading
...
added : command -T#
added : ZSTD_resetCStream() (zstdmt_compress)
added : FIO_setNbThreads() (fileio)
2017-01-19 17:04:28 -08:00
7a8a03c20d
util.h: restore BSD license for Facebook Open-Source
2016-12-21 15:08:44 +01:00
97a258d71d
updated comments
2016-12-21 14:00:41 +01:00
e679741b18
_CRT_SECURE_NO_WARNINGS moved to util.h
2016-12-21 13:47:11 +01:00
2f6ccee6af
platform.h: removed Compiler Options
2016-12-21 13:23:34 +01:00
a35b9448ac
improved MinGW support
2016-12-21 11:18:45 +01:00
20b089e53d
simplified zstdcli.c
2016-12-21 09:19:15 +01:00
16ae6563a2
executables use new util.h and platform.h
2016-12-21 09:06:14 +01:00
f8046b8e72
Merge remote-tracking branch 'refs/remotes/facebook/dev' into v112
...
# Conflicts:
# appveyor.yml
2016-12-19 08:20:26 +01:00
b866e72826
tools use platform.h
2016-12-16 14:24:01 +01:00
c71e552b2e
fixed "strategy" in advanced compression parameters
2016-12-13 20:04:32 +01:00
8349d675e0
fileio.c: support advanced compression parameters
2016-12-13 13:24:59 +01:00
7c6bbc3298
updated FIO_passThrough
2016-12-05 18:31:14 +01:00
6b508b1770
updated test-gzstd
2016-12-05 18:02:40 +01:00
b493e3b3d3
introduced srcBufferLoaded
2016-12-05 17:39:38 +01:00
3c69760275
improved FIO_decompressGzFrame
2016-12-05 15:58:23 +01:00
8489f184f6
improved detection of -lz
2016-12-05 13:47:00 +01:00
743b33f57e
fix zstdcat
2016-12-02 15:18:57 -08:00
5bd4237beb
minor refactor
2016-12-02 12:40:57 -08:00
690753ea1a
improved formatting in FIO_decompressSrcFile
2016-12-02 16:20:16 +01:00
4e49580407
removed testing artifacts
2016-12-02 15:19:00 +01:00
c5eebca128
rewritten FIO_decompressGzFile
2016-12-02 15:01:31 +01:00
b0f2ef2119
improved gzip_* functions
2016-12-02 13:50:29 +01:00
4b504f131a
added gzip_open, gzip_close, gzip_read
2016-12-02 13:11:39 +01:00
daaf754539
detect stream with ungetc
2016-12-01 13:29:19 +01:00
19aad42ee1
added FIO_decompressGzFile
2016-12-01 11:56:31 +01:00
abfb51f5f2
gzstd: decompresses .gz files
2016-11-30 15:05:54 +01:00
0e14675df2
fileio.c: detect .gz files
2016-11-30 13:34:21 +01:00
cdff19c4b3
minor comment change
2016-11-11 17:26:54 -08:00
0018ca28dc
zstd cli : displays total decoded size, even when a stream consists of multiple frames
2016-11-07 14:42:37 -08:00
1f60eb6b17
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
2016-11-03 11:38:04 +01:00
26306fcacf
BMK_SetNbIterations renamed to BMK_SetNbSeconds
2016-11-03 11:38:01 +01:00
179b19776f
fileio.c does no longer need ZSTD_LEGACY_SUPPORT, and does no longer depend on zstd_legacy.h
...
Added : ZSTD_isFrame() in experimental section
2016-11-02 17:30:49 -07:00
fcf22e3473
set permissions, access and modification times
2016-11-02 14:08:07 +01:00
a42794df61
preserve file modification time
2016-11-02 13:08:39 +01:00
d4cda27b63
new command -M#, to limit memory usage during decompression ( #403 )
2016-10-14 13:32:20 -07:00
3e82f2b61a
Remove references to GPLv2 license
...
License headers added to Makefiles were taken from `zstd/Makefile`.
2016-10-11 15:28:40 -07:00
993060e0f2
cli : better adaptation to small files
2016-09-21 16:46:08 +02:00
43eeea4725
fileio uses ZSTD_resetCStream()
2016-09-15 15:38:44 +02:00
3ecbe6a37c
fileio uses ZSTD_resetDStream()
2016-09-14 17:26:59 +02:00
d77f2ceee6
Merge branch 'dev' of github.com:Cyan4973/zstd into dev
2016-09-12 03:14:03 +02:00
5c6d244973
Merge branch 'dev' of github.com:facebook/zstd into dev
2016-09-07 14:54:54 +02:00
ac8bace6b1
support large skippable frames
2016-09-07 14:54:23 +02:00
03d3f238de
minor comment
2016-09-07 07:01:33 +02:00
aad9fe5470
don't remove() /dev/null ( #316 )
2016-09-07 07:00:08 +02:00
4ded9e591c
added boilerplate
2016-08-30 11:06:28 -07:00
1c69baa1a9
CLI supports Legacy formats v0.4+
2016-08-28 12:47:17 -07:00
5f53b0335e
fixed continuation context
2016-08-28 10:00:49 -07:00
ab267e772c
removed fileio specific legacy support;
...
now depends on ZSTD_decompressStream() transparent legacy support
2016-08-28 08:46:25 -07:00
e80d15304a
Merge pull request #296 from inikep/Other
...
Other
2016-08-18 11:48:48 +02:00
5a54870047
fixed Intel Compiler warnings with Visual Studio
...
http://encode.ru/threads/2119-Zstandard?p=49504&viewfull=1#post49504
2016-08-18 09:00:25 +02:00
bfdc34e5a3
Merge pull request #294 from inikep/Other
...
improved Makefiles
2016-08-17 19:41:56 +02:00
78f3e0630b
disabled %llu warnings with MinGW on Windows
2016-08-17 14:52:11 +02:00
3d9211e961
Merge pull request #290 from david-y-lam/fix_file_name
...
Remove file name truncation in zstd CLI output
2016-08-17 14:05:29 +02:00
53e17fbd5e
updated streaming API
2016-08-17 01:39:22 +02:00
94ca85d01b
fixed Visual warning
2016-08-16 15:11:28 +02:00
6263ba5451
fileio uses new streaming API
2016-08-16 15:11:28 +02:00
fb5e385164
minor changes
2016-08-16 15:11:28 +02:00
0f2704581e
Remove truncation of filename in zstd CLI log lines
2016-08-13 11:26:21 -07:00
60ba31c570
zbuff uses ZSTD_compressEnd()
2016-07-28 19:55:09 +02:00
fbd557d5c2
multi-files -t doesn't stop after detecting magic number read failure
2016-07-26 17:13:58 +02:00
a1dd6b97d2
fixed null-length round trip
2016-07-26 16:44:09 +02:00
7adc2328a3
fixed --test on zero-length files, reported by @amnilsson
2016-07-26 15:49:24 +02:00
24a3d90bf0
strengthened integrity tests
2016-07-26 01:33:20 +02:00
b402490546
fixed #260 , reported by @amnilsson
2016-07-26 00:49:47 +02:00
17508f1a16
fixed a few minor coverity warnings
2016-07-14 17:18:20 +02:00
44f684ded3
fixed minor coverity warning
2016-07-13 20:12:33 +02:00
5e80dd3261
fixed minor coverity warnings
2016-07-13 19:21:57 +02:00
3c242e79d3
updated compression levels table
2016-07-13 14:56:24 +02:00
2cac5b30b9
changed default compression level to 3 (can be modified with macro ZSTDCLI_DEFAULT_CLEVEL)
2016-07-13 14:15:08 +02:00
19c27d27f1
simplified legacy functions, no longer need magic number
2016-07-07 14:40:13 +02:00
c62cda9def
fileio.c no longer depends on zstd_internal.h
2016-07-03 01:36:57 +02:00
92d75667e4
fix for fast mode
2016-07-03 01:10:53 +02:00
b2b5309c6f
removes Visual warning on strerror()
2016-07-03 00:20:03 +02:00
ed7fb8413c
extended use of strerror(errno)
2016-07-02 21:01:54 +02:00
b71adf45c1
extended use of strerror(errno)
2016-07-02 01:51:12 +02:00
b7f197633a
use strerror(errno) to return system error message
2016-07-02 00:59:25 +02:00
cebab1d322
Explicit error message when dst file cannot be created / opened
2016-07-02 00:12:14 +02:00
6c6e1751f6
use ZSTD_getParams() to simplify code
2016-06-27 15:28:45 +02:00
391a128794
fix : segfault in command line during automatic overwrite protection mode
2016-06-21 17:06:25 +02:00
c98f8e7b57
Modified : zstd CLI enables checksum by default
2016-06-20 16:31:24 +02:00
b09b12ce10
Added command `--rm` : remove source file after successful de/compression
2016-06-09 23:01:19 +02:00
2cc72f1fd3
fixed initialization issue in bench
2016-06-06 17:50:07 +02:00
673f0d7cdc
new frame format, allowing custom window size
2016-06-06 00:26:38 +02:00
d3b7f8d21f
Merged `zstd_static.h` into `zstd.h` . Now requires `ZSTD_STATIC_LINKING_ONLY` macro
2016-06-04 19:47:02 +02:00
5347aee8f7
merged `zbuff_static.h` into `zbuff.h` . Now requires `ZBUFF_STATIC_LINKING_ONLY` macro
2016-06-04 19:12:48 +02:00
0d31160b4e
better dynamic memory adaptation when using CLI
2016-06-04 00:09:02 +02:00
87cfbe3e5f
checksum generation can be selected from command line (`--checksum`)
2016-06-01 19:22:15 +02:00
a6b942018d
Merge remote-tracking branch 'refs/remotes/origin/dev' into dev070
...
# Conflicts:
# .travis.yml
# Makefile
# lib/common/zstd_static.h
# programs/Makefile
# projects/VS2008/zstd/zstd.vcproj
# projects/VS2008/zstdlib/zstdlib.vcproj
# projects/cmake/lib/CMakeLists.txt
# projects/cmake/programs/CMakeLists.txt
2016-06-01 00:07:09 +02:00
f772bf54a5
support for skippable frames
2016-05-31 12:43:46 +02:00
6fca9e7545
Changed : max dictionary file size increased to 8 MB
2016-05-31 02:40:42 +02:00
6381e99fb2
Added : ability to create compressed frames without dictID
2016-05-31 02:29:45 +02:00
c46fb924df
added dictionary ID (incomplete)
2016-05-29 05:01:04 +02:00
e75909ef93
ZBUFF_createCCtx_advanced moved to zbuff_static.h
2016-05-25 11:31:16 +02:00
02c244bf78
Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev
2016-05-24 17:15:04 +02:00
ddbb8e27bf
fixed minor conversion warning
2016-05-24 00:52:14 +02:00
de95f96462
Pass-Through mode support (using `-df`), for compatibility with gzip
2016-05-23 19:46:47 +02:00
75424d1139
Added : Sparse write support
...
--[no-]sparse command
2016-05-23 16:56:56 +02:00
60af95d971
fixed compression with -c with multiple files on Windows
2016-05-19 10:29:49 +02:00
b3a9938d0c
Merge pull request #182 from inikep/dev
...
support for Visual Studio 2012, 2013, 2015
2016-05-10 17:18:12 +02:00
bd39d54576
zbuff supports byte-by-byte decompression scenarios
2016-05-10 14:14:19 +02:00
ed9a08538c
Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev
...
# Conflicts:
# lib/common/util.h
# programs/paramgrill.c
# visual/2013/fullbench/fullbench.vcxproj.filters
# visual/2013/fuzzer/fuzzer.vcxproj.filters
2016-05-10 13:20:01 +02:00
d693117113
fixed : null-length string roundtrip
2016-05-10 05:56:09 +02:00
8b23eea694
minor refactor : more accurate variables scope
...
Added null-length string test
2016-05-10 05:37:43 +02:00
f6ca09b5ff
Reduced console display on loading lots of files with `zstd --train`. Reported by @KrzysFR, see #177
2016-05-09 04:44:45 +02:00
95af06f6ef
fixed legacy dictionary decompression
2016-05-08 08:23:51 +02:00
0c0f314550
CLI supports dictionary decompression in legacy mode v0.5.x
2016-05-08 08:11:24 +02:00
f849462d4c
minor refactor : more accurate variable scope
2016-05-07 22:43:40 +02:00
13c8424ea0
code cleaning
2016-05-05 13:58:56 +02:00
0bd0faec32
fixed compatibility issues
2016-05-05 13:10:57 +02:00
9c22e57bfb
Compiler Options moved to util.h
2016-05-05 11:53:42 +02:00
bab4317961
util.h must the the first include to #define _POSIX_C_SOURCE
2016-04-29 15:19:40 +02:00
d5ff2c3d9a
ordering of #include
2016-04-28 14:40:45 +02:00
83c76b4594
bench.c: time functions moved to common/util.h
2016-04-28 13:16:01 +02:00
69fcd7c0ae
getFileSize moved to common/util.h
2016-04-28 12:23:33 +02:00
db3964382c
introduced ZSTD_NODECOMPRESS to link only compressor
2016-04-22 18:22:30 +02:00
3c7c3527d0
introduced ZSTD_NOCOMPRESS to generate decompressor only
2016-04-22 13:59:05 +02:00
23a0889301
separation of lib/ into common/, compress/, decompress/, dictBuilder/, legacy/
2016-04-22 12:43:18 +02:00
9e8b09a7bd
Fixed memory initialization issue, reported by Maciej Adamczyk
2016-04-07 19:35:23 +02:00
3b71925c2d
separate params into compressionParams and frameParams
2016-03-30 19:48:05 +02:00
b44be74244
Fixed zdict
...
more dictionary compression tests
2016-03-26 20:52:14 +01:00
09b21ee2f6
fix fileio bug with new zbuff
...
simplified zbuff decoding
2016-03-15 12:56:03 +01:00
886274bc79
Merge pull request #147 from Cyan4973/dev
...
Decompression error return code, by @chipturner
2016-03-14 18:41:24 +01:00
9da7f86591
use remove instead of unlink
2016-03-14 07:44:59 -07:00
6de382cdbb
propagate errors when decompressing; also unlink dstfile if we create it but fail
...
to successfully compress or decompress
2016-03-13 22:33:11 -07:00
8a1d1a6a6c
implemented `--ultra` mode protection
2016-03-10 21:02:25 +01:00
459a6b712f
Fix corner case multiple files bug
...
Added test cases
2016-02-15 20:37:23 +01:00
accfd80c37
Added : option `-t` to test compressed files integrity
2016-02-15 19:33:16 +01:00
6f3acbac0d
notificationLevel into ZDICT_param_t
2016-02-12 20:19:48 +01:00
62ae5fb0d2
zstd_buffered => zbuff
2016-02-12 18:59:11 +01:00
1f1f239ed9
zstd cli can decompress multiple files with no specific suffix condition into stdout
2016-02-12 18:33:26 +01:00
f062436fa6
cli : added ability to compress multiple files into stdout (-c)
2016-02-12 15:56:46 +01:00
4488661678
minor cosmetics
2016-02-11 04:17:50 +01:00
2ce4923238
modified buffer management rules
2016-02-02 14:36:49 +01:00
1c8e194297
modified streaming compression API
2016-01-26 16:31:22 +01:00
7b51a2948f
modified streaming decompression API
2016-01-26 15:58:49 +01:00
c1450f721d
fix confirmation prompt
2015-12-29 11:57:15 +01:00
deb078b9b1
added : -m : decompress multiple files
2015-12-17 20:30:14 +01:00
9d90922d49
refactored file compression
2015-12-17 14:09:55 +01:00
24c98f2b4b
fixed : conversion warning
2015-12-17 02:48:26 +01:00
4f13703b2f
added : -m : compress multiple files in a single command
2015-12-17 02:23:58 +01:00
7a3ab588c7
minor fixes
2015-12-15 11:25:12 +01:00
5d4fa0ed9b
fixed arm-specific shadow warning
2015-12-13 13:58:51 +01:00
60348b91f3
fixed silent conversion warnings
2015-12-13 13:44:23 +01:00
f6f3d7526a
external dictionary capability added to command line
2015-12-13 13:35:21 +01:00
a85a8dd94a
Fixed #80 (reported by @luben)
...
Added : tests for ZSTD_LEGACY=disable build mode
extended ZSTD_LEGACY=disable to dll build
2015-11-30 11:53:11 +01:00
b096cbf0d2
can build without legacy support using ZSTD_LEGACY=disable
2015-11-27 16:18:56 +01:00
31d1806a77
zbuff properly supports small dst buffers
2015-11-27 14:07:36 +01:00
88fcd2916e
Added : zstd buffered API
2015-11-25 14:42:45 +01:00
d7233d6e14
more compression tests
2015-11-22 14:40:51 +01:00
5be2dd25f2
Changed Library source tree
...
- no more zstdhc ; zstd.h is enough
- ZSTD_compress() now needs a compressionLevel
- separated zstd_compress.c and zstd_decompress.c
- updated zstdcli, fullbench, fuzzer with new API
2015-11-11 13:43:58 +01:00
b7d6e8f6bc
fixed Visual warning
2015-11-09 18:20:39 +01:00
9f432926b4
fileio can use small blocks params
2015-11-09 17:42:17 +01:00
aa074051d6
decoding support for legacy v0.2 format
2015-10-30 11:21:50 +01:00
4114f95ce9
fix Visual Studio projects
2015-10-30 06:40:22 +01:00
2acb5d3d48
HC modes can be selected from command line
2015-10-29 16:49:43 +01:00
eeb8ba18d1
uncoupled maxdlog and chainlog
2015-10-22 16:55:40 +01:00
f3eca25322
quick first zstdhc version
2015-10-22 15:31:46 +01:00
353c5d26cf
unified ctx naming convention
2015-10-21 14:39:26 +01:00
b1f3f4b565
update huff0
2015-10-18 22:18:32 +01:00
be50aaa0ee
Added : Frame concatenation ability
2015-09-10 23:26:09 +01:00
5abd8203cb
Fix : decompression i/o detects flush write errors
2015-08-27 03:16:04 +01:00
61d08c52a5
More informative write error messages
2015-08-25 18:13:32 +01:00
5b147600f9
Fixed #42
2015-08-25 17:41:46 +01:00
1885029ba1
Fixed a few issues found by AFL (American Fuzzy Lop)
2015-08-24 20:17:11 +01:00
fee8e240c7
Fixed decoding error #11 (reported by @magv)
2015-08-24 15:47:04 +01:00
b5e06dc7c3
Fixed a few minor visual analyzer warnings
2015-07-04 23:20:56 -08:00
7393c5a51d
Advanced API change : Cctx & DCtx are now incomplete types for stricter type checking
2015-07-04 18:20:42 -08:00
078a9a2804
Added Wcast-qual compilation flag
...
Updated xxHash
Removed log traces
2015-06-20 19:37:53 -08:00
213089c078
Updated FSE
...
Added clang and g++ tests
2015-06-18 07:43:16 -08:00
c5d46b5c90
New streaming API behavior, to solve issue 19 ( https://github.com/Cyan4973/zstd/issues/19 )
2015-02-16 18:06:26 +01:00
aacace37f7
Fixed a few clang warnings
2015-02-01 11:57:30 +01:00
e1e6f7d0b1
Removed minor warnings under minGW
2015-01-25 15:50:24 +01:00
4856a00164
Initial release
2015-01-24 01:58:16 +01:00
Przemyslaw Skibinski
Yann Collet
Nick Terrell
Yann Collet
David Lam
Chip Turner