Commit Graph

1119 Commits (899f2a29f6f70f7812c3f79259b593c3dde67401)

Author SHA1 Message Date
Yann Collet ef569bf75f Merge branch 'dev' of github.com:facebook/zstd into dev 2017-02-27 15:58:38 -08:00
Yann Collet dccd6b6f65 cli : fix : --rm is silent when input is stdin
previously, app would produce an error message, and stop.
2017-02-27 15:57:50 -08:00
Przemyslaw Skibinski 862698f479 minor tweaks in FIO_decompressGzFrame 2017-02-27 13:21:05 +01:00
Yann Collet 0b9b894b2d reduced ZSTD_DDict memory usage
saved 128 KB
2017-02-27 00:27:30 -08:00
Przemyslaw Skibinski 337ec875b6 minor tweaks 2017-02-22 10:31:30 +01:00
Przemyslaw Skibinski 684858e7b7 fix memory leaks 2017-02-21 18:17:24 +01:00
Przemyslaw Skibinski 74dcd8d15f bench.c: use a single ticksPerSecond 2017-02-21 12:22:05 +01:00
Przemyslaw Skibinski 1b59333c82 util.h: restore times() 2017-02-21 07:33:45 +01:00
Przemyslaw Skibinski da4a0f30af util.h: use mach_absolute_time for macOS 2017-02-20 12:18:15 +01:00
Przemyslaw Skibinski e052c60540 introduce UTIL_freq_t 2017-02-20 11:27:11 +01:00
Przemyslaw Skibinski 83775d9e05 replace times() with clock_gettime(CLOCK_MONOTONIC, x) 2017-02-20 11:11:50 +01:00
Przemyslaw Skibinski acb6e57ad2 use FindFirstFileA instead of FindFirstFile 2017-02-15 17:13:35 +01:00
Przemyslaw Skibinski 6e59b3ce01 added UTIL_fseek 2017-02-15 17:03:16 +01:00
Przemyslaw Skibinski 3aaa1dae4e simplified zlib detection 2017-02-15 09:17:39 +01:00
Yann Collet f0b9a8dddb Merge pull request #547 from inikep/dev11
Avoid fseek()'s 2GiB barrier with MacOS and *BSD
2017-02-14 12:29:00 -08:00
Yann Collet c09d16ba8c preset behavior for gzip, gunzip and gzcat
when zstd is called through a link named gzip, gunzip or gzcat,
provides the same behavior as the related program.
gzip compresses using --format=gz
both gzip and gunzip enable --rm by default
2017-02-14 10:45:19 -08:00
Przemyslaw Skibinski ce13d087d9 fix LONG_SEEK 2017-02-14 09:52:52 +01:00
Przemyslaw Skibinski 970419535f fixed function name (2) 2017-02-14 09:47:29 +01:00
Przemyslaw Skibinski bf336572bf Avoid fseek()'s 2GiB barrier with MacOS and *BSD 2017-02-14 09:45:33 +01:00
Przemyslaw Skibinski abd6302423 Windows resources updated to v1.1.4 2017-02-14 09:39:09 +01:00
Przemyslaw Skibinski 442c75f132 removed UTIL_doesFileExists (replaced with UTIL_isRegFile) 2017-02-14 09:38:51 +01:00
Przemyslaw Skibinski b876b96ce1 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2017-02-14 09:26:03 +01:00
Przemyslaw Skibinski 98509a70ac fixed function name 2017-02-14 09:23:32 +01:00
Przemyslaw Skibinski 1a195b3b7a fixed unitialized variable warning 2017-02-13 22:56:31 +01:00
Przemyslaw Skibinski 48466b36e8 Resolve conflict with branch 'refs/remotes/facebook/dev'
# Conflicts:
#	programs/fileio.c
2017-02-13 21:35:39 +01:00
Yann Collet db2666c10c Merge pull request #536 from iburinoc/multiframe
Simple API multiframe decoding
2017-02-13 12:18:16 -08:00
Przemyslaw Skibinski 64f7221958 limit zlib compression level to Z_BEST_COMPRESSION 2017-02-13 21:00:41 +01:00
Przemyslaw Skibinski 5a3bb05bb2 Merge remote-tracking branch 'refs/remotes/facebook/dev' into gz_compress
# Conflicts:
#	programs/Makefile
2017-02-13 20:47:01 +01:00
Przemyslaw Skibinski 35bf23c086 MinGW-w64 requires _FILE_OFFSET_BITS 64 2017-02-13 13:57:29 +01:00
ds77 6220bfc924 fix indentation in previous commit 2017-02-13 12:00:59 +01:00
ds77 168d9b8006 fix seeking 2GB+ files under Windows
Replace fseek() in FIO_fwriteSparse() and FIO_fwriteSparseEnd() with macro expanding to 64-bit fseek version provided by the platform (includes  fallback workaround using Win32 API).
2017-02-12 16:05:19 +01:00
Yann Collet acfa151622 Merge pull request #540 from ds77/dev-stat64-fix
zstdcli: Fix reporting incorrect sizes of large flies on MinGW
2017-02-11 21:08:08 -08:00
Przemyslaw Skibinski eb132530cd revert last commit 2017-02-10 21:15:49 +01:00
Przemyslaw Skibinski 645f5b9856 fix for original MinGW 2017-02-10 20:09:28 +01:00
Sean Purcell 5069b6c2c3 Merge branch 'dev' into multiframe 2017-02-10 10:08:55 -08:00
ds77 45f0c207ab use _stati64() in UTIL_getFileSize() when compiling with mingw, get rid of introduces previously preprocessor checks. 2017-02-10 18:37:57 +01:00
Przemyslaw Skibinski 456330948a Merge remote-tracking branch 'refs/remotes/facebook/dev' into mingw
# Conflicts:
#	programs/platform.h
2017-02-10 13:38:54 +01:00
Yann Collet bbba42acd1 Merge pull request #537 from terrelln/small-bugs
Fix small bugs
2017-02-10 04:35:43 -08:00
- 7ec315df0d fix previous commit
* struct _stat64 is not defined by (non-w64) MinGW releases, __stat64 should be everywhere
* proper detection of _stat64() availability (as in MinGW sys/stat.h)
2017-02-10 13:27:43 +01:00
- 19f61b534e use _stat64 only when targetting Win2k or later 2017-02-10 10:56:45 +01:00
Przemyslaw Skibinski 429e13099a fix 64-bit file support for MinGW 2017-02-10 10:36:44 +01:00
Nick Terrell d08019813b Improvement from @inikep 2017-02-09 14:20:52 -08:00
ds77 9cde3f8b2e use _stat64() on MinGW
On MinGW, use _stat64() and struct _stat64 instead of stat() and struct stat_t. This fixes reporting incorrect sizes for large files.
2017-02-09 22:49:31 +01:00
Przemyslaw Skibinski 896638a8a2 echo replaced with printf 2017-02-09 17:01:17 +01:00
Przemyslaw Skibinski 13127fd05b don't use "echo -e" 2017-02-09 11:32:21 +01:00
Nick Terrell a0f9006e5a #undef _POSIX_C_SOURCE if already defined 2017-02-08 17:25:01 -08:00
Sean Purcell 0f5c95af44 Disambiguate pledgedSrcSize == 0
- Modify ZSTD CLI to only set contentSizeFlag if it _knows_ the size
- Change pzstd to stop setting contentSizeFlag without accurate pledgedSrcSize
2017-02-08 15:12:46 -08:00
Sean Purcell 4e709712e1 Decompressed size functions now handle multiframes and distinguish cases
- Add ZSTD_findDecompressedSize
    - Traverses multiple frames to find total output size
- Add ZSTD_getFrameContentSize
    - Gets the decompressed size of a single frame by reading header
- Deprecate ZSTD_getDecompressedSize
2017-02-08 14:50:10 -08:00
Przemyslaw Skibinski 93901fe85c remove redundant "ifeq ($(HAVE_ZLIB), 1)" 2017-02-08 21:11:18 +01:00
Przemyslaw Skibinski 4f9eaa7bb3 fixed gcc warnings 2017-02-08 18:08:09 +01:00
Przemyslaw Skibinski cb56306a50 .gz suffix for gzip compressed files 2017-02-08 17:37:14 +01:00
Przemyslaw Skibinski 02018c83cf added FIO_compressGzFrame 2017-02-08 16:54:23 +01:00
Przemyslaw Skibinski cfd4dc299a add "--format=gzip" option 2017-02-08 15:17:55 +01:00
Przemyslaw Skibinski ca20edd960 fixed zlib detection with MinGW 2017-02-08 14:32:49 +01:00
Przemyslaw Skibinski cdf5a7bd9f Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2017-02-08 13:49:35 +01:00
Przemyslaw Skibinski 40580ff669 added description of "--priority=rt" 2017-02-08 13:49:06 +01:00
Przemyslaw Skibinski 0665a359aa "--rt-prio" renamed to "--priority=rt" 2017-02-07 20:12:59 +01:00
Przemyslaw Skibinski d05014c739 added the "--rt-prio" option 2017-02-07 16:48:01 +01:00
Przemyslaw Skibinski 94abd6a26c SET_REALTIME_PRIORITY 2017-02-07 16:36:19 +01:00
Nick Terrell 7e3fc73795 Ensure <zlib.h> can be included in HAVE_ZLIB test 2017-02-06 11:54:31 -08:00
Nick Terrell 2cb8ee8784 Change zlib include to be a system include 2017-02-06 11:32:13 -08:00
Yann Collet f6e9c4f1e1 Merge pull request #529 from facebook/gzstd_default
Gzstd default
2017-02-05 18:03:36 -08:00
cyan4973 613087c02b Silence zlib detection routine
When it fails, $(CC) sends error message into stderr
redirected to /dev/null
2017-02-04 23:36:12 -08:00
Nick Terrell b4016ff02f Add cover dictionary training to zstd.1
Tested with `make install && man zstd` and visual inspection.
2017-02-03 16:42:07 -08:00
Yann Collet 762ddeeb9e fixed zstdmt compilation under Windows minGW/MSYS2, by @inikep 2017-02-03 14:35:42 -08:00
Yann Collet 21eb80d485 remove zlib detection artefact
result of compilation test is sent to /dev/null
2017-02-03 14:34:41 -08:00
Yann Collet b02ac8d613 fixed pointer conversion warnings (C++)
in gz module
2017-02-03 08:43:06 -08:00
Yann Collet c2a4632789 release builds use less debug symbols and warnings
release build are triggered through either `make`,
or their specific target `make zstd-release` and `make lib-release`.
2017-02-02 20:54:41 -08:00
Yann Collet c3cba9d858 fixed silent conversion warnings in GZDECOMPRESS path 2017-02-02 17:12:50 -08:00
Yann Collet 030ac243a0 Changed Makefile to generate zstd with .gz support by default
.gz support is detected by a runtime test.
2017-02-02 16:49:34 -08:00
Yann Collet 3dc85bae66 minor : fixed zstd-frugal
fixed a minor unused variable warning when compiling zstd-frugal target
2017-02-02 16:19:09 -08:00
Yann Collet 48bed91606 Merge pull request #527 from facebook/zstdmt
zstdmt refinements
2017-01-31 16:36:46 -08:00
Yann Collet 8d8513fb64 fixed C constant restrictions 2017-01-30 14:37:08 -08:00
Yann Collet 3672d06d06 zstdmt : section size is set to be a minimum of overlapSize
the minimum size condition size is applied transparently (no warning, no error)
like previous minimum section size condition (1 KB) which still applies.
2017-01-30 13:35:45 -08:00
Yann Collet 6ccd37c8d4 cli : added advanced parameter overlapLog
as a hidden (undocumented) parameter for now
2017-01-30 13:07:24 -08:00
Yann Collet 6be2337c26 added command --block-size=
for Multi-threading only.
alias : -B#
2017-01-30 11:17:26 -08:00
Yann Collet cc3d1bc262 Merge pull request #525 from terrelln/covermt
Multithreaded COVER dictionary training
2017-01-30 10:15:33 -08:00
Yann Collet 5d9b894e46 Fixed status display for zstdmt
There is a large buffering effect when using zstdmt in MT mode.
Consequently, data is read first, pushed to workers,
and only later will the compressed result come out.
That means there is no longer immediate correlation
between amount of data read, and amount of data written.

This patch disables the displaying of % compression
when multi-threading is enabled.

It adds the displaying of total size when it can be determined
(it usually can be determined for files, but not for stdin)
so the user has a sense of "how far from the end" the compression compressed is.

There is no modification to decompression side,
since decompression is only single-threaded for now.
2017-01-27 13:30:18 -08:00
Nick Terrell 2fe9126591 Add multithread support to COVER 2017-01-27 11:56:02 -08:00
Nick Terrell 9c018cc140 Add BUCK files for Nuclide support 2017-01-27 10:43:12 -08:00
Przemyslaw Skibinski eb2d23a90c improved #232 fix 2017-01-27 10:43:03 -08:00
Przemyslaw Skibinski 92a4dbf2e4 Fixed https://github.com/facebook/zstd/issues/232 2017-01-27 10:43:03 -08:00
Nick Terrell 83c387eb8e Fix zstdmt_compress.h include 2017-01-26 15:25:32 -08:00
Yann Collet 717c65d690 Merge pull request #519 from inikep/dev11
Dev11
2017-01-26 14:23:44 -08:00
Yann Collet 8dafb1acf5 CLI : automatically set overlap size to max (windowSize) for max compression level 2017-01-25 17:01:13 -08:00
Yann Collet bb0027405a fixed zstdmt corruption issue when enabling overlapped sections
see Asana board for detailed explanation on why and how to fix it
2017-01-25 16:25:38 -08:00
Yann Collet 943cff9c37 fixed zstdmt cli freeze issue with large nb of threads
fileio.c was continually pushing more content without giving a chance to flush compressed one.
It would block the job queue when input data was accumulated too fast (requiring to define many threads).
Fixed : fileio flushes whatever it can after each input attempt.
2017-01-25 12:35:19 -08:00
Przemyslaw Skibinski 5022a18d51 improved #232 fix 2017-01-25 13:11:26 +01:00
Przemyslaw Skibinski 64fa2dbc5e Fixed https://github.com/facebook/zstd/issues/232 2017-01-25 13:02:33 +01:00
Yann Collet 512cbe8c10 zstdmt cli and API allow selection of section sizes
By default, section sizes are 4x window size.
This new setting allow manual selection of section sizes.
The larger they are, the (slightly) better the compression ratio,
but also the higher the memory allocation cost,
and eventually the lesser the nb of possible threads,
since each section is compressed by a single thread.

It also introduces a prototype to set generic parameters,
ZSTDMT_setMTCtxParameter()

The idea is that it's possible to add enums
to extend the list of parameters that can be set this way.
This is more long-term oriented than a fixed-size struct.
Consider it as a test.
2017-01-24 17:08:53 -08:00
Yann Collet f8804d1014 convert tabs to space
joys of using multiple editors from multiple environments ...
2017-01-20 17:23:19 -08:00
Yann Collet d7e3cb58c5 Resolved merge conflict dev+zstdmt 2017-01-20 16:44:50 -08:00
cyan4973 5fba09fa41 updated util's time for Windows compatibility
Correctly measures time on Posix systems when running with
Multi-threading

Todo : check Windows measurement under multi-threading
2017-01-20 12:57:31 -08:00
Yann Collet 458c8a94b4 minor refactoring : cleaner MT integration within bench 2017-01-19 17:44:15 -08:00
Yann Collet 500014af49 zstd cli can now compress using multi-threading
added : command -T#
added : ZSTD_resetCStream() (zstdmt_compress)
added : FIO_setNbThreads()  (fileio)
2017-01-19 17:04:28 -08:00
Yann Collet 0f984d94c4 changed MT enabling macro to ZSTD_MULTITHREAD 2017-01-19 14:05:07 -08:00
Yann Collet f22adae984 fixed minor warning (unused variable) in fuzzer 2017-01-19 13:46:30 -08:00
Yann Collet 736788f8e8 added streaming fuzzer tests for MT API
Also : fixed corner case, where nb of jobs completed becomes > jobQueueSize
which is possible when many flushes are issued
while there is not enough dst buffer to flush completed ones.
2017-01-19 12:15:29 -08:00
Przemyslaw Skibinski 8a0bc30a2d Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2017-01-17 13:02:29 +01:00
Przemyslaw Skibinski d72f4b6b7a added "Makefile is validated" 2017-01-17 12:40:06 +01:00
Yann Collet 5eb749e734 ZSTDMT_compress() creates a single frame
The new strategy involves cutting frame at block level.
The result is a single frame, preserving ZSTD_getDecompressedSize()

As a consequence, bench can now make a full round-trip,
since the result is compatible with ZSTD_decompress().

This strategy will not make it possible to decode the frame with multiple threads
since the exact cut between independent blocks is not known.
MT decoding needs further discussions.
2017-01-11 18:21:25 +01:00
Nick Terrell c220d4c74d Use COVER_MEMMULT when training with COVER. 2017-01-09 16:49:04 -08:00
Yann Collet 56958500fc minor man page update 2017-01-09 19:47:09 +01:00
Nick Terrell 3a1fefcf00 Simplify COVER parameters 2017-01-02 17:51:38 -08:00
Nick Terrell df8415c502 Add COVER to the zstd cli 2017-01-02 14:43:08 -08:00
Yann Collet 6334b04d61 compile object files, for faster recompilation 2017-01-02 03:22:18 +01:00
Yann Collet 747452677d fixed cmake tests 2017-01-02 02:05:45 +01:00
Yann Collet cdb2763f4a new Makefile target zstdmt 2017-01-02 01:43:56 +01:00
Yann Collet f1cb55192c fixed linux warnings 2017-01-02 01:11:55 +01:00
Yann Collet 0ec6a95ba1 minor fixes 2017-01-02 00:49:42 +01:00
Yann Collet c6a6417458 bench correctly measures time for multi-threaded compression (posix only) 2016-12-31 03:31:26 +01:00
Yann Collet f765a375a5 Merge pull request #504 from terrelln/thread-pool
[zstdmt] Add thread pool
2016-12-30 15:31:49 +01:00
Yann Collet e70912c72b Changed : input divided into roughly equal parts.
Debug : can measure time waiting for mutexes to unlock.
2016-12-29 01:24:01 +01:00
Yann Collet 8d7432914f Merge pull request #503 from inikep/dev11
Dev11
2016-12-28 16:50:39 +01:00
Yann Collet ab7a579180 added -T command , to set nb of threads 2016-12-28 16:11:09 +01:00
Przemyslaw Skibinski 75f3a3a335 changed default PREFIX and MANDIR 2016-12-28 12:32:41 +01:00
Yann Collet 3d93f2fce7 first zstdmt sketch 2016-12-27 07:19:36 +01:00
Yann Collet c07d2e3a31 Merge pull request #499 from inikep/dev11
improved *BSD and Solaris compatibility
2016-12-23 21:32:03 +01:00
Przemyslaw Skibinski 63b0014b96 BSD: improved "make install" 2016-12-23 10:05:49 +01:00
Andrew Janke fce374a100 zstdless: add shebang and quote $@ 2016-12-22 17:40:10 -05:00
Przemyslaw Skibinski b999170311 Solaris: working "make -C lib install" 2016-12-22 20:14:37 +01:00
Przemyslaw Skibinski aab442133d Solaris: working "make -C programs install" 2016-12-22 19:26:01 +01:00
Yann Collet 8333106b8a Merge branch 'dev' of github.com:facebook/zstd into dev 2016-12-21 16:44:24 +01:00
Yann Collet 0d7e84899f Merge pull request #489 from inikep/v112
improved detection of POSIX
2016-12-21 16:42:46 +01:00
Yann Collet 9518744025 Merge branch 'dev' of github.com:facebook/zstd into dev 2016-12-21 16:20:41 +01:00
Yann Collet 1f57c2ed32 added : ZSTD_createCDict_byReference() 2016-12-21 16:20:11 +01:00
Przemyslaw Skibinski 7a8a03c20d util.h: restore BSD license for Facebook Open-Source 2016-12-21 15:08:44 +01:00
Przemyslaw Skibinski 97a258d71d updated comments 2016-12-21 14:00:41 +01:00
Przemyslaw Skibinski e679741b18 _CRT_SECURE_NO_WARNINGS moved to util.h 2016-12-21 13:47:11 +01:00
Przemyslaw Skibinski 2f6ccee6af platform.h: removed Compiler Options 2016-12-21 13:23:34 +01:00
Przemyslaw Skibinski 101df4f636 fixed Visual Studio compilation 2016-12-21 11:43:11 +01:00
Przemyslaw Skibinski a35b9448ac improved MinGW support 2016-12-21 11:18:45 +01:00
Przemyslaw Skibinski 5736db219e fix basic types redefinition 2016-12-21 09:26:00 +01:00
Przemyslaw Skibinski 20b089e53d simplified zstdcli.c 2016-12-21 09:19:15 +01:00
Przemyslaw Skibinski 16ae6563a2 executables use new util.h and platform.h 2016-12-21 09:06:14 +01:00
Przemyslaw Skibinski ead350bdc0 improved util.h and platform.h 2016-12-21 09:04:59 +01:00
Przemyslaw Skibinski 1c1db6b845 windres updated to v1.1.3 2016-12-20 11:21:26 +01:00
Przemyslaw Skibinski f8046b8e72 Merge remote-tracking branch 'refs/remotes/facebook/dev' into v112
# Conflicts:
#	appveyor.yml
2016-12-19 08:20:26 +01:00
Yann Collet 1496c3dc47 Fix : size estimation when some samples are very large 2016-12-18 11:58:23 +01:00
Yann Collet d46ecb58a5 added dll compilation tests 2016-12-17 16:28:12 +01:00
Przemyslaw Skibinski 0b37205098 util.h: minor improvement 2016-12-16 17:12:23 +01:00
Przemyslaw Skibinski b0f3663edc imporved support for POSIX-type OSes 2016-12-16 15:41:18 +01:00
Przemyslaw Skibinski 3cdfe266cf use PLATFORM_POSIX_VERSION 2016-12-16 15:00:50 +01:00
Przemyslaw Skibinski b0e670a054 util.h uses platform.h 2016-12-16 14:25:12 +01:00
Przemyslaw Skibinski b866e72826 tools use platform.h 2016-12-16 14:24:01 +01:00
Przemyslaw Skibinski b3843afcf5 introduced platform.h 2016-12-16 14:13:15 +01:00
Przemyslaw Skibinski 25314428c9 zstd.1: added advanced compression options 2016-12-14 16:10:13 +01:00
Przemyslaw Skibinski c71e552b2e fixed "strategy" in advanced compression parameters 2016-12-13 20:04:32 +01:00
Przemyslaw Skibinski 98ef0f98df fixed conversion warning 2016-12-13 14:52:21 +01:00
Przemyslaw Skibinski 8349d675e0 fileio.c: support advanced compression parameters 2016-12-13 13:24:59 +01:00
Przemyslaw Skibinski 897b8bb5eb bench.c: support advanced compression parameters 2016-12-13 13:03:41 +01:00
Przemyslaw Skibinski 334b8962b0 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2016-12-13 12:19:19 +01:00
Przemyslaw Skibinski 671f28d1e5 added parseCompressionParameters 2016-12-13 12:18:07 +01:00
Dimitry Andric 83cc2fb083 Enable using isatty() and nanosleep() on *BSD 2016-12-12 19:24:51 +01:00
Yann Collet 4c6ce5a57e Merge pull request #476 from inikep/dev11
Dev11
2016-12-12 16:44:55 +01:00
Przemyslaw Skibinski c855da7533 allow all operationg systems with _POSIX_C_SOURCE >= 200112L 2016-12-12 15:37:43 +01:00
Przemyslaw Skibinski 242c036875 turn on the '-r' option for HP-UX and AIX 2016-12-12 11:59:17 +01:00
Przemyslaw Skibinski c01ac6c30a turn on the '-r' option for *BSD and Solaris 2016-12-12 11:23:21 +01:00
Yann Collet fda539f50b minor coding style changes 2016-12-12 01:03:23 +01:00
Przemyslaw Skibinski 128acb35db improved MSYS support 2016-12-08 10:49:59 +01:00
Przemyslaw Skibinski 13de72e0cc programs\Makefile: use Linux paths 2016-12-08 10:43:55 +01:00
Yann Collet 2a826e592b compatibility with zstd-frugal (noBench mode) 2016-12-06 17:56:20 -08:00
Yann Collet e63c631aaf decode benchmark, multi-files 2016-12-06 17:46:49 -08:00
Yann Collet d946501d2c decode benchmark - single file (hidden option) 2016-12-06 16:49:23 -08:00
Yann Collet 6b9a983261 changed gzstd build messages 2016-12-06 11:23:25 -08:00
Przemyslaw Skibinski 7c6bbc3298 updated FIO_passThrough 2016-12-05 18:31:14 +01:00
Przemyslaw Skibinski 6b508b1770 updated test-gzstd 2016-12-05 18:02:40 +01:00
Przemyslaw Skibinski b493e3b3d3 introduced srcBufferLoaded 2016-12-05 17:39:38 +01:00
Przemyslaw Skibinski 3c69760275 improved FIO_decompressGzFrame 2016-12-05 15:58:23 +01:00
Przemyslaw Skibinski 8489f184f6 improved detection of -lz 2016-12-05 13:47:00 +01:00
Yann Collet db85a6e09a added zstdgrep 2016-12-02 15:57:07 -08:00
Yann Collet efaf104b2d added zstdless 2016-12-02 15:24:40 -08:00
Yann Collet 743b33f57e fix zstdcat 2016-12-02 15:18:57 -08:00
Yann Collet 5bd4237beb minor refactor 2016-12-02 12:40:57 -08:00
Przemyslaw Skibinski 690753ea1a improved formatting in FIO_decompressSrcFile 2016-12-02 16:20:16 +01:00
Przemyslaw Skibinski 4e49580407 removed testing artifacts 2016-12-02 15:19:00 +01:00
Przemyslaw Skibinski c5eebca128 rewritten FIO_decompressGzFile 2016-12-02 15:01:31 +01:00
Przemyslaw Skibinski b0f2ef2119 improved gzip_* functions 2016-12-02 13:50:29 +01:00
Przemyslaw Skibinski 4b504f131a added gzip_open, gzip_close, gzip_read 2016-12-02 13:11:39 +01:00
Przemyslaw Skibinski daaf754539 detect stream with ungetc 2016-12-01 13:29:19 +01:00
Przemyslaw Skibinski 19aad42ee1 added FIO_decompressGzFile 2016-12-01 11:56:31 +01:00
Przemyslaw Skibinski 166830ed0a autodetect -lz 2016-11-30 16:43:07 +01:00
Przemyslaw Skibinski abfb51f5f2 gzstd: decompresses .gz files 2016-11-30 15:05:54 +01:00
Przemyslaw Skibinski 0e14675df2 fileio.c: detect .gz files 2016-11-30 13:34:21 +01:00
Yann Collet 167c494748 Merge branch 'dev' of github.com:facebook/zstd into dev 2016-11-29 14:05:15 -08:00
Yann Collet 4f5350f610 long matches support overflow 2016-11-29 13:12:24 -08:00
Przemyslaw Skibinski fd0ac93024 bench.c: use ZSTD_maxCLevel() 2016-11-23 21:45:29 +01:00
Przemyslaw Skibinski 5ddcd9d9ae bench.c: fixed MAX_CLEVEL 2016-11-21 16:37:56 +01:00
Przemyslaw Skibinski 5a17223691 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2016-11-18 11:47:01 +01:00
Przemyslaw Skibinski 2558b4cdbc bench.c without dict uses ZSTD_compressCCtx 2016-11-18 11:46:30 +01:00
Przemyslaw Skibinski 3d18088b38 updated windres 2016-11-17 18:04:41 +01:00
Yann Collet 52afb3993e zbuff API now generates deprecation warnings 2016-11-16 08:50:54 -08:00
Yann Collet cdff19c4b3 minor comment change 2016-11-11 17:26:54 -08:00
Yann Collet 0018ca28dc zstd cli : displays total decoded size, even when a stream consists of multiple frames 2016-11-07 14:42:37 -08:00
Przemyslaw Skibinski d0815583d9 Changed stdinmark and stdoutmark 2016-11-04 11:37:27 +01:00
Przemyslaw Skibinski 38b590ad69 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
# Conflicts:
#	lib/Makefile
2016-11-04 10:10:54 +01:00
Yann Collet 407a11f63e fixed Visual compatibility 2016-11-03 15:52:01 -07:00
Przemyslaw Skibinski 3a415594b1 fixed MinGW compilation 2016-11-03 12:59:20 +01:00
Przemyslaw Skibinski 1f60eb6b17 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2016-11-03 11:38:04 +01:00
Przemyslaw Skibinski 26306fcacf BMK_SetNbIterations renamed to BMK_SetNbSeconds 2016-11-03 11:38:01 +01:00
Przemyslaw Skibinski b40884f43d preserve file modification time for Visual C++ 2016-11-03 09:54:53 +01:00
Yann Collet 7347869fb6 fixed make install 2016-11-02 22:28:37 -07:00
Yann Collet 179b19776f fileio.c does no longer need ZSTD_LEGACY_SUPPORT, and does no longer depend on zstd_legacy.h
Added : ZSTD_isFrame() in experimental section
2016-11-02 17:30:49 -07:00
Yann Collet f3f13211ae Fix #419 : no warning when setting custom LDFLAGS 2016-11-02 17:02:45 -07:00
Przemyslaw Skibinski fcf22e3473 set permissions, access and modification times 2016-11-02 14:08:07 +01:00
Przemyslaw Skibinski a42794df61 preserve file modification time 2016-11-02 13:08:39 +01:00
Przemyslaw Skibinski d872b64f52 added UTIL_setModificationTime, UTIL_getModificationTime 2016-11-02 12:52:20 +01:00
Yann Collet 3d197d4c8c updated man page 2016-10-31 18:10:32 -07:00
Yann Collet b9550d6d43 fixed benchmark mode typo .
added benchmark mode tests
2016-10-28 14:43:24 -07:00
Yann Collet 22de81e87e updated man page 2016-10-28 13:58:31 -07:00
Przemyslaw Skibinski bd0bd0fb65 generate RES files during each compilation 2016-10-28 18:15:34 +02:00
Przemyslaw Skibinski 589a8f27b7 generate_res.bat moved to programs/windres 2016-10-28 13:17:35 +02:00
Przemyslaw Skibinski 76d66dffd0 generate_res moved to programs/windres 2016-10-28 13:16:06 +02:00
Przemyslaw Skibinski 347903cfb5 Windows resources moved to programs/windres/ 2016-10-25 10:34:41 +02:00
Yann Collet a7a4690b0a use implicit rules to compile zstd_decompress.c 2016-10-18 16:01:03 -07:00
Yann Collet 33fdd099bb fix command line interpretation 2016-10-17 17:48:48 -07:00
Yann Collet 70077bc9bb refactor for long commands 2016-10-14 14:41:17 -07:00
Yann Collet d7b120ab5c added long commands --memory= and --memlimit-decompress= 2016-10-14 14:22:32 -07:00
Yann Collet 1122349ac2 added long comment --memlimit= 2016-10-14 14:07:11 -07:00
Yann Collet d4cda27b63 new command -M#, to limit memory usage during decompression (#403) 2016-10-14 13:32:20 -07:00
Yann Collet e19111c42f make creates libzstd binaries (#415) 2016-10-12 11:09:36 -07:00
Nick Terrell 3e82f2b61a Remove references to GPLv2 license
License headers added to Makefiles were taken from `zstd/Makefile`.
2016-10-11 15:28:40 -07:00
inikep 23cec875cd fixed Makefile targets: zstd-small, zstd-decompress, zstd-compress 2016-10-05 11:56:22 +02:00
Nick Terrell 2b4de225e1 Don't redefine macro in util.h 2016-09-22 18:02:39 -07:00
Nick Terrell 5b2c0dbed0 Add include guards to datagen.h 2016-09-22 17:12:50 -07:00
Yann Collet 993060e0f2 cli : better adaptation to small files 2016-09-21 16:46:08 +02:00
Yann Collet 714464f05d fixed : cli : forgotten mandatory argument 2016-09-21 16:05:03 +02:00
Yann Collet 27b5ac666e Implemented "command must be followed by argument" protection
suggested by @terrelln (#375)
2016-09-21 14:31:22 +02:00
Yann Collet 0977f7ece6 minor refactor for clarity 2016-09-21 12:24:43 +02:00
Yann Collet dbe70bad48 completed change from projects to build 2016-09-19 15:08:43 +02:00
Yann Collet 60038948e6 added -- command in help 2016-09-16 18:52:52 +02:00
inikep 3e70cf8036 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev 2016-09-15 20:41:40 +02:00
inikep 6173931868 fixed memory leak reported by bryongloden 2016-09-15 18:58:18 +02:00
Yann Collet a6bdf55759 fixed memory leak 2016-09-15 17:02:06 +02:00
Yann Collet 43eeea4725 fileio uses ZSTD_resetCStream() 2016-09-15 15:38:44 +02:00
Yann Collet 0b01a8ef09 Merge pull request #366 from inikep/dev
zstd.exe has FileVersion and ProductVersion with 32-bit gcc (MinGW)
2016-09-14 19:42:29 +02:00
Yann Collet 3ecbe6a37c fileio uses ZSTD_resetDStream() 2016-09-14 17:26:59 +02:00
inikep c03f15e89d zstd.exe has FileVersion and ProductVersion with 32-bit gcc (MinGW) 2016-09-14 16:16:24 +02:00
Yann Collet 64a84edef5 added -r support for Mac OS-X 2016-09-13 17:54:37 +02:00
Yann Collet c4cc9bf973 -r generates an error on systems which do not support it 2016-09-13 17:50:08 +02:00
inikep 9f25fcf804 fixed precision warnigns 2016-09-13 16:38:54 +02:00
inikep 362708d4d2 zstd.exe has FileVersion and ProductVersion 2016-09-13 13:53:43 +02:00
inikep 1c5ba8a5e7 util.h: removed dependency from PATH_MAX 2016-09-13 13:13:10 +02:00
Yann Collet a3481d6de0 make uninstall 2016-09-12 05:04:26 +02:00
Yann Collet d77f2ceee6 Merge branch 'dev' of github.com:Cyan4973/zstd into dev 2016-09-12 03:14:03 +02:00
Yann Collet 5c6d244973 Merge branch 'dev' of github.com:facebook/zstd into dev 2016-09-07 14:54:54 +02:00
Yann Collet ac8bace6b1 support large skippable frames 2016-09-07 14:54:23 +02:00
Yann Collet 03d3f238de minor comment 2016-09-07 07:01:33 +02:00
Yann Collet aad9fe5470 don't remove() /dev/null (#316) 2016-09-07 07:00:08 +02:00
Yann Collet 7ae67bb18a small compression speed gains with using_CDict 2016-09-06 06:28:05 +02:00
Thomas Klausner b85cdabd50 Enable install targets for NetBSD. 2016-09-04 14:37:57 +02:00
Yann Collet 33a0465a51 fixed a few links 2016-09-02 22:11:49 -07:00
Yann Collet 5b8c024716 fixed zstd-pgo (#329) reported by @octoploid 2016-09-02 09:44:00 -07:00
Yann Collet 60181e3aaf zstd cli correctly detects console on Mac OS-X 2016-09-01 17:14:02 -07:00
Yann Collet c843142ad0 zstd -d writes to stdout when input is stdin 2016-09-01 15:05:57 -07:00
Yann Collet 1c59c20903 removed redundant files 2016-08-31 07:15:44 -07:00
Yann Collet 599c69d917 minor Makefile updates 2016-08-30 13:33:20 -07:00
Yann Collet 4ded9e591c added boilerplate 2016-08-30 11:06:28 -07:00
Yann Collet 09c3c8e885 Merge pull request #307 from inikep/dev08
updated README.md
2016-08-29 16:32:33 -07:00
inikep 6416b0d705 updated README.md 2016-08-29 13:04:26 +02:00
Yann Collet 54caf6893e Removed fileio_legacy; now legacy supports relies entirely on streaming API.
CLI legacy support now limited to v0.4+
2016-08-28 19:20:13 -07:00
Yann Collet 9097f7b156 fixed versionsTest 2016-08-28 16:25:56 -07:00
Yann Collet 1c69baa1a9 CLI supports Legacy formats v0.4+ 2016-08-28 12:47:17 -07:00
Yann Collet 5f53b0335e fixed continuation context 2016-08-28 10:00:49 -07:00
Yann Collet ab267e772c removed fileio specific legacy support;
now depends on ZSTD_decompressStream() transparent legacy support
2016-08-28 08:46:25 -07:00
Yann Collet 87c18b2ebd fixed multiple minor warnings for XCode 2016-08-26 01:43:47 +02:00
Yann Collet 0baa64a763 increased maximum memory size for 64-bits bench to 16 GB 2016-08-25 22:54:13 +02:00
inikep de9d13068f updated zstd_compression_format.md 2016-08-25 14:59:08 +02:00
inikep 0132375ba6 improved zstd_compression_format.md 2016-08-25 12:20:38 +02:00
inikep 637d33566d minor README.md corrections 2016-08-25 10:42:49 +02:00
inikep ab2f7708c1 created programs\README.md 2016-08-25 10:07:20 +02:00
inikep e416e30019 remove unnecessary comments 2016-08-24 17:32:09 +02:00
inikep 7bcdbcf51d Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into Other 2016-08-24 17:09:19 +02:00
Yann Collet d1733f7417 fixed crc bug in rare timing conditions within bench.c 2016-08-21 01:04:46 +02:00
inikep c629776b31 make clean will remove decompress/zstd_decompress.o 2016-08-19 10:45:59 +02:00
Yann Collet b571a0eab4 Merge pull request #298 from inikep/Other
Other
2016-08-18 17:14:03 +02:00
inikep b6514d2496 improved .gitignore 2016-08-18 16:53:29 +02:00
Yann Collet 49d105cfcf better warning and error messages in case of dictionary training failure (#292) 2016-08-18 15:02:11 +02:00
inikep 2868d565e4 test-related files moved to tests/ 2016-08-18 13:18:11 +02:00
Yann Collet e80d15304a Merge pull request #296 from inikep/Other
Other
2016-08-18 11:48:48 +02:00
inikep a7bb322a93 removed never referenced functions 2016-08-18 10:30:21 +02:00
inikep 5a54870047 fixed Intel Compiler warnings with Visual Studio
http://encode.ru/threads/2119-Zstandard?p=49504&viewfull=1#post49504
2016-08-18 09:00:25 +02:00
Yann Collet bfdc34e5a3 Merge pull request #294 from inikep/Other
improved Makefiles
2016-08-17 19:41:56 +02:00
inikep 79b4062c81 fixed paramgrill dependency 2016-08-17 16:29:57 +02:00
inikep d0c270fe85 improved programs\Makefile 2016-08-17 15:59:50 +02:00
inikep 78f3e0630b disabled %llu warnings with MinGW on Windows 2016-08-17 14:52:11 +02:00
Yann Collet 3d9211e961 Merge pull request #290 from david-y-lam/fix_file_name
Remove file name truncation in zstd CLI output
2016-08-17 14:05:29 +02:00
Yann Collet 53e17fbd5e updated streaming API 2016-08-17 01:39:22 +02:00
Yann Collet 94ca85d01b fixed Visual warning 2016-08-16 15:11:28 +02:00
Yann Collet 6263ba5451 fileio uses new streaming API 2016-08-16 15:11:28 +02:00
Yann Collet fb5e385164 minor changes 2016-08-16 15:11:28 +02:00
Yann Collet d7883a2c0e added zstreamtest 2016-08-16 15:11:28 +02:00
David Lam 0f2704581e Remove truncation of filename in zstd CLI log lines 2016-08-13 11:26:21 -07:00
David Lam ac43e9582e Add more descriptive message to -v option in zstd CLI 2016-08-12 17:47:18 -07:00
Yann Collet fab023010d changed test, in compliance with new --ultra policy 2016-08-12 19:00:18 +02:00
Yann Collet fe0590f63f new --ultra policy 2016-08-12 18:04:15 +02:00
inikep 7132fb15ba bench.c: removed benchResult_t 2016-08-10 14:59:18 +02:00
inikep 3d2c58c3a9 fixed compilation with Visual in C++ mode 2016-08-10 14:28:47 +02:00
Yann Collet 666398e7ed added : xxhash namespace enforced from xxhash.h.
added : xxhash namespace test.
removed : -DXXH_NAMESPACE
2016-08-10 08:16:51 +02:00
Yann Collet 6a21971f4a bench : implemented avgSize 2016-08-03 00:06:24 +02:00