Yann Collet
f47284fec0
reorganized Makefile for multiple targets
2017-05-02 16:55:57 -07:00
Yann Collet
3791d2105d
added xzstd4 target
...
support for all formats, xz/lzma/lz4 included
2017-05-02 16:38:37 -07:00
Yann Collet
2e63a877f3
fixed xzstd
...
--format=xz was missing a break, making the execution continue into lz4 error message
2017-05-02 15:40:42 -07:00
Nick Terrell
f376d47c11
[CLI] Switch dictionary builder on CLI to cover
2017-05-02 11:18:27 -07:00
Nick Terrell
020b960e13
[cover] Make optimization faster
2017-05-02 11:02:48 -07:00
Nick Terrell
f2d9ef1dc0
[cover] Optimize case where d <= 8
2017-05-02 11:02:43 -07:00
Nick Terrell
feba969a69
Fix LZ4_MSG in xzstd
2017-05-02 11:02:16 -07:00
Yann Collet
31533bacce
Changed ZSTD_createCDict_advanced()
...
It now only uses compressionParameters as argument.
It produces many changes throughout user code,
though hopefully they tend to be simple :
just provide the cParams part from existing ZSTD_parameters.
Some programs might depend on ZSTD_createCDict_advanced() to pass frame parameters.
This change will force them to revisit this strategy and fix it,
since frame parameters are effectively silently ignored in current version.
2017-04-27 00:29:04 -07:00
Sean Purcell
eab41c1872
Fix LZ4 wrapper deprecation warnings
2017-04-26 10:17:38 -07:00
Sean Purcell
2c4b6fe6b3
Make lz4 compression/decompression compatible with library r123
2017-04-25 11:00:54 -07:00
Sean Purcell
4de8632957
Add LZ4 compress/decompress support to CLI
2017-04-24 16:48:25 -07:00
Yann Collet
230d7acc7d
cli : add support for --threads=# command
...
updated documentation
add relevant test case
2017-04-21 11:38:13 -07:00
Sean Purcell
9606256a8d
Fix no thread message
2017-04-18 13:52:00 -07:00
Yann Collet
1c286950af
Merge pull request #663 from iburinoc/zstdmt-default
...
Compile CLI using files instead of objs
2017-04-18 02:59:54 -07:00
Sean Purcell
c8b2df7d62
Compile CLI using files instead of objs
...
This avoids conflicts between how the library was
configured and how the CLI was configured.
2017-04-17 17:13:47 -07:00
Sean Purcell
e6c504dbe6
Update -T0 comment in man page
2017-04-17 17:12:02 -07:00
Sean Purcell
5935c990a0
Add zstdmt and -T0 to man page
2017-04-17 16:05:20 -07:00
Sean Purcell
f6ef4db20e
Install zstdmt even without threading support
2017-04-17 12:21:11 -07:00
Sean Purcell
5a61f36474
Make zstd compile with mt by default
2017-04-17 12:06:02 -07:00
Baptiste Daroussin
7dd14d03b0
Enable multithreading on BSD
2017-04-15 16:25:08 +02:00
Sean Purcell
e4f3235c85
Add 0 initializers to static variables
2017-04-13 16:34:28 -07:00
Sean Purcell
42bac7fa84
Change ifndef's to undef's
2017-04-13 15:35:05 -07:00
Sean Purcell
9227aae001
Fix clang linux compilation
2017-04-13 14:06:40 -07:00
Sean Purcell
3b6207d4bd
Fix compilation on windows
2017-04-13 14:04:10 -07:00
Sean Purcell
f876f1200c
Fix compilation on macOS
2017-04-13 12:33:45 -07:00
Sean Purcell
afa48518e2
-T0 detects number of physical cores
2017-04-13 12:28:28 -07:00
Yann Collet
5c42d0edc8
cli : better status display for zstdmt in 1-thread mode
2017-04-11 16:57:32 -07:00
Yann Collet
0e30059ba1
cli : FIO_createDictBuffer() replaces FIO_loadFile()
...
makes it more explicit that it allocates a buffer
and that it's meant to be used for dictionary.
Also : simplified function a bit,
now only works for dictionaries up to DICTSIZE_MAX
2017-04-11 14:41:02 -07:00
Sean Purcell
279be2015b
Let zstd overwrite read-only files
2017-04-06 12:56:40 -07:00
Yann Collet
c2007388a5
fixed bench.c : optional advanced parameters applied
...
before creating cdict
2017-04-04 15:35:06 -07:00
Yann Collet
81d6380139
minor bench.c adjustments
...
shorter debug messages
no need to check decompressedLength==0 twice
2017-04-04 15:21:09 -07:00
Yann Collet
a36330a27d
minor man update, for `overlapSize`
2017-04-03 14:57:24 -07:00
Yann Collet
805c5a3efb
updated documentation on multithreading modes
2017-04-01 00:36:31 -07:00
Nick Terrell
eb38617175
Clean up default sparse logic
2017-03-31 15:20:50 -07:00
Nick Terrell
96fe545a18
Turn off sparse mode for OS X by default
2017-03-31 15:16:43 -07:00
Nick Terrell
5152fb2cb2
Convert all tabs to spaces
2017-03-29 18:51:58 -07:00
Yann Collet
5bde4be5a1
fix : bench automatically adapts parameters to srcSize
2017-03-29 12:10:38 -07:00
Sean Purcell
efdaf8bb7c
Fix inline compile errors
2017-03-27 12:26:40 -07:00
Sean Purcell
894bf49713
Fix IS_CONSOLE returning 1 for NUL on windows
2017-03-27 12:19:30 -07:00
Yann Collet
0246d5c531
Merge pull request #630 from facebook/advancedCliCommands
...
changed advanced commands --maxdict= and --dictID=
2017-03-26 00:13:35 -07:00
Sean Purcell
ef30af843f
Ignore extension in command name matching
2017-03-24 17:06:09 -07:00
Yann Collet
96aa3019b2
changed advanced commands --maxdict= and --dictID=
...
now works with the `=` variant, which is the recommended one.
Old variant `--dictID #` still works, for compatibility with existing scripts.
Long term objective is to remove the old variant..
2017-03-24 16:04:29 -07:00
Yann Collet
f332ece468
dictBuilder fails to create dictionary on certain input
...
Properly expressed with an error code (see zstd_errors.h)
and a cli return code != 0
2017-03-23 16:24:02 -07:00
Sean Purcell
dee08caabb
Fix displayLevel redefinition issues
2017-03-23 12:09:35 -07:00
Sean Purcell
680e4e0953
Ignore symbolic links unless --force specified
2017-03-23 11:52:09 -07:00
Sean Purcell
042ba122ae
Change g_displayLevel to int and fix DISPLAYUPDATE flush
2017-03-23 11:21:59 -07:00
Nick Terrell
dfc75f74da
Add $(EXT) to binary to fix zlib detection for some MinGW versions
2017-03-22 13:24:52 -07:00
Nick Terrell
741e3e8314
Add $(FLAGS) when testing for zlib/lzma support.
...
* If zlib/lzma isn't in the usual spot, it won't be used,
even if `$CFLAGS` and `$LDFLAGS` add the location it is in.
* Update the test code snippets to not trigger any warnings.
2017-03-22 12:33:08 -07:00
Yann Collet
35c147ab6a
Merge pull request #605 from iburinoc/term
...
Fix zstd not erroring on compressing to terminal without forceStdout
2017-03-17 14:02:48 -07:00
Yann Collet
0a189b63fe
fix minor details in man page
2017-03-16 16:33:53 -07:00
Sean Purcell
b8e52d3c83
Fix zstd not erroring on compressing to terminal without forceStdout
2017-03-16 16:06:19 -07:00
Sean Purcell
72a243e3bd
Convert manpage to Markdown, converted with ronn
2017-03-16 15:37:32 -07:00
Yann Collet
3773bd318f
Merge pull request #601 from terrelln/nxz
...
Add xz and lzma support.
2017-03-13 22:00:03 -07:00
Nick Terrell
aa8bcf360f
Add xz and lzma support.
...
Finish feature started by @inikep.
* Add xz and lzma compression and decompression support to target `xzstd`.
* Fix bug in gzip decompression that silently accepted truncated files.
* Add gzip frame composition tests.
* Add xz/lzma compatibility tests.
* Add xz/lzma frame composition tests.
2017-03-13 18:45:41 -07:00
Sean Purcell
120df494e9
Update builds to not support legacy v01-v03
2017-03-13 14:44:08 -07:00
Sean Purcell
334cb34edb
ZSTD_LEGACY_SUPPORT defines lowest supported version
2017-03-13 14:32:30 -07:00
Dmitry V. Levin
201e8c8157
programs/Makefile: remove zstd-internal target
...
zstd-internal was intended to be a helper target, but it doesn't help
at all, what it does in practice is a useless rebuild of zstd every time
"make zstd" is invoked.
Fixes: 030ac243a0
("Changed Makefile to generate zstd with .gz support by default")
2017-03-09 22:47:14 +00:00
Nick Terrell
eeb31eed39
s/ZSTD_btopt2/ZSTD_btultra/g
2017-03-09 11:44:25 -08:00
Yann Collet
8b1d004031
added -Wformat-security flag, as recommended by @pixelb
2017-03-05 21:17:32 -08:00
Yann Collet
78208bd8be
fixed : build zstd cli after libzstd
2017-03-01 21:02:06 -08:00
Yann Collet
c1c040eae1
added gzip tests
...
also : made sure zstd --format=gzip -V
would fail if gzip compatibility is not supported
2017-03-01 16:49:20 -08:00
Przemyslaw Skibinski
8e5032a965
cli : fix : --rm is silent when input is stdin (decompression)
2017-02-28 09:42:37 +01:00
Yann Collet
a81c2e7e44
Merge pull request #573 from facebook/ddict
...
Improved DDict memory usage
2017-02-27 20:54:42 -08:00
Yann Collet
ef569bf75f
Merge branch 'dev' of github.com:facebook/zstd into dev
2017-02-27 15:58:38 -08:00
Yann Collet
dccd6b6f65
cli : fix : --rm is silent when input is stdin
...
previously, app would produce an error message, and stop.
2017-02-27 15:57:50 -08:00
Przemyslaw Skibinski
862698f479
minor tweaks in FIO_decompressGzFrame
2017-02-27 13:21:05 +01:00
Yann Collet
0b9b894b2d
reduced ZSTD_DDict memory usage
...
saved 128 KB
2017-02-27 00:27:30 -08:00
Przemyslaw Skibinski
337ec875b6
minor tweaks
2017-02-22 10:31:30 +01:00
Przemyslaw Skibinski
684858e7b7
fix memory leaks
2017-02-21 18:17:24 +01:00
Przemyslaw Skibinski
74dcd8d15f
bench.c: use a single ticksPerSecond
2017-02-21 12:22:05 +01:00
Przemyslaw Skibinski
1b59333c82
util.h: restore times()
2017-02-21 07:33:45 +01:00
Przemyslaw Skibinski
da4a0f30af
util.h: use mach_absolute_time for macOS
2017-02-20 12:18:15 +01:00
Przemyslaw Skibinski
e052c60540
introduce UTIL_freq_t
2017-02-20 11:27:11 +01:00
Przemyslaw Skibinski
83775d9e05
replace times() with clock_gettime(CLOCK_MONOTONIC, x)
2017-02-20 11:11:50 +01:00
Przemyslaw Skibinski
acb6e57ad2
use FindFirstFileA instead of FindFirstFile
2017-02-15 17:13:35 +01:00
Przemyslaw Skibinski
6e59b3ce01
added UTIL_fseek
2017-02-15 17:03:16 +01:00
Przemyslaw Skibinski
3aaa1dae4e
simplified zlib detection
2017-02-15 09:17:39 +01:00
Yann Collet
f0b9a8dddb
Merge pull request #547 from inikep/dev11
...
Avoid fseek()'s 2GiB barrier with MacOS and *BSD
2017-02-14 12:29:00 -08:00
Yann Collet
c09d16ba8c
preset behavior for gzip, gunzip and gzcat
...
when zstd is called through a link named gzip, gunzip or gzcat,
provides the same behavior as the related program.
gzip compresses using --format=gz
both gzip and gunzip enable --rm by default
2017-02-14 10:45:19 -08:00
Przemyslaw Skibinski
ce13d087d9
fix LONG_SEEK
2017-02-14 09:52:52 +01:00
Przemyslaw Skibinski
970419535f
fixed function name (2)
2017-02-14 09:47:29 +01:00
Przemyslaw Skibinski
bf336572bf
Avoid fseek()'s 2GiB barrier with MacOS and *BSD
2017-02-14 09:45:33 +01:00
Przemyslaw Skibinski
abd6302423
Windows resources updated to v1.1.4
2017-02-14 09:39:09 +01:00
Przemyslaw Skibinski
442c75f132
removed UTIL_doesFileExists (replaced with UTIL_isRegFile)
2017-02-14 09:38:51 +01:00
Przemyslaw Skibinski
b876b96ce1
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
2017-02-14 09:26:03 +01:00
Przemyslaw Skibinski
98509a70ac
fixed function name
2017-02-14 09:23:32 +01:00
Przemyslaw Skibinski
1a195b3b7a
fixed unitialized variable warning
2017-02-13 22:56:31 +01:00
Przemyslaw Skibinski
48466b36e8
Resolve conflict with branch 'refs/remotes/facebook/dev'
...
# Conflicts:
# programs/fileio.c
2017-02-13 21:35:39 +01:00
Yann Collet
db2666c10c
Merge pull request #536 from iburinoc/multiframe
...
Simple API multiframe decoding
2017-02-13 12:18:16 -08:00
Przemyslaw Skibinski
64f7221958
limit zlib compression level to Z_BEST_COMPRESSION
2017-02-13 21:00:41 +01:00
Przemyslaw Skibinski
5a3bb05bb2
Merge remote-tracking branch 'refs/remotes/facebook/dev' into gz_compress
...
# Conflicts:
# programs/Makefile
2017-02-13 20:47:01 +01:00
Przemyslaw Skibinski
35bf23c086
MinGW-w64 requires _FILE_OFFSET_BITS 64
2017-02-13 13:57:29 +01:00
ds77
6220bfc924
fix indentation in previous commit
2017-02-13 12:00:59 +01:00
ds77
168d9b8006
fix seeking 2GB+ files under Windows
...
Replace fseek() in FIO_fwriteSparse() and FIO_fwriteSparseEnd() with macro expanding to 64-bit fseek version provided by the platform (includes fallback workaround using Win32 API).
2017-02-12 16:05:19 +01:00
Yann Collet
acfa151622
Merge pull request #540 from ds77/dev-stat64-fix
...
zstdcli: Fix reporting incorrect sizes of large flies on MinGW
2017-02-11 21:08:08 -08:00
Przemyslaw Skibinski
eb132530cd
revert last commit
2017-02-10 21:15:49 +01:00
Przemyslaw Skibinski
645f5b9856
fix for original MinGW
2017-02-10 20:09:28 +01:00
Sean Purcell
5069b6c2c3
Merge branch 'dev' into multiframe
2017-02-10 10:08:55 -08:00
ds77
45f0c207ab
use _stati64() in UTIL_getFileSize() when compiling with mingw, get rid of introduces previously preprocessor checks.
2017-02-10 18:37:57 +01:00
Przemyslaw Skibinski
456330948a
Merge remote-tracking branch 'refs/remotes/facebook/dev' into mingw
...
# Conflicts:
# programs/platform.h
2017-02-10 13:38:54 +01:00
Yann Collet
bbba42acd1
Merge pull request #537 from terrelln/small-bugs
...
Fix small bugs
2017-02-10 04:35:43 -08:00
-
7ec315df0d
fix previous commit
...
* struct _stat64 is not defined by (non-w64) MinGW releases, __stat64 should be everywhere
* proper detection of _stat64() availability (as in MinGW sys/stat.h)
2017-02-10 13:27:43 +01:00
-
19f61b534e
use _stat64 only when targetting Win2k or later
2017-02-10 10:56:45 +01:00
Przemyslaw Skibinski
429e13099a
fix 64-bit file support for MinGW
2017-02-10 10:36:44 +01:00
Nick Terrell
d08019813b
Improvement from @inikep
2017-02-09 14:20:52 -08:00
ds77
9cde3f8b2e
use _stat64() on MinGW
...
On MinGW, use _stat64() and struct _stat64 instead of stat() and struct stat_t. This fixes reporting incorrect sizes for large files.
2017-02-09 22:49:31 +01:00
Przemyslaw Skibinski
896638a8a2
echo replaced with printf
2017-02-09 17:01:17 +01:00
Przemyslaw Skibinski
13127fd05b
don't use "echo -e"
2017-02-09 11:32:21 +01:00
Nick Terrell
a0f9006e5a
#undef _POSIX_C_SOURCE if already defined
2017-02-08 17:25:01 -08:00
Sean Purcell
0f5c95af44
Disambiguate pledgedSrcSize == 0
...
- Modify ZSTD CLI to only set contentSizeFlag if it _knows_ the size
- Change pzstd to stop setting contentSizeFlag without accurate pledgedSrcSize
2017-02-08 15:12:46 -08:00
Sean Purcell
4e709712e1
Decompressed size functions now handle multiframes and distinguish cases
...
- Add ZSTD_findDecompressedSize
- Traverses multiple frames to find total output size
- Add ZSTD_getFrameContentSize
- Gets the decompressed size of a single frame by reading header
- Deprecate ZSTD_getDecompressedSize
2017-02-08 14:50:10 -08:00
Przemyslaw Skibinski
93901fe85c
remove redundant "ifeq ($(HAVE_ZLIB), 1)"
2017-02-08 21:11:18 +01:00
Przemyslaw Skibinski
4f9eaa7bb3
fixed gcc warnings
2017-02-08 18:08:09 +01:00
Przemyslaw Skibinski
cb56306a50
.gz suffix for gzip compressed files
2017-02-08 17:37:14 +01:00
Przemyslaw Skibinski
02018c83cf
added FIO_compressGzFrame
2017-02-08 16:54:23 +01:00
Przemyslaw Skibinski
cfd4dc299a
add "--format=gzip" option
2017-02-08 15:17:55 +01:00
Przemyslaw Skibinski
ca20edd960
fixed zlib detection with MinGW
2017-02-08 14:32:49 +01:00
Przemyslaw Skibinski
cdf5a7bd9f
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
2017-02-08 13:49:35 +01:00
Przemyslaw Skibinski
40580ff669
added description of "--priority=rt"
2017-02-08 13:49:06 +01:00
Przemyslaw Skibinski
0665a359aa
"--rt-prio" renamed to "--priority=rt"
2017-02-07 20:12:59 +01:00
Przemyslaw Skibinski
d05014c739
added the "--rt-prio" option
2017-02-07 16:48:01 +01:00
Przemyslaw Skibinski
94abd6a26c
SET_REALTIME_PRIORITY
2017-02-07 16:36:19 +01:00
Nick Terrell
7e3fc73795
Ensure <zlib.h> can be included in HAVE_ZLIB test
2017-02-06 11:54:31 -08:00
Nick Terrell
2cb8ee8784
Change zlib include to be a system include
2017-02-06 11:32:13 -08:00
Yann Collet
f6e9c4f1e1
Merge pull request #529 from facebook/gzstd_default
...
Gzstd default
2017-02-05 18:03:36 -08:00
cyan4973
613087c02b
Silence zlib detection routine
...
When it fails, $(CC) sends error message into stderr
redirected to /dev/null
2017-02-04 23:36:12 -08:00
Nick Terrell
b4016ff02f
Add cover dictionary training to zstd.1
...
Tested with `make install && man zstd` and visual inspection.
2017-02-03 16:42:07 -08:00
Yann Collet
762ddeeb9e
fixed zstdmt compilation under Windows minGW/MSYS2, by @inikep
2017-02-03 14:35:42 -08:00
Yann Collet
21eb80d485
remove zlib detection artefact
...
result of compilation test is sent to /dev/null
2017-02-03 14:34:41 -08:00
Yann Collet
b02ac8d613
fixed pointer conversion warnings (C++)
...
in gz module
2017-02-03 08:43:06 -08:00
Yann Collet
c2a4632789
release builds use less debug symbols and warnings
...
release build are triggered through either `make`,
or their specific target `make zstd-release` and `make lib-release`.
2017-02-02 20:54:41 -08:00
Yann Collet
c3cba9d858
fixed silent conversion warnings in GZDECOMPRESS path
2017-02-02 17:12:50 -08:00
Yann Collet
030ac243a0
Changed Makefile to generate zstd with .gz support by default
...
.gz support is detected by a runtime test.
2017-02-02 16:49:34 -08:00
Yann Collet
3dc85bae66
minor : fixed zstd-frugal
...
fixed a minor unused variable warning when compiling zstd-frugal target
2017-02-02 16:19:09 -08:00
Yann Collet
48bed91606
Merge pull request #527 from facebook/zstdmt
...
zstdmt refinements
2017-01-31 16:36:46 -08:00
Yann Collet
8d8513fb64
fixed C constant restrictions
2017-01-30 14:37:08 -08:00
Yann Collet
3672d06d06
zstdmt : section size is set to be a minimum of overlapSize
...
the minimum size condition size is applied transparently (no warning, no error)
like previous minimum section size condition (1 KB) which still applies.
2017-01-30 13:35:45 -08:00
Yann Collet
6ccd37c8d4
cli : added advanced parameter overlapLog
...
as a hidden (undocumented) parameter for now
2017-01-30 13:07:24 -08:00
Yann Collet
6be2337c26
added command --block-size=
...
for Multi-threading only.
alias : -B#
2017-01-30 11:17:26 -08:00
Yann Collet
cc3d1bc262
Merge pull request #525 from terrelln/covermt
...
Multithreaded COVER dictionary training
2017-01-30 10:15:33 -08:00
Yann Collet
5d9b894e46
Fixed status display for zstdmt
...
There is a large buffering effect when using zstdmt in MT mode.
Consequently, data is read first, pushed to workers,
and only later will the compressed result come out.
That means there is no longer immediate correlation
between amount of data read, and amount of data written.
This patch disables the displaying of % compression
when multi-threading is enabled.
It adds the displaying of total size when it can be determined
(it usually can be determined for files, but not for stdin)
so the user has a sense of "how far from the end" the compression compressed is.
There is no modification to decompression side,
since decompression is only single-threaded for now.
2017-01-27 13:30:18 -08:00
Nick Terrell
2fe9126591
Add multithread support to COVER
2017-01-27 11:56:02 -08:00
Nick Terrell
9c018cc140
Add BUCK files for Nuclide support
2017-01-27 10:43:12 -08:00
Przemyslaw Skibinski
eb2d23a90c
improved #232 fix
2017-01-27 10:43:03 -08:00
Przemyslaw Skibinski
92a4dbf2e4
Fixed https://github.com/facebook/zstd/issues/232
2017-01-27 10:43:03 -08:00
Nick Terrell
83c387eb8e
Fix zstdmt_compress.h include
2017-01-26 15:25:32 -08:00
Yann Collet
717c65d690
Merge pull request #519 from inikep/dev11
...
Dev11
2017-01-26 14:23:44 -08:00
Yann Collet
8dafb1acf5
CLI : automatically set overlap size to max (windowSize) for max compression level
2017-01-25 17:01:13 -08:00
Yann Collet
bb0027405a
fixed zstdmt corruption issue when enabling overlapped sections
...
see Asana board for detailed explanation on why and how to fix it
2017-01-25 16:25:38 -08:00
Yann Collet
943cff9c37
fixed zstdmt cli freeze issue with large nb of threads
...
fileio.c was continually pushing more content without giving a chance to flush compressed one.
It would block the job queue when input data was accumulated too fast (requiring to define many threads).
Fixed : fileio flushes whatever it can after each input attempt.
2017-01-25 12:35:19 -08:00
Przemyslaw Skibinski
5022a18d51
improved #232 fix
2017-01-25 13:11:26 +01:00
Przemyslaw Skibinski
64fa2dbc5e
Fixed https://github.com/facebook/zstd/issues/232
2017-01-25 13:02:33 +01:00
Yann Collet
512cbe8c10
zstdmt cli and API allow selection of section sizes
...
By default, section sizes are 4x window size.
This new setting allow manual selection of section sizes.
The larger they are, the (slightly) better the compression ratio,
but also the higher the memory allocation cost,
and eventually the lesser the nb of possible threads,
since each section is compressed by a single thread.
It also introduces a prototype to set generic parameters,
ZSTDMT_setMTCtxParameter()
The idea is that it's possible to add enums
to extend the list of parameters that can be set this way.
This is more long-term oriented than a fixed-size struct.
Consider it as a test.
2017-01-24 17:08:53 -08:00
Yann Collet
f8804d1014
convert tabs to space
...
joys of using multiple editors from multiple environments ...
2017-01-20 17:23:19 -08:00
Yann Collet
d7e3cb58c5
Resolved merge conflict dev+zstdmt
2017-01-20 16:44:50 -08:00
cyan4973
5fba09fa41
updated util's time for Windows compatibility
...
Correctly measures time on Posix systems when running with
Multi-threading
Todo : check Windows measurement under multi-threading
2017-01-20 12:57:31 -08:00
Yann Collet
458c8a94b4
minor refactoring : cleaner MT integration within bench
2017-01-19 17:44:15 -08:00
Yann Collet
500014af49
zstd cli can now compress using multi-threading
...
added : command -T#
added : ZSTD_resetCStream() (zstdmt_compress)
added : FIO_setNbThreads() (fileio)
2017-01-19 17:04:28 -08:00
Yann Collet
0f984d94c4
changed MT enabling macro to ZSTD_MULTITHREAD
2017-01-19 14:05:07 -08:00
Yann Collet
f22adae984
fixed minor warning (unused variable) in fuzzer
2017-01-19 13:46:30 -08:00
Yann Collet
736788f8e8
added streaming fuzzer tests for MT API
...
Also : fixed corner case, where nb of jobs completed becomes > jobQueueSize
which is possible when many flushes are issued
while there is not enough dst buffer to flush completed ones.
2017-01-19 12:15:29 -08:00
Przemyslaw Skibinski
8a0bc30a2d
Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11
2017-01-17 13:02:29 +01:00
Przemyslaw Skibinski
d72f4b6b7a
added "Makefile is validated"
2017-01-17 12:40:06 +01:00
Yann Collet
5eb749e734
ZSTDMT_compress() creates a single frame
...
The new strategy involves cutting frame at block level.
The result is a single frame, preserving ZSTD_getDecompressedSize()
As a consequence, bench can now make a full round-trip,
since the result is compatible with ZSTD_decompress().
This strategy will not make it possible to decode the frame with multiple threads
since the exact cut between independent blocks is not known.
MT decoding needs further discussions.
2017-01-11 18:21:25 +01:00
Nick Terrell
c220d4c74d
Use COVER_MEMMULT when training with COVER.
2017-01-09 16:49:04 -08:00
Yann Collet
56958500fc
minor man page update
2017-01-09 19:47:09 +01:00
Nick Terrell
3a1fefcf00
Simplify COVER parameters
2017-01-02 17:51:38 -08:00
Nick Terrell
df8415c502
Add COVER to the zstd cli
2017-01-02 14:43:08 -08:00
Yann Collet
6334b04d61
compile object files, for faster recompilation
2017-01-02 03:22:18 +01:00
Yann Collet
747452677d
fixed cmake tests
2017-01-02 02:05:45 +01:00
Yann Collet
cdb2763f4a
new Makefile target zstdmt
2017-01-02 01:43:56 +01:00
Yann Collet
f1cb55192c
fixed linux warnings
2017-01-02 01:11:55 +01:00
Yann Collet
0ec6a95ba1
minor fixes
2017-01-02 00:49:42 +01:00
Yann Collet
c6a6417458
bench correctly measures time for multi-threaded compression (posix only)
2016-12-31 03:31:26 +01:00
Yann Collet
f765a375a5
Merge pull request #504 from terrelln/thread-pool
...
[zstdmt] Add thread pool
2016-12-30 15:31:49 +01:00
Yann Collet
e70912c72b
Changed : input divided into roughly equal parts.
...
Debug : can measure time waiting for mutexes to unlock.
2016-12-29 01:24:01 +01:00
Yann Collet
8d7432914f
Merge pull request #503 from inikep/dev11
...
Dev11
2016-12-28 16:50:39 +01:00
Yann Collet
ab7a579180
added -T command , to set nb of threads
2016-12-28 16:11:09 +01:00
Przemyslaw Skibinski
75f3a3a335
changed default PREFIX and MANDIR
2016-12-28 12:32:41 +01:00
Yann Collet
3d93f2fce7
first zstdmt sketch
2016-12-27 07:19:36 +01:00
Yann Collet
c07d2e3a31
Merge pull request #499 from inikep/dev11
...
improved *BSD and Solaris compatibility
2016-12-23 21:32:03 +01:00
Przemyslaw Skibinski
63b0014b96
BSD: improved "make install"
2016-12-23 10:05:49 +01:00
Andrew Janke
fce374a100
zstdless: add shebang and quote $@
2016-12-22 17:40:10 -05:00
Przemyslaw Skibinski
b999170311
Solaris: working "make -C lib install"
2016-12-22 20:14:37 +01:00
Przemyslaw Skibinski
aab442133d
Solaris: working "make -C programs install"
2016-12-22 19:26:01 +01:00
Yann Collet
8333106b8a
Merge branch 'dev' of github.com:facebook/zstd into dev
2016-12-21 16:44:24 +01:00
Yann Collet
0d7e84899f
Merge pull request #489 from inikep/v112
...
improved detection of POSIX
2016-12-21 16:42:46 +01:00
Yann Collet
9518744025
Merge branch 'dev' of github.com:facebook/zstd into dev
2016-12-21 16:20:41 +01:00
Yann Collet
1f57c2ed32
added : ZSTD_createCDict_byReference()
2016-12-21 16:20:11 +01:00
Przemyslaw Skibinski
7a8a03c20d
util.h: restore BSD license for Facebook Open-Source
2016-12-21 15:08:44 +01:00
Przemyslaw Skibinski
97a258d71d
updated comments
2016-12-21 14:00:41 +01:00
Przemyslaw Skibinski
e679741b18
_CRT_SECURE_NO_WARNINGS moved to util.h
2016-12-21 13:47:11 +01:00
Przemyslaw Skibinski
2f6ccee6af
platform.h: removed Compiler Options
2016-12-21 13:23:34 +01:00
Przemyslaw Skibinski
101df4f636
fixed Visual Studio compilation
2016-12-21 11:43:11 +01:00
Przemyslaw Skibinski
a35b9448ac
improved MinGW support
2016-12-21 11:18:45 +01:00
Przemyslaw Skibinski
5736db219e
fix basic types redefinition
2016-12-21 09:26:00 +01:00
Przemyslaw Skibinski
20b089e53d
simplified zstdcli.c
2016-12-21 09:19:15 +01:00
Przemyslaw Skibinski
16ae6563a2
executables use new util.h and platform.h
2016-12-21 09:06:14 +01:00
Przemyslaw Skibinski
ead350bdc0
improved util.h and platform.h
2016-12-21 09:04:59 +01:00
Przemyslaw Skibinski
1c1db6b845
windres updated to v1.1.3
2016-12-20 11:21:26 +01:00