0558850735
bench stops immediately on decoding error
2017-09-01 11:46:15 -07:00
17d8e0bdcc
Merge remote-tracking branch 'upstream/longRangeMatcher' into ldm-integrate
2017-09-01 10:19:38 -07:00
8081becadc
Add long distance matching as a CCtxParam
2017-09-01 09:18:58 -07:00
d7ad99b2ab
Merge branch 'longRangeMatcher' into dev
2017-08-31 18:08:37 -07:00
c7818fc676
Merge branch 'modTests' into dev
...
fixed conflict
2017-08-31 17:00:16 -07:00
4299c27132
improved console log of utils.h
...
removed a warning when compiling on Windows
2017-08-31 16:58:47 -07:00
8e298382a8
changed target allarch into allzstd
...
allzstd contains only zstd-related tests.
allmost = allzstd + zwrapper tests (which require zlib)
2017-08-31 14:30:52 -07:00
6a546efb8c
Add long distance matcher
...
Move last literals section to ZSTD_block_internal
2017-08-31 12:53:19 -07:00
b0cb081dc8
last batch of header files changed to reflect new license ( #825 )
...
only remains to update contrib/linux-kernel (@terrelln)
2017-08-31 12:20:50 -07:00
c88fb9267f
Replace 'byReference' with enum
2017-08-29 11:55:02 -07:00
cf689b84f9
Sort input file list
...
in order to make builds reproducible
in spite of indeterministic filesystem readdir order.
See https://reproducible-builds.org/ for why this is good.
2017-08-26 17:08:00 +02:00
6f1a21c7e9
Remove formatting-only changes
2017-08-23 10:24:19 -07:00
232d62b637
fixed a few headers that were too hastily copy/pasted during last license change
2017-08-21 11:24:32 -07:00
91b30dbe84
Remove test parameter
2017-08-21 10:09:06 -07:00
f181f33bdf
Disable tests and refactor
2017-08-21 01:59:08 -07:00
023b24e6d4
Add cctx param tests
2017-08-20 22:55:07 -07:00
1c108c811e
cli : Display supported formats on -vV command
...
Requested and inspired by patch from @ib (#771 )
2017-08-19 13:33:50 -07:00
2ecd34ee5e
fixed unused variables warnings
2017-08-19 01:23:49 -07:00
23706fb743
updated doc on compilation variables
2017-08-19 01:14:36 -07:00
9203003d5f
fixed zstd-nolegacy
...
and added it to allVariants for CI testings
2017-08-19 01:01:53 -07:00
4b387729b6
fixed zstd-small
...
and added it to shortest for CI tests
2017-08-19 00:48:29 -07:00
7729ab83bb
Merge branch 'dev' into variants
2017-08-19 00:37:06 -07:00
32e943b3ef
Merge branch 'dev' of github.com:facebook/zstd into dev
2017-08-19 00:36:37 -07:00
582e19b01c
Merge pull request #801 from facebook/newLicense
...
New license
2017-08-19 00:34:34 -07:00
288e263ec4
added zstd-decompress to the list of variants tested
2017-08-18 18:39:39 -07:00
166645e7b3
fixed zstd-compress
...
file-information is dependent on decompression functions.
it should only be enabled when ZSTD_NODECOMPRESS is not set.
also : added zstd-compress compilation test into `make shortest`
2017-08-18 18:30:41 -07:00
f207b39f55
blindfix for Windows conversion warning
...
long type is 32-bits on Windows 64,
while it's 64-bits on Unix.
64-to-32 shortening conversion for long is a specific Windows issue.
2017-08-18 17:06:12 -07:00
32fb407c9d
updated a bunch of headers
...
for the new license
2017-08-18 16:52:05 -07:00
88d2f72df9
fixed --list command in presence of special blocks
...
block type RLE is special, compressed size is always 1.
block type 3 is "reserved", aka not supported.
2017-08-18 16:18:20 -07:00
c523c93b26
improved and fixed --list command, original patches by @ib ( #772 )
...
accepts all skippable frame identifiers.
display in MB or KB, depending on frame size.
fixed combination of skippable and zstd frames.
2017-08-18 15:57:53 -07:00
3f54d788e9
removed --list from cli help (-h), reported by Agostino Sarubbo (@asarubbo) ( #800 )
...
redundant with shorter -l.
both -l and --list do the same thing, and are documented in man page.
2017-08-18 15:15:31 -07:00
0fb7b23fcc
fix typo in lz4 support code
2017-08-14 14:04:24 +02:00
d0b27483ae
[zstdcli] Fix -t in streaming mode
2017-07-18 14:45:49 -07:00
3a60efd3a9
policy change : ZSTDMT automatically caps nbThreads to ZSTDMT_NBTHREADS_MAX ( #760 )
...
Previously, ZSTDMT would refuse to create the compressor.
Also : increased ZSTDMT_NBTHREADS_MAX to 256,
updated doc,
and added relevant test
2017-07-13 10:17:23 -07:00
3345a91964
cli : use new advanced API by default
2017-07-05 17:34:15 -07:00
b8280fe929
minor code refactoring
2017-07-03 15:14:55 -07:00
368b974387
fixed gz error reporting
2017-07-03 13:47:46 -07:00
c9f21c8a71
gzip : only display inflateEnd error when there is no error before
2017-07-03 13:45:09 -07:00
6c35112d08
gzip decoding does no longer exit() on invalid input
2017-07-03 13:24:50 -07:00
e97ff3b54a
decoding malformed lz4 frames does no longer exit()
2017-07-03 11:27:29 -07:00
c34185df13
fileio : decoding malformed lzma frame does no longer exit()
...
makes it possible to continue decoding file list
2017-07-03 10:27:16 -07:00
1d39550471
Merge pull request #744 from terrelln/dev
...
[pzstd] Remove appveyor tests
2017-07-01 05:58:33 -07:00
56e3964d85
[man] Specify that strategies start at 1
2017-06-30 16:29:37 -07:00
811deaea6f
Merge pull request #736 from terrelln/cover-default-api
...
[zdict] Make COVER the default algorithm
2017-06-28 20:25:36 -07:00
5b7fd7c422
[zdict] Make COVER the default algorithm
2017-06-26 21:09:22 -07:00
d88a441406
[zstdcli] Document HAVE_LZMA and zstd-noxz
2017-06-26 11:25:00 -07:00
6aeb50ea0d
[zstdcli] Add unlzma and unxz symlinks
2017-06-26 11:24:36 -07:00
849ecf3510
[zstdcli] Support xz by default when liblzma is available
2017-06-23 17:11:38 -07:00
ef269c1b68
Merge pull request #725 from facebook/advancedAPI2
...
New Advanced API
2017-06-23 09:50:47 -07:00
991f9dfcde
switched fileio.c to ZSTD_DEBUG trigger macro
2017-06-21 15:16:13 -07:00
5be0f5544e
added newline to align output
2017-06-21 12:41:10 -07:00
6f5fe71041
added error check for when file could not be opened
2017-06-21 12:37:23 -07:00
74a725da69
reversed calculation of ratio
2017-06-21 12:27:20 -07:00
7bd1a2900e
added ZSTD_dictMode_e to control dictionary loading mode
2017-06-21 11:50:33 -07:00
a73c2a444a
added tests with null files, changed condition to check that the file is more than 0 bytes
2017-06-20 14:33:08 -07:00
b44ab82f7a
ensure new ZSTD_strategy starts at value 1
2017-06-20 14:11:49 -07:00
0765602903
fixed error where extremely small files were not being detected as not compressed with zstd
2017-06-20 14:04:46 -07:00
3701dde3ba
changed to use LONG_SEEK
2017-06-20 13:44:05 -07:00
93a5510962
added progress metric to display
2017-06-20 13:26:25 -07:00
bda57754a2
moved signifcant operations to fileio.c
2017-06-20 12:43:10 -07:00
c594507871
removed double if statements for same condition
2017-06-20 11:59:45 -07:00
72e0ac9929
minor update to logic
2017-06-20 11:58:27 -07:00
cea55a9217
size_t const value
2017-06-20 11:56:11 -07:00
bed0dc95bc
changed displayInfo alignment and const
2017-06-20 11:47:24 -07:00
cb2dbe6434
changed fileInfo initialization, updated error code
2017-06-20 09:35:21 -07:00
c08e649e95
first implementation of bench.c with new API ZSTD_compress_generic()
...
Doesn't speed optimize this buffer-to-buffer scenario yet.
Still internally defers to streaming implementation.
Also : fixed a long standing bug in ZSTDMT streaming API.
2017-06-19 18:25:35 -07:00
fe234bf48b
fix attempts : fullbench for VS2008
2017-06-19 15:23:19 -07:00
55b9b28dad
made style changes, added description in man file
2017-06-19 09:55:55 -07:00
4a0e4c856c
fixing more spacing on loops
2017-06-15 18:08:22 -07:00
b3e62446c6
added in correct error return from main in zstdcli.c
2017-06-15 17:46:49 -07:00
0d7c4d766a
initialized info manually to avoid error
2017-06-15 17:30:18 -07:00
d3b34e4a25
changed spacing on if statements
2017-06-15 17:12:41 -07:00
b316691f91
added const
2017-06-15 17:03:59 -07:00
60a2e55e2e
added case for when file is not compressed with zstd (incorrect magic number)
2017-06-15 17:00:59 -07:00
acaefb531b
refactoring brackets to allow for const values
2017-06-15 16:27:38 -07:00
0757eae6ff
rearranging code to allow for consts
2017-06-15 16:12:04 -07:00
e49afae2ab
changed implicit precision loss again
2017-06-15 16:03:40 -07:00
54e0b6c66b
got rid of imlpicit precision loss
2017-06-15 16:01:52 -07:00
b766211e10
made another function static
2017-06-15 15:55:49 -07:00
e7f02fc58a
closed file at end of function, created another variable to keep exit points simpler
2017-06-15 15:53:02 -07:00
8b3ff7c9bc
changed info to static allocation
2017-06-15 15:35:37 -07:00
96cabf7233
cleaned up if statements
2017-06-15 15:25:20 -07:00
9437cc7498
added --list command
2017-06-15 15:07:54 -07:00
4495e9a826
allow analysis even if file does not have zst extension
2017-06-15 15:02:54 -07:00
a9b77c83e5
cleaning up code for analyzing frames
2017-06-15 14:13:28 -07:00
e208992529
cleaning up code
2017-06-15 12:27:32 -07:00
fc428ab350
changing formatting again
2017-06-14 13:26:19 -07:00
607ee9f4bc
print message when no files are given, rearrange code for readability
2017-06-14 13:23:36 -07:00
ebd60349f5
changed U64 to unsigned long to get rid of build test errors
2017-06-14 13:11:18 -07:00
81fa33b55a
cleaning up code
2017-06-13 10:06:49 -07:00
a56dcbfcf8
changed print format for non-verbose version
2017-06-12 17:47:33 -07:00
c828b52111
yet more refactoring
2017-06-12 17:19:53 -07:00
28dbf8ee6c
more refactoring for C90
2017-06-12 17:09:08 -07:00
618a7b66e2
refactoring for C90 syntax
2017-06-12 16:53:50 -07:00
6e33c74133
formatting issues resolved
2017-06-12 16:29:00 -07:00
173a739717
code refactorings with explicit conversion
2017-06-12 16:14:04 -07:00
f3d6ab28a2
formated information for verbose output
2017-06-12 15:59:28 -07:00
6e0204470a
displayed decompressed size
2017-06-12 15:51:59 -07:00
dcd6ba6dc6
incremented decompressedSize instead of setting value
2017-06-12 15:40:47 -07:00
6996bd2598
removed useless lines
2017-06-12 15:24:50 -07:00
9cb602ee25
added in logic for parsing through blocks/frames
2017-06-12 15:22:48 -07:00
786b7cac27
added code to analyze the first frame header
2017-06-12 13:46:39 -07:00
a3d54cf73d
added line spacing for clarity
2017-06-12 10:58:34 -07:00
bb0aaf9579
minor man update on -B# option in benchmark mode
2017-06-12 05:19:15 -07:00
9e6a2eaab6
added MT support to NEWAPI
2017-06-11 18:39:46 -07:00
0f06f4f266
added display for compressed size
2017-06-06 09:21:42 -07:00
4128f67a4a
added in check suffix check to ensure file was compressed with zstd
2017-06-05 15:00:06 -07:00
901435e9ef
setup basic functions for adding --list functionality
2017-06-05 14:45:31 -07:00
8bcbf42617
fixed g++ prototype mismatch
2017-06-04 23:52:00 -07:00
6d4fef36de
Added ZSTD_compress_generic()
...
Used in fileio.c (zstd cli).
Need to set macro ZSTD_NEWAPI to trigger it.
2017-05-17 18:36:15 -07:00
23c256e44b
removed useless variable from CCtx
...
CStream's pledgedSrcSize is no longer necessary
srcSize control is realized within bufferless interface.
2017-05-16 18:10:11 -07:00
e12ae02357
minor coding style update
...
reduced some long lines
2017-05-16 17:32:33 -07:00
2d4d31c18a
removed gcc compilation flag -Wbad-function-cast
...
It makes it more difficult to directly cast the result of a function,
requiring to store the result in an intermediate variable.
It does not necessarily help readability,
and this restriction can be difficult to overcome in some constructions,
like some macros.
also : fixed minor Visual conversion warnings in datagencli.c
2017-05-16 11:34:38 -07:00
46ac9ade9b
fixed vla warning on linux
2017-05-15 18:15:08 -07:00
83d0c764dc
added several compilation flags
2017-05-15 17:15:46 -07:00
a99ece9e79
Update zstd.1.md to use btultra
2017-05-10 16:51:18 -07:00
5f2c7213c7
Merge remote-tracking branch 'upstream/dev' into btopt
...
* upstream/dev: (305 commits)
added test for ZSTD_estimateCStreamSize()
changed variable name, for clarity
fixed ZSTD_estimateCStreamSize()
shortened ZSTD_createCStream_Advanced()
fixed symbols test
added ZSTD_estimateDStreamSize()
changed name frameParams into frameHeader
regroup memory usage function declarations
separated ZSTD_estimateCStreamSize() from ZSTD_estimateCCtxSize()
bumped version number
added ZSTD_estimateCDictSize() and ZSTD_estimateDDictSize()
Updated ZSTD_freeCCtx()
updated ZSTD_estimateCCtxSize()
Updated ZSTD_sizeof_CCtx()
merged CCtx and CStream as a single same object
cli : -d and -t do not stop after a failed decompression
added dev branch CircleCI badge
added dev branch Appveyor badge
keep dev branch status only
creates a binary archive without the `programs` directory
...
2017-05-10 16:49:58 -07:00
01a1abfdb5
cli : -d and -t do not stop after a failed decompression
...
The problematic srcfile will be named on console/log,
but decompression/test will continue onto next file in the list.
2017-05-05 19:15:24 -07:00
a00e9599f1
removed -g from DEBUGFLAGS
...
It inflates binary sizes, which is negative for the Windows build.
It also makes it impossible to check if 2 different source codes
get nonetheless compiled to the same binary,
since checksum will be different, due to integrated source code.
2017-05-04 17:24:29 -07:00
c08e56861e
updated dict graphs to 2D mode
2017-05-03 14:33:28 -07:00
710497d8ea
updated programs/README.md, to introduce compilation variables
...
make it possible to enable/disable features individually
2017-05-02 17:18:24 -07:00
f47284fec0
reorganized Makefile for multiple targets
2017-05-02 16:55:57 -07:00
3791d2105d
added xzstd4 target
...
support for all formats, xz/lzma/lz4 included
2017-05-02 16:38:37 -07:00
2e63a877f3
fixed xzstd
...
--format=xz was missing a break, making the execution continue into lz4 error message
2017-05-02 15:40:42 -07:00
f376d47c11
[CLI] Switch dictionary builder on CLI to cover
2017-05-02 11:18:27 -07:00
020b960e13
[cover] Make optimization faster
2017-05-02 11:02:48 -07:00
f2d9ef1dc0
[cover] Optimize case where d <= 8
2017-05-02 11:02:43 -07:00
feba969a69
Fix LZ4_MSG in xzstd
2017-05-02 11:02:16 -07:00
31533bacce
Changed ZSTD_createCDict_advanced()
...
It now only uses compressionParameters as argument.
It produces many changes throughout user code,
though hopefully they tend to be simple :
just provide the cParams part from existing ZSTD_parameters.
Some programs might depend on ZSTD_createCDict_advanced() to pass frame parameters.
This change will force them to revisit this strategy and fix it,
since frame parameters are effectively silently ignored in current version.
2017-04-27 00:29:04 -07:00
eab41c1872
Fix LZ4 wrapper deprecation warnings
2017-04-26 10:17:38 -07:00
2c4b6fe6b3
Make lz4 compression/decompression compatible with library r123
2017-04-25 11:00:54 -07:00
4de8632957
Add LZ4 compress/decompress support to CLI
2017-04-24 16:48:25 -07:00
230d7acc7d
cli : add support for --threads=# command
...
updated documentation
add relevant test case
2017-04-21 11:38:13 -07:00
9606256a8d
Fix no thread message
2017-04-18 13:52:00 -07:00
1c286950af
Merge pull request #663 from iburinoc/zstdmt-default
...
Compile CLI using files instead of objs
2017-04-18 02:59:54 -07:00
c8b2df7d62
Compile CLI using files instead of objs
...
This avoids conflicts between how the library was
configured and how the CLI was configured.
2017-04-17 17:13:47 -07:00
e6c504dbe6
Update -T0 comment in man page
2017-04-17 17:12:02 -07:00
5935c990a0
Add zstdmt and -T0 to man page
2017-04-17 16:05:20 -07:00
f6ef4db20e
Install zstdmt even without threading support
2017-04-17 12:21:11 -07:00
5a61f36474
Make zstd compile with mt by default
2017-04-17 12:06:02 -07:00
7dd14d03b0
Enable multithreading on BSD
2017-04-15 16:25:08 +02:00
e4f3235c85
Add 0 initializers to static variables
2017-04-13 16:34:28 -07:00
42bac7fa84
Change ifndef's to undef's
2017-04-13 15:35:05 -07:00
9227aae001
Fix clang linux compilation
2017-04-13 14:06:40 -07:00
3b6207d4bd
Fix compilation on windows
2017-04-13 14:04:10 -07:00
f876f1200c
Fix compilation on macOS
2017-04-13 12:33:45 -07:00
afa48518e2
-T0 detects number of physical cores
2017-04-13 12:28:28 -07:00
5c42d0edc8
cli : better status display for zstdmt in 1-thread mode
2017-04-11 16:57:32 -07:00
0e30059ba1
cli : FIO_createDictBuffer() replaces FIO_loadFile()
...
makes it more explicit that it allocates a buffer
and that it's meant to be used for dictionary.
Also : simplified function a bit,
now only works for dictionaries up to DICTSIZE_MAX
2017-04-11 14:41:02 -07:00
279be2015b
Let zstd overwrite read-only files
2017-04-06 12:56:40 -07:00
Yann Collet
Stella Lau
Yann Collet
Bernhard M. Wiedemann
codicodi
Nick Terrell
Paul Cruz
cyan4973
Sean Purcell
Baptiste Daroussin