Commit Graph

1229 Commits (59bb5f7d5878cf88b88c555dc5c99524f0f48f97)

Author SHA1 Message Date
Yann Collet 93702a7a62
Merge pull request #1198 from facebook/msdebug
made Visual Studio compatible with DEBUGLEVEL >= 2
2018-06-20 12:26:31 -07:00
cyan4973 ae0b7ffa0a made Visual Studio compatible with DEBUGLEVEL >= 2 2018-06-20 09:45:02 -07:00
Yann Collet 6768cf53fd
Merge pull request #1190 from terrelln/ldm-adjust
Adjust advanced parameters to source size
2018-06-19 14:40:56 -07:00
Yann Collet c0b6ce95b1
Merge pull request #1179 from supertopher/dev
Improves UX for --list command's lack of support for pipes
2018-06-19 14:36:30 -07:00
Nick Terrell 1d0fcde45d Use debug.h in fileio.c 2018-06-18 15:51:21 -07:00
Nick Terrell 3841dbac84 Adjust advanced parameters to source size
In the new advanced API, adjust the parameters even if they are explicitly
set. This mainly applies to the `windowLog`, and accordingly the `hashLog`
and `chainLog`, when the source size is known.
2018-06-18 15:49:31 -07:00
Topher Lubaway 6bca3fb4bf Reduce noise in diff
putting the code block back on the exact line it came from
2018-06-13 14:32:59 -07:00
Topher Lubaway ec24f98cca Removes duplicate IS_CONSOLE from PR
I misunderstood that this function was included already
2018-06-13 13:39:23 -07:00
Yann Collet c986dbf241
Merge pull request #1168 from GeorgeLu97/paramgrillfeatures
Have paramgrill share bench.c benchmarking function
2018-06-13 11:38:29 -04:00
George Lu 01d940b670 Requested changes
-Remove g_displaylevel/setNotificationLevel function
-Add extern "C"
-Remove averaging
-Reorder arguments

More fixes

-Added BMK_return_t (result + possible error)
-Correct comment'
-Nullcheck ctx, dctx when allocated
-Remove extra assert
2018-06-12 17:02:44 -04:00
Topher Lubaway b024e1e1f4 Keep windows specific headers
Accidentially deleted this existing windows only header
2018-06-12 10:16:27 -07:00
Topher Lubaway 88ae51acb3 Multi-OS support for --list detecting stream input
IS_CONSOLE stolen wholesale from Options.cpp
not sure if i should have extracted that code for DRY-ness
tested in OSX and functionality seems appropriate
unstested in a windows environment
2018-06-12 07:59:17 -07:00
Topher Lubaway 881defaeb3 Only check for tty in non-windows environments
unistd.h is for unix standard tools.
There does not appear to be a simple isatty for windows
this we only run the logic and header include in
non-windows environments
2018-06-11 15:26:35 -07:00
Topher Lubaway 5ca1d5c6f4 Properly brackets isatty if statement
¯\_(ツ)_/¯ this is my first commit in c
2018-06-11 12:19:15 -07:00
Topher Lubaway 4c16608e3c Improves UX for --list command's lack of support for pipes
--list does not support piped input
This checks for a terminal and exits 1 with a well formatted
error message if the STDIN is not from a terminal
2018-06-11 10:13:00 -07:00
Ryan Schmidt b567ce9d68 Fix name of macOS 2018-06-09 14:31:17 -05:00
George Lu 0e808d608b Make paramgrill use bench.c benchmarking 2018-06-08 12:01:05 -07:00
Yann Collet d3615c28db
Merge pull request #1159 from GeorgeLu97/suffixlist
Unknown Suffix Error
2018-06-01 14:00:10 -07:00
George Lu 8984cc93d6 update display 2018-05-31 18:04:05 -07:00
George Lu 547096d672 update man 2018-05-31 18:03:52 -07:00
George Lu c9b1068298 removed strcats 2018-05-31 17:47:29 -07:00
George Lu 5ff30fe2e5 Unknown Suffix Error
Changed so only compiled formats are printed in list of supported extensions
2018-05-31 16:13:36 -07:00
George Lu 140f59d38e Added --format=zstd
title
2018-05-31 15:29:35 -07:00
Yann Collet 174bd3d4a7
Merge pull request #1131 from facebook/zstdcli
minor: control numeric argument overflow
2018-05-14 11:53:58 -07:00
Yann Collet 9cd5c63771 cli: control numeric argument overflow
exit on overflow
backported from paramgrill
added associated test case
2018-05-12 14:29:33 -07:00
Yann Collet b824d213cb fix #1115 2018-05-12 10:21:30 -07:00
cyan4973 62487b5e76 fixed decoding bogus lz4 frame
FIO would keep presenting data after an LZ4F decoding error
resulting in a NULL pointer dereference
when associated with older liblz4 version (< v1.8.1.2)
2018-04-23 18:50:16 -07:00
Yann Collet 1da629f2ad
Merge pull request #1104 from terrelln/fast-train
Allow negative compression levels in training
2018-04-09 14:16:20 -07:00
Nick Terrell 569e2abccd Allow negative compression levels in training
* Set `dictCLevel` in `zstdcli.c`.
* Only set to default level if the compression level `== 0`, not `<= 0`.
2018-04-09 12:12:03 -07:00
Björn Ketelaars e5ea8d272a fix typo in programs/zstd.{1,1.md}
s/nodictID/no-dictID/g
2018-04-05 06:44:46 +02:00
Yann Collet 7188862d32
Merge pull request #1086 from hagemt/hagemt-patch-1
Correct small typo in manual (man file and markdown)
2018-03-30 20:45:10 -06:00
Tor E Hagemann c7a5e60bc6
Update zstd.1.md 2018-03-30 15:25:32 -07:00
Tor E Hagemann 292d370ab4
Update zstd.1 2018-03-30 14:53:57 -07:00
Yann Collet 525f3fab33 restored ability to manually set overlapLog 2018-03-28 11:33:41 -06:00
Yann Collet 01082a39bd restored simple status line during zstd compression
the more advanced one, featuring amount of data buffered,
is triggered on `-v`.
2018-03-22 17:49:46 -07:00
Yann Collet 153bc1c004 removed limit ZSTD_TARGETLENGTH_MAX
this makes it possible to specify extremely large negative compression levels,
achieving the side effect as "no compression".

It will also be possible to define larger targetlength for ultra compression mode.

There is no adverse side effect due to removing this limit.
2018-03-21 15:50:05 -07:00
Yann Collet 353117c5d7 implemented ZSTD_DCtx_loadDictionary*()
this required updating ZSTD_createDDict_advanced()
to accept a dictContentType parameter (raw, full, auto).
2018-03-20 13:40:29 -07:00
Yann Collet 4c5cbac179
Merge pull request #1041 from facebook/fasterFast
Negative compression levels
2018-03-13 21:32:46 -07:00
Yann Collet bd7bb94361
Merge pull request #1044 from baldurk/remove-utf8-characters
Remove non-ASCII characters in header file comments
2018-03-13 13:22:07 -07:00
Baldur Karlsson 430a2fec19 Remove non-ASCII characters in header file comments
* Replaced a non-breaking space and an en dash with a plain space and
  a hyphen.
* This means the files are simple ASCII and less likely to run into
  codepage issues.
2018-03-13 20:05:53 +00:00
Jesse Talavera-Greenberg 2f70fbf2a3
Made -H's printout specify the semantics of -T0 2018-03-12 20:43:32 -04:00
Yann Collet a57d43d4d4 updated documentation of targetLength 2018-03-12 11:35:01 -07:00
Yann Collet f24566b597 minor bench improvements
- do not test level 0, as it is converted into level 3,
  which feels strange when compressing multiple levels
- Use direct synchronous mode when a single worker is requested.
2018-03-12 04:02:57 -07:00
Yann Collet 6a9b41b731 create command --fast[=#]
access negative compression levels from command line
for both compression and benchmark modes.

also : ensure proper propagation of parameters
through ZSTD_compress_generic() interface.

added relevant cli tests.
2018-03-11 20:01:23 -07:00
Yann Collet a70f7e10fa Merge branch 'benchDecode' into longOffsetMode 2018-03-05 14:09:00 -08:00
Yann Collet 03e7e14192 fix benchmark issue when measuring only decoding speed
zstd bench module can focus on decompression speed _only_.
This is useful when trying to measure performance
on large input data compressed using a high level
as compression time becomes problematic (too long).

This mode is triggered by command : zstd -b -d

Problem was : in such a mode,
measured decoding speed was > 10% slower
than in nominal mode (compression + decompression),
making decompression benchmark mode much less useful.

This patch fixes the issue.
It's not completely clear why, but
moving the `memcpy()` operation sooner in the pipeline fixed it.

I can still measure some difference, but it is in the < 2% range,
so it's much more tolerable.

also : it doesn't matter anymore in which order are selected
commands `-b` and `-d`.
The combination always triggers bench_decodeOnly mode.
2018-03-05 13:57:41 -08:00
Yann Collet 41bd10446e Merge branch 'dev' into longOffsetMode 2018-03-05 13:10:10 -08:00
Yann Collet b91ddf0ae6 Merge branch 'dev' into longOffsetMode 2018-03-05 11:59:54 -08:00
Conrad Meyer 606374269c FIO_addFInfo: Fully initialize output 'total' struct
Silence a Coverity warning about 'windowSize' being uninitialized.
(Yes, nothing that calls this routine actually uses the windowSize
value.  Still, appeasing Coverity is pretty harmless in this case.)
2018-02-28 15:23:05 -08:00
Yann Collet 25d00d10fc fixed minor conversion warning 2018-02-20 16:52:28 -08:00