created programs\README.md
This commit is contained in:
inikep
parent
7f18c6e5ef
commit
ab2f7708c1
@ -24,9 +24,9 @@
|
||||
# zstd : Command Line Utility, supporting gzip-like arguments
|
||||
# zstd32 : Same as zstd, but forced to compile in 32-bits mode
|
||||
# zstd_nolegacy : zstd without support of decompression of legacy versions
|
||||
# zstd-small: minimal zstd without dictBuilder and bench
|
||||
# zstd-compress: compressor-only version of zstd
|
||||
# zstd-decompress: decompressor-only version of zstd
|
||||
# zstd-small : minimal zstd without dictionary builder and benchmark
|
||||
# zstd-compress : compressor-only version of zstd
|
||||
# zstd-decompress : decompressor-only version of zstd
|
||||
# ##########################################################################
|
||||
|
||||
DESTDIR?=
|
||||
|
||||
93
programs/README.md
Normal file
93
programs/README.md
Normal file
@ -0,0 +1,93 @@
|
||||
zstd - Command Line Interface
|
||||
================================
|
||||
|
||||
Command Line Interface (CLI) can be created using the `make` command without any additional parameters.
|
||||
There are however other Makefile targets that create different variations of CLI:
|
||||
- `zstd` : default CLI supporting gzip-like arguments; includes dictionary builder, benchmark, and support for decompression of legacy zstd versions
|
||||
- `zstd32` : Same as `zstd`, but forced to compile in 32-bits mode
|
||||
- `zstd_nolegacy` : Same as `zstd` except of support for decompression of legacy zstd versions
|
||||
- `zstd-small` : CLI optimized for minimal size; without dictionary builder, benchmark, and support for decompression of legacy zstd versions
|
||||
- `zstd-compress` : compressor-only version of CLI; without dictionary builder, benchmark, and support for decompression of legacy zstd versions
|
||||
- `zstd-decompress` : decompressor-only version of CLI; without dictionary builder, benchmark, and support for decompression of legacy zstd versions
|
||||
|
||||
|
||||
#### Aggregation of parameters
|
||||
CLI supports aggregation of parameters i.e. `-b1`, `-e18`, and `-i1` can be joined into `-b1e18i1`.
|
||||
|
||||
|
||||
#### Dictionary builder in Command Line Interface
|
||||
Zstd offers a training mode, which can be used to tune the algorithm for a selected
|
||||
type of data, by providing it with a few samples. The result of the training is stored
|
||||
in a file selected with the `-o` option (default name is `dictionary`),
|
||||
which can be loaded before compression and decompression.
|
||||
|
||||
Using a dictionary, the compression ratio achievable on small data improves dramatically.
|
||||
These compression gains are achieved while simultaneously providing faster compression and decompression speeds.
|
||||
Dictionary work if there is some correlation in a family of small data (there is no universal dictionary).
|
||||
Hence, deploying one dictionary per type of data will provide the greater benefits.
|
||||
Dictionary gains are mostly effective in the first few KB. Then, the compression algorithm
|
||||
will rely more and more on previously decoded content to compress the rest of the file.
|
||||
|
||||
Usage of the dictionary builder and created dictionaries with CLI:
|
||||
1) Create the dictionary : `zstd --train FullPathToTrainingSet/* -o dictionaryName`
|
||||
2) Compress with dictionary: `zstd FILE -D dictionaryName`
|
||||
3) Decompress with dictionary: `zstd --decompress FILE.zst -D dictionaryName`
|
||||
|
||||
|
||||
|
||||
#### Benchmark in Command Line Interface
|
||||
CLI includes in-memory compression benchmark module for zstd.
|
||||
The benchmark is conducted using given filenames which are read into memory and joined together.
|
||||
It makes benchmark more precise as it eliminates I/O overhead.
|
||||
Many filenames can be supplied as multiple parameters, parameters with wildcards or
|
||||
names of directories can be used as parameters with the `-r` option.
|
||||
|
||||
The benchmark measures ratio, compressed size, compression and decompression speed.
|
||||
One can select compression levels starting from `-b` and ending with `-e`.
|
||||
The `-i` parameter selects minimal time used for each of tested levels.
|
||||
|
||||
|
||||
|
||||
#### Usage of Command Line Interface
|
||||
The full list of options can be obtained with `-h` or `-H` parameter:
|
||||
```
|
||||
Usage :
|
||||
zstd [args] [FILE(s)] [-o file]
|
||||
|
||||
FILE : a filename
|
||||
with no FILE, or when FILE is - , read standard input
|
||||
Arguments :
|
||||
-# : # compression level (1-19, default:3)
|
||||
-d : decompression
|
||||
-D file: use `file` as Dictionary
|
||||
-o file: result stored into `file` (only if 1 input file)
|
||||
-f : overwrite output without prompting
|
||||
--rm : remove source file(s) after successful de/compression
|
||||
-k : preserve source file(s) (default)
|
||||
-h/-H : display help/long help and exit
|
||||
|
||||
Advanced arguments :
|
||||
-V : display Version number and exit
|
||||
-v : verbose mode; specify multiple times to increase log level (default:2)
|
||||
-q : suppress warnings; specify twice to suppress errors too
|
||||
-c : force write to standard output, even if it is the console
|
||||
-r : operate recursively on directories
|
||||
--ultra : enable levels beyond 19, up to 22 (requires more memory)
|
||||
--no-dictID : don't write dictID into header (dictionary compression)
|
||||
--[no-]check : integrity check (default:enabled)
|
||||
--test : test compressed file integrity
|
||||
--[no-]sparse : sparse mode (default:enabled on file, disabled on stdout)
|
||||
|
||||
Dictionary builder :
|
||||
--train ## : create a dictionary from a training set of files
|
||||
-o file : `file` is dictionary name (default: dictionary)
|
||||
--maxdict ## : limit dictionary to specified size (default : 112640)
|
||||
-s# : dictionary selectivity level (default: 9)
|
||||
--dictID ## : force dictionary ID to specified value (default: random)
|
||||
|
||||
Benchmark arguments :
|
||||
-b# : benchmark file(s), using # compression level (default : 1)
|
||||
-e# : test all compression levels from -bX to # (default: 1)
|
||||
-i# : minimum evaluation time in seconds (default : 3s)
|
||||
-B# : cut file into independent blocks of size # (default: no block)
|
||||
```
|
||||
@ -4,11 +4,11 @@ projects for various integrated development environments (IDE)
|
||||
#### Included projects
|
||||
|
||||
The following projects are included with the zstd distribution:
|
||||
- cmake - CMake project contributed by Artyom Dymchenko
|
||||
- VS2005 - Visual Studio 2005 project
|
||||
- VS2008 - Visual Studio 2008 project
|
||||
- VS2010 - Visual Studio 2010 project (which also works well with Visual Studio 2012, 2013, 2015)
|
||||
- build - command line scripts prepared for Visual Studio compilation without IDE
|
||||
- `cmake` - CMake project contributed by Artyom Dymchenko
|
||||
- `VS2005` - Visual Studio 2005 project
|
||||
- `VS2008` - Visual Studio 2008 project
|
||||
- `VS2010` - Visual Studio 2010 project (which also works well with Visual Studio 2012, 2013, 2015)
|
||||
- `build` - command line scripts prepared for Visual Studio compilation without IDE
|
||||
|
||||
|
||||
#### How to compile zstd with Visual Studio
|
||||
|
||||
@ -2,24 +2,24 @@ programs and scripts for automated testing of zstd
|
||||
================================
|
||||
|
||||
This directory contains the following programs and scripts:
|
||||
- datagen : Synthetic and parametrable data generator, for tests
|
||||
- fullbench : Precisely measure speed for each zstd inner functions
|
||||
- fuzzer : Test tool, to check zstd integrity on target platform
|
||||
- paramgrill : parameter tester for zstd
|
||||
- test-zstd-speed.py : script for testing zstd speed difference between commits
|
||||
- test-zstd-versions.py : compatibility test between zstd versions stored on Github (v0.1+)
|
||||
- zbufftest : Test tool, to check ZBUFF integrity on target platform
|
||||
- zstreamtest : Fuzzer test tool for zstd streaming API
|
||||
- `datagen` : Synthetic and parametrable data generator, for tests
|
||||
- `fullbench` : Precisely measure speed for each zstd inner functions
|
||||
- `fuzzer` : Test tool, to check zstd integrity on target platform
|
||||
- `paramgrill` : parameter tester for zstd
|
||||
- `test-zstd-speed.py` : script for testing zstd speed difference between commits
|
||||
- `test-zstd-versions.py` : compatibility test between zstd versions stored on Github (v0.1+)
|
||||
- `zbufftest` : Test tool to check ZBUFF integrity on target platform
|
||||
- `zstreamtest` : Fuzzer test tool for zstd streaming API
|
||||
|
||||
|
||||
#### test-zstd-versions.py - script for testing zstd interoperability between versions
|
||||
#### `test-zstd-versions.py` - script for testing zstd interoperability between versions
|
||||
|
||||
This script creates `versionsTest` directory to which zstd repository is cloned.
|
||||
Then all taged (released) versions of zstd are compiled.
|
||||
In the following step interoperability between zstd versions is checked.
|
||||
|
||||
|
||||
#### test-zstd-speed.py - script for testing zstd speed difference between commits
|
||||
#### `test-zstd-speed.py` - script for testing zstd speed difference between commits
|
||||
|
||||
This script creates `speedTest` directory to which zstd repository is cloned.
|
||||
Then it compiles all branches of zstd and performs a speed benchmark for a given list of files (the `testFileNames` parameter).
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user