Commit Graph

86 Commits (28a9dc78c3b4e6d35041c3ffa3dfda241ec102a1)

Author SHA1 Message Date
senhuang42 60ee0519ca Remove extraneous FIO_ctx_t functions 2020-09-03 20:22:56 -04:00
senhuang42 a480b02044 Adjust comments and struct to better reflect FIO_ctx_t 2020-09-03 09:26:30 -04:00
senhuang42 3a7d625d6b Cleanup comments, add function to set FIO_ctx_t.nbFilesProcessed 2020-09-01 12:54:21 -04:00
senhuang42 a6414f1247 Integrate refactor into status print for multifiles, adjust logic for printing as needed 2020-09-01 12:34:43 -04:00
senhuang42 d54566f334 First pass at refactoring and creating new FIO_ctx_t */ 2020-08-28 11:01:04 -04:00
senhuang42 b6abbc3ce5 Fix single file decompression summary, remove escape code dependency, add currFileIdx field to FIO_prefs, general cleanups/bugfixes 2020-08-26 11:35:07 -04:00
senhuang42 da38891a87 Add initial live update with displayLevel = 2, add new field to FIO_prefs_t 2020-08-25 16:46:47 -04:00
Xin Xie 9a8ccd4ba3 Add output-dir-mirror option 2020-06-24 22:12:11 -07:00
W. Felix Handte 7dcca6bc64 Also Move programs/ Directory to Relative Includes 2020-05-04 15:20:26 -04:00
Nick Terrell ac58c8d720 Fix copyright and license lines
* All copyright lines now have -2020 instead of -present
* All copyright lines include "Facebook, Inc"
* All licenses are now standardized

The copyright in `threading.{h,c}` is not changed because it comes from
zstdmt.

The copyright and license of `divsufsort.{h,c}` is not changed.
2020-03-26 17:02:06 -07:00
Bimba Shrestha 6d8e761ba3 Inverting content size prefs bool 2020-03-09 14:16:10 -05:00
Bimba Shrestha 167244a64f --no-content-size cli 2020-03-09 13:07:29 -05:00
Bimba Shrestha f25a6e9f8f Adding new cli endpoint --patch-from= (#1940)
* Adding new cli endpoint --diff-from=

* Appveyor conversion nit

* Using bool set trick instead of direct set

* Removing --diff-from and only leaving --diff-from=#

* Throwing error when both dictFileName vars are set

* Clean up syntax

* Renaming diff-from to patch-from

* Revering comma separated syntax clean up

* Updating playtests with patch-from

* Uncommenting accidentally commented

* Updating remaining docs and var names to be patch-from instead of diff-from

* Constifying

* Using existing log2 function and removing newly created one

* Argument order (moving prefs to end)

* Using comma separated syntax

* Moving to outside #ifndef
2020-01-10 14:25:24 -08:00
Shashank Tavildar 0f2bff2faf Addressing comments, removing cyclic dependency with header file, updating tests 2019-10-28 18:21:47 -07:00
W. Felix Handte 4eccc82875 Minor Fixes 2019-10-25 13:59:19 -04:00
W. Felix Handte 91c3f545cc Fix Build; Refactor 2019-10-24 20:18:57 -04:00
Felix Handte 506e1a1176
Merge branch 'tzst-ext-tmp' into tzst-ext 2019-10-24 17:49:34 -04:00
Yann Collet 0a24d4ef18 removed regular file test on Windows
since it does not work well on this platform
(tested with MinGW).

Note : could be an issue within UTIL_isRegularFile()
2019-10-17 16:39:47 -07:00
Yann Collet 0ee360982d improved test mode `-t`
The test mode do no longer open a file `/dev/null` nor write anything to output.

This is supposed to be more efficient than writing to `/dev/null`,
and more universal too : the previous method was failing on Windows.
2019-10-17 16:09:53 -07:00
Yann Collet 1795133c45 refactored FIO_compressMultipleFilenames() prototype
for consistency
2019-10-17 15:32:03 -07:00
Sen Huang 64bc441d7d Now constructs final destination path without allocating new table 2019-10-03 13:53:04 -04:00
Sergey Ponomarev 59f369a6da Add short tar's extensions .tgz (.tar.gz), .txz (.tar.xz), .tzst (.tar.zst) 2019-09-14 21:30:15 +03:00
Sen Huang 30bff50e06 fixes for tests and segfault 2019-09-06 11:10:53 -07:00
Sen Huang 6beb3c0159 added decompression support 2019-09-05 17:56:24 -07:00
Sen Huang 7f98b46876 adding support for -O flag: multiple files into one directory for compressions (decompression to come) 2019-09-05 16:03:35 -07:00
Nick Magerko 493f95c7df Fix merge conflicts 2019-08-22 11:51:41 -07:00
Nick Magerko dffbac5f89 Add --size-hint=# option 2019-08-19 11:38:49 -07:00
Nick Magerko af0c9501d1 Add --stream-size=# command 2019-08-16 11:34:39 -07:00
Ephraim Park 9007701670 Adding targetCBlockSize param 2019-07-03 15:41:52 -07:00
Nick Terrell 0c53c5ad4a [zstdcli] Add a flag to control literals compression 2019-02-15 15:00:45 -08:00
Yann Collet 07e047873e fixed fileio.c compilation with LZ4 enabled
was broken by #1505.

I'm surprised it passed CI tests.
LZ4 tests are part of the "Extended" tests on Travis CI,
which are run on "master" and in "cron" jobs.

Since latest cron job did not failed,
especially this one : https://travis-ci.org/facebook/zstd/jobs/484365040
it suggests cron jobs are no longer using `dev` branch.

To be investigated
2019-01-25 14:42:44 -08:00
Karl Ostmo 5e220bf4b5 Remove global parameters, pass into public functions instead 2019-01-23 16:06:27 -08:00
Nick Terrell bdfcaecc0a [zstdcli] Add --no-progress flag
The `--no-progress` flag disables zstd's progress bars, but leaves
the summary.

I've added simple tests to `playTests.sh` to make sure the parsing
works.
2018-12-14 11:50:25 -08:00
Yann Collet 41c7d0b1e1 changed hashEveryLog into hashRateLog 2018-11-21 14:36:57 -08:00
Nick Terrell f9a671ac37 [cli] Add rsyncable 2018-11-14 17:00:16 -08:00
Yann Collet 6c51bf420c bounds for --adapt mode
can supply min and max compression level through advanced command :
--adapt=min=#,max=#
2018-09-25 16:03:28 -07:00
Yann Collet 2f78228f65 Merge branch 'dev' into adapt 2018-09-19 12:43:42 -07:00
Casey McGinty d4337b6f1d
Move ABRThandler func out of internal lib 2018-09-11 11:39:49 -07:00
Yann Collet 33f7709c71 fileio: changed parameter type from ptr to plain structure
safer : this parameter is read-only,
we don't want original structure to be modified
2018-08-13 13:02:03 -07:00
Yann Collet e7a49c6683 introduced command --adapt 2018-08-11 20:48:06 -07:00
Yann Collet 209df52ba2 Changed nbThreads for nbWorkers
This makes it easier to explain that nbWorkers=0 --> single-threaded mode,
while nbWorkers=1 --> asynchronous mode (one mode thread on top of the "main" caller thread).
No need for an additional asynchronous mode flag.
nbWorkers>=2 works the same as nbThreads>=2 previously.
2018-02-01 19:29:30 -08:00
Nick Terrell 4680e85bdf Allow -o with multiple files 2017-12-13 17:44:34 -08:00
Stella Lau eb3327c10a Merge branch 'dev' of https://github.com/facebook/zstd into ldm-mergeDev 2017-09-11 15:00:01 -07:00
Yann Collet 3128e03be6 updated license header
to clarify dual-license meaning as "or"
2017-09-08 00:09:23 -07:00
Stella Lau 67d4a6161c Add ldmBucketSizeLog param 2017-09-02 21:55:29 -07:00
Stella Lau a1f04d518d Move hashEveryLog to cctxParams and update cli 2017-09-01 15:05:47 -07:00
Stella Lau 17d8e0bdcc Merge remote-tracking branch 'upstream/longRangeMatcher' into ldm-integrate 2017-09-01 10:19:38 -07:00
Stella Lau 6a546efb8c Add long distance matcher
Move last literals section to ZSTD_block_internal
2017-08-31 12:53:19 -07:00
Yann Collet 32fb407c9d updated a bunch of headers
for the new license
2017-08-18 16:52:05 -07:00
Paul Cruz bda57754a2 moved signifcant operations to fileio.c 2017-06-20 12:43:10 -07:00