638 Commits

Author SHA1 Message Date
sen
6030cdfede
Add --progress flag (#2595) 2021-05-06 14:50:28 -04:00
W. Felix Handte
4f9c6fdb7f Attempt to Fix Windows Build Error 2021-05-05 13:13:56 -04:00
W. Felix Handte
da61918c75 Also Pass Mode Bits in on Windows
I think in some unix emulation environments on Windows, (cygwin?) mode bits
are somehow respected. So we might as well pass them in. Can't hurt.
2021-05-05 13:10:34 -04:00
W. Felix Handte
45c4918ccf Fix Build for Windows 2021-05-05 13:10:34 -04:00
W. Felix Handte
1fb10ba831 Don't Block Removing File on Being Able to Read It
`open()`'s mode bits are only applied to files that are created by the call.
If the output file already exists, but is not readable, the `fopen()` would
fail, preventing us from removing it, which would mean that the file would
not end up with the correct permission bits.

It's not clear to me why the `fopen()` is there at all. `UTIL_isRegularFile()`
should be sufficient, AFAICT.
2021-05-05 13:10:34 -04:00
W. Felix Handte
b87f97b3ea Create Files with Desired Permissions; Avoid chmod(); Remove UTIL_chmod() 2021-05-05 13:10:34 -04:00
W. Felix Handte
33f3e293e8 Allow Reading from Block Devices with --force 2021-05-04 16:25:26 -04:00
Nick Terrell
09149beaf8 [1.5.0] Move zstd_errors.h and zdict.h to lib/ root
`zstd_errors.h` and `zdict.h` are public headers, so they deserve to be
in the root `lib/` directory with `zstd.h`, not mixed in with our private
headers.
2021-04-30 15:13:54 -07:00
Nick Terrell
4694423c4f Add and integrate lazy row hash strategy 2021-04-07 09:53:34 -07:00
Nick Terrell
a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:30:43 -07:00
Yann Collet
9fb4a42c7b fix #2549 2021-03-20 17:29:41 -07:00
Stephen Kitt
adb54293ab
Stop using deprecated reset?Stream functions
These are replaced by the corresponding context resets. When
converting resetCStream, CCtx_setPledgedSrcSize isn't called if the
source size is "unknown".

This helps reduce the reliance on "static only" symbols, as well as
reducing the use of deprecated functions.

Signed-off-by: Stephen Kitt <steve@sk2.org>
2021-02-23 21:56:01 +01:00
W. Felix Handte
a774c57973 Use umask() to Constrain Created File Permissions
This commit addresses #2491.

Note that a downside of this solution is that it is global: `umask()` affects
all file creation calls in the process. I believe this is safe since
`fileio.c` functions should only ever be used in the zstd binary, and these
are (almost) the only files ever created by zstd, and AIUI they're only
created in a single thread. So we can get away with messing with global state.

Note that this doesn't change the permissions of files created by `dibio.c`.
I'm not sure what those should be...
2021-02-17 15:27:39 -05:00
Yann Collet
0d793a675a removed internal dependency from CLI
ZSTD_cycleLog() is a very short function,
creating a rather large dependency onto libzstd's internal just for it is overkill.
Prefer duplicating this 2-lines function.

This PR makes the zstd CLI one step closer to being linkable to the dynamic library (see #2450)
More steps are still needed to reach this goal.
2021-01-06 01:35:52 -08:00
Nick Terrell
66e811d782 [license] Update year to 2021 2021-01-04 17:53:52 -05:00
sen
f54109c589
Merge pull request #2378 from senhuang42/free_cress_ptr
[minor] Pass cRess_t by const ptr instead of by value
2020-11-01 20:33:25 -05:00
senhuang42
043b934ba2 Pass cRess_t by const const pointer 2020-10-14 20:19:46 -04:00
senhuang42
467e4383b0 Merge branch 'dev' of github.com:senhuang42/zstd into change_ldm_mt_config 2020-10-14 10:17:50 -04:00
senhuang42
a39614de1f Fix incorrect usage of cycleLog() in --patch-from 2020-10-13 13:00:27 -04:00
senhuang42
dd3dac9ce0 Fixed logic for stdout output 2020-10-07 13:43:27 -04:00
senhuang42
1ebe360d0f Add new stdoutOutput field 2020-10-07 13:42:34 -04:00
Yann Collet
83461ce963
Merge pull request #2322 from senhuang42/guard_against_stdin_for_warning_prompts
Don't let warning messages consume input from stdin
2020-09-30 08:26:50 -07:00
Yann Collet
236b98e7de
Merge pull request #2310 from senhuang42/fix_multifile_status_bar
Fix multifile status bar and summary - clear out extraneous characters
2020-09-25 07:48:33 -07:00
senhuang42
9f7212a48b Update unit tests 2020-09-24 16:44:33 -04:00
senhuang42
93d63eaeb8 Expand UTIL_requireUserConfirmation to include stdin input check 2020-09-24 15:58:06 -04:00
senhuang42
432186cbea Add FIO_determineHasStdinInput() function and member to fCtx 2020-09-24 15:55:30 -04:00
senhuang42
0e8ac6b995 Add fCtx to FIO_openDstFile() 2020-09-24 15:49:30 -04:00
senhuang42
6b6cc80196 Support .zstd suffix only for decompression 2020-09-18 12:49:51 -04:00
senhuang42
29b8f2eb08 Fix multifile decompression summary 2020-09-16 10:38:13 -04:00
senhuang42
1d5c6fda9f Add filename truncations to decompression 2020-09-16 10:28:45 -04:00
senhuang42
ab0d332563 Add display clears 2020-09-15 15:53:32 -04:00
senhuang42
cad6bf99af Add padding to remove previous line's leftovers, and keep the printed line to around the same size 2020-09-15 13:01:46 -04:00
Yann Collet
05622992d5
Merge pull request #2292 from senhuang42/multifile_status_update_fio_refactor
Make multifile (de)compression print out a summary
2020-09-14 14:06:26 -07:00
W. Felix Handte
d46306087b Enable Dedicated Dict Search in the CLI 2020-09-10 18:51:52 -04:00
senhuang42
5123496104 Adjust function signatures 2020-09-07 13:13:05 -04:00
senhuang42
28a9dc78c3 Always update bytes status in FIO_ctx_t 2020-09-03 20:23:30 -04:00
senhuang42
60ee0519ca Remove extraneous FIO_ctx_t functions 2020-09-03 20:22:56 -04:00
senhuang42
23feaecbe5 Fix FIO_removeMultiFilesWarning conflict 2020-09-03 09:49:13 -04:00
senhuang42
49949f1d51 Merge branch 'dev' into multifile_status_update_fio_refactor 2020-09-03 09:44:47 -04:00
senhuang42
202b2954a2 Remove comma-separated decls 2020-09-03 09:28:40 -04:00
senhuang42
a480b02044 Adjust comments and struct to better reflect FIO_ctx_t 2020-09-03 09:26:30 -04:00
senhuang42
7842f43197 Fix 1 file multifile logic, remove unnecessary variable assignments 2020-09-03 09:22:07 -04:00
senhuang42
dbe5305250 Adjusted extra explanation warning to always pop up if applicable 2020-09-02 08:44:42 -04:00
senhuang42
99039988a5 Fixed newline issue and adjusted wording in comment 2020-09-01 13:18:30 -04:00
senhuang42
3a7d625d6b Cleanup comments, add function to set FIO_ctx_t.nbFilesProcessed 2020-09-01 12:54:21 -04:00
senhuang42
a03917c751 Fix the logic in printing final status updates 2020-09-01 12:46:06 -04:00
senhuang42
a6414f1247 Integrate refactor into status print for multifiles, adjust logic for printing as needed 2020-09-01 12:34:43 -04:00
senhuang42
d54566f334 First pass at refactoring and creating new FIO_ctx_t */ 2020-08-28 11:01:04 -04:00
senhuang42
01828b27e2 Improve documentation, update man 2020-08-27 17:57:20 -04:00
senhuang42
7e867ad61f Fix potential memory leak 2020-08-26 18:52:32 -04:00