Commit Graph

1942 Commits (bd547232bc0aa43591b9eabf51a78ae3cb10fdcf)

Author SHA1 Message Date
sen 6030cdfede
Add --progress flag (#2595) 2021-05-06 14:50:28 -04:00
Yann Collet 2f7bbd6539
Merge pull request #2620 from facebook/winFilelist
fix --filelist compatibility with Windows cr+lf line ending
2021-05-06 11:35:16 -07:00
Yann Collet df05b2ba7c fix --filelist compatibility with Windows cr+lf line ending 2021-05-05 18:01:55 -07:00
Yann Collet 9750f3c87b improved benchmark experience on Windows
benchmark results are not progressively displayed on Windows terminal.
For long benchmark sessions, nothing is displayed,
until the end, where everything is flushed.

Force display to be flushed after each update.
Updates happen roughtly every second, or even less,
so it's not a substantial workload.
2021-05-05 16:52:21 -07:00
W. Felix Handte 4f9c6fdb7f Attempt to Fix Windows Build Error 2021-05-05 13:13:56 -04:00
W. Felix Handte da61918c75 Also Pass Mode Bits in on Windows
I think in some unix emulation environments on Windows, (cygwin?) mode bits
are somehow respected. So we might as well pass them in. Can't hurt.
2021-05-05 13:10:34 -04:00
W. Felix Handte 45c4918ccf Fix Build for Windows 2021-05-05 13:10:34 -04:00
W. Felix Handte 1fb10ba831 Don't Block Removing File on Being Able to Read It
`open()`'s mode bits are only applied to files that are created by the call.
If the output file already exists, but is not readable, the `fopen()` would
fail, preventing us from removing it, which would mean that the file would
not end up with the correct permission bits.

It's not clear to me why the `fopen()` is there at all. `UTIL_isRegularFile()`
should be sufficient, AFAICT.
2021-05-05 13:10:34 -04:00
W. Felix Handte b87f97b3ea Create Files with Desired Permissions; Avoid chmod(); Remove UTIL_chmod() 2021-05-05 13:10:34 -04:00
Felix Handte 2d10544b84
Merge pull request #2613 from felixhandte/allow-block-device
Allow Reading from Block Devices with `--force`
2021-05-05 13:06:32 -04:00
Yann Collet 455fd1a067 updated documentation regarding minimum job size 2021-05-05 09:03:11 -07:00
W. Felix Handte 33f3e293e8 Allow Reading from Block Devices with `--force` 2021-05-04 16:25:26 -04:00
Nick Terrell a8ecf4ff88
Merge pull request #2597 from terrelln/public-headers
[1.5.0] Move `zstd_errors.h` and `zdict.h` to `lib/` root
2021-05-04 11:28:41 -07:00
W. Felix Handte ee122baacf Detect Presence of `md5` on Darwin
This fixes #2568.
2021-05-04 12:33:19 -04:00
Nick Terrell 09149beaf8 [1.5.0] Move `zstd_errors.h` and `zdict.h` to `lib/` root
`zstd_errors.h` and `zdict.h` are public headers, so they deserve to be
in the root `lib/` directory with `zstd.h`, not mixed in with our private
headers.
2021-04-30 15:13:54 -07:00
Nick Terrell 4694423c4f Add and integrate lazy row hash strategy 2021-04-07 09:53:34 -07:00
Niclas Rosenvik e7647180cd Stop complaining about hash tool not found
If build_dir is set the zstd build complains about md5sum not being found.
Fix this by checking if build_dir is set before checking and using the hash tool
just like in lib/Makefile .
2021-04-02 13:00:19 -07:00
Nick Terrell a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:30:43 -07:00
Sen Huang f27e326456 Restrict dictmode regression tests only to advanced API, fix some compiler warnings 2021-03-25 10:39:08 -07:00
Yann Collet 0f99a0d987 fix man page typo
fix #2553 detected by @jwilk
2021-03-24 05:55:04 -07:00
Yann Collet 9fb4a42c7b fix #2549 2021-03-20 17:29:41 -07:00
Paul Bone 4d6c78fb89 Only set numPhysicalCores if ratio is valid 2021-03-03 10:59:00 +11:00
Paul Bone eb1a09df61 If cpuinfo parsing fails fallback to sysconf 2021-03-03 10:58:51 +11:00
W. Felix Handte 61db590ad8 Detect `..` in Paths Correctly
This commit addresses #2509.
2021-02-26 12:29:42 -05:00
Yann Collet 61b63e9060
Merge pull request #2492 from niacat/dev
Use standard md5 tool on NetBSD.
2021-02-24 16:38:10 -08:00
Stephen Kitt adb54293ab
Stop using deprecated reset?Stream functions
These are replaced by the corresponding context resets. When
converting resetCStream, CCtx_setPledgedSrcSize isn't called if the
source size is "unknown".

This helps reduce the reliance on "static only" symbols, as well as
reducing the use of deprecated functions.

Signed-off-by: Stephen Kitt <steve@sk2.org>
2021-02-23 21:56:01 +01:00
Felix Handte a2adc6df9f
Merge pull request #2495 from felixhandte/umask
Use umask() to Constrain Created File Permissions
2021-02-17 17:03:23 -05:00
W. Felix Handte a774c57973 Use umask() to Constrain Created File Permissions
This commit addresses #2491.

Note that a downside of this solution is that it is global: `umask()` affects
all file creation calls in the process. I believe this is safe since
`fileio.c` functions should only ever be used in the zstd binary, and these
are (almost) the only files ever created by zstd, and AIUI they're only
created in a single thread. So we can get away with messing with global state.

Note that this doesn't change the permissions of files created by `dibio.c`.
I'm not sure what those should be...
2021-02-17 15:27:39 -05:00
senhuang42 444c4650a0 Add newline to end of cli help message 2021-02-17 12:30:42 -05:00
nia 74f85818a6 Use standard md5 tool on NetBSD.
This avoids a GNU coreutils dependency.

-n is used to match the output format of coreutils:
http://man.netbsd.org/md5.1
2021-02-11 10:50:11 +01:00
Nick Terrell e59c9459a5 [trace] Keep track of a uint64_t tracing context
The most common information that you want to track between begin() and
end() is the timestamp of the begin function, so you can measure the
duration of the (de)compression call. Allow the tracing library to put
this information inside the `ZSTD_TraceCtx`, so it doesn't need to keep
a global map in this case. If a single uint64_t is not enough, the
tracing library can return a unique identifier (like the context
pointer) instead, and use it as a key in a map.

This keeps the simple case simple.
2021-02-09 11:37:05 -08:00
Nick Terrell 54a4998a80 Add basic tracing functionality 2021-02-05 16:28:52 -08:00
Yann Collet 7e6729055a
Merge pull request #2475 from facebook/parallel_build
parallel make build on linux
2021-01-19 10:19:41 -08:00
Yann Collet 0bad3e5c0f parallel make build on linux
fix #2474
2021-01-18 11:33:03 -08:00
W. Felix Handte 927859f5e8 Also Update Man Page Documentation 2021-01-11 17:55:58 -05:00
W. Felix Handte 8b6a4b5b7c Allow Input From Console When `--force` is Passed
Also update option flag documentation.
2021-01-11 17:53:20 -05:00
Yann Collet 3324e87cff Added library version check 2021-01-07 10:37:27 -08:00
Yann Collet cefdc023f7 The CLI can be linked to libzstd dynamic library
invoking target zstd-dll
2021-01-06 18:00:24 -08:00
Yann Collet 890d85bdb4 removed CLI dependency to legacy unsafe function
makint the CLI ons step closer to being linkable to the dynamic library
2021-01-06 16:19:42 -08:00
Yann Collet 0d793a675a removed internal dependency from CLI
ZSTD_cycleLog() is a very short function,
creating a rather large dependency onto libzstd's internal just for it is overkill.
Prefer duplicating this 2-lines function.

This PR makes the zstd CLI one step closer to being linkable to the dynamic library (see #2450)
More steps are still needed to reach this goal.
2021-01-06 01:35:52 -08:00
Nick Terrell 66e811d782 [license] Update year to 2021 2021-01-04 17:53:52 -05:00
Yann Collet f2ac2b7bcf try to fix cross-compiler tests 2020-12-21 15:43:14 -08:00
Yann Collet 9a9d3f76c4 fixed zstd+sanitizer build 2020-12-20 17:53:04 -08:00
Yann Collet f9884036c2 fixed zstd recipe 2020-12-20 17:19:23 -08:00
Yann Collet 9648bf027b try to keep libzstd.a "as is" once created
to be compatible with scenarios such as
`make -j allmost`
2020-12-20 17:10:57 -08:00
Yann Collet 7c495e8ea2 updated version number to v1.4.8 2020-12-18 15:52:11 -08:00
Yann Collet f647a759fe updated version number to v1.4.7
and updated doc
2020-12-15 20:53:05 -08:00
ihsinme f37896db02
fix Integer Overflow 2020-12-13 12:08:31 +03:00
Yann Collet c2939b96be refinement : only copy if binary is different 2020-12-10 14:31:35 -08:00
Yann Collet c56723ab03 replace final links by direct copy
link can behave slightly differently from real binaries,
breaking a few scripts relying on "real binary" assumption.
2020-12-10 13:25:08 -08:00