Commit Graph

1927 Commits (2e4fca38d8585e0cecb584ad5279306640897b4a)

Author SHA1 Message Date
Nick Terrell 4694423c4f Add and integrate lazy row hash strategy 2021-04-07 09:53:34 -07:00
Niclas Rosenvik e7647180cd Stop complaining about hash tool not found
If build_dir is set the zstd build complains about md5sum not being found.
Fix this by checking if build_dir is set before checking and using the hash tool
just like in lib/Makefile .
2021-04-02 13:00:19 -07:00
Nick Terrell a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:30:43 -07:00
Sen Huang f27e326456 Restrict dictmode regression tests only to advanced API, fix some compiler warnings 2021-03-25 10:39:08 -07:00
Yann Collet 0f99a0d987 fix man page typo
fix #2553 detected by @jwilk
2021-03-24 05:55:04 -07:00
Yann Collet 9fb4a42c7b fix #2549 2021-03-20 17:29:41 -07:00
Paul Bone 4d6c78fb89 Only set numPhysicalCores if ratio is valid 2021-03-03 10:59:00 +11:00
Paul Bone eb1a09df61 If cpuinfo parsing fails fallback to sysconf 2021-03-03 10:58:51 +11:00
W. Felix Handte 61db590ad8 Detect `..` in Paths Correctly
This commit addresses #2509.
2021-02-26 12:29:42 -05:00
Yann Collet 61b63e9060
Merge pull request #2492 from niacat/dev
Use standard md5 tool on NetBSD.
2021-02-24 16:38:10 -08:00
Stephen Kitt adb54293ab
Stop using deprecated reset?Stream functions
These are replaced by the corresponding context resets. When
converting resetCStream, CCtx_setPledgedSrcSize isn't called if the
source size is "unknown".

This helps reduce the reliance on "static only" symbols, as well as
reducing the use of deprecated functions.

Signed-off-by: Stephen Kitt <steve@sk2.org>
2021-02-23 21:56:01 +01:00
Felix Handte a2adc6df9f
Merge pull request #2495 from felixhandte/umask
Use umask() to Constrain Created File Permissions
2021-02-17 17:03:23 -05:00
W. Felix Handte a774c57973 Use umask() to Constrain Created File Permissions
This commit addresses #2491.

Note that a downside of this solution is that it is global: `umask()` affects
all file creation calls in the process. I believe this is safe since
`fileio.c` functions should only ever be used in the zstd binary, and these
are (almost) the only files ever created by zstd, and AIUI they're only
created in a single thread. So we can get away with messing with global state.

Note that this doesn't change the permissions of files created by `dibio.c`.
I'm not sure what those should be...
2021-02-17 15:27:39 -05:00
senhuang42 444c4650a0 Add newline to end of cli help message 2021-02-17 12:30:42 -05:00
nia 74f85818a6 Use standard md5 tool on NetBSD.
This avoids a GNU coreutils dependency.

-n is used to match the output format of coreutils:
http://man.netbsd.org/md5.1
2021-02-11 10:50:11 +01:00
Nick Terrell e59c9459a5 [trace] Keep track of a uint64_t tracing context
The most common information that you want to track between begin() and
end() is the timestamp of the begin function, so you can measure the
duration of the (de)compression call. Allow the tracing library to put
this information inside the `ZSTD_TraceCtx`, so it doesn't need to keep
a global map in this case. If a single uint64_t is not enough, the
tracing library can return a unique identifier (like the context
pointer) instead, and use it as a key in a map.

This keeps the simple case simple.
2021-02-09 11:37:05 -08:00
Nick Terrell 54a4998a80 Add basic tracing functionality 2021-02-05 16:28:52 -08:00
Yann Collet 7e6729055a
Merge pull request #2475 from facebook/parallel_build
parallel make build on linux
2021-01-19 10:19:41 -08:00
Yann Collet 0bad3e5c0f parallel make build on linux
fix #2474
2021-01-18 11:33:03 -08:00
W. Felix Handte 927859f5e8 Also Update Man Page Documentation 2021-01-11 17:55:58 -05:00
W. Felix Handte 8b6a4b5b7c Allow Input From Console When `--force` is Passed
Also update option flag documentation.
2021-01-11 17:53:20 -05:00
Yann Collet 3324e87cff Added library version check 2021-01-07 10:37:27 -08:00
Yann Collet cefdc023f7 The CLI can be linked to libzstd dynamic library
invoking target zstd-dll
2021-01-06 18:00:24 -08:00
Yann Collet 890d85bdb4 removed CLI dependency to legacy unsafe function
makint the CLI ons step closer to being linkable to the dynamic library
2021-01-06 16:19:42 -08:00
Yann Collet 0d793a675a removed internal dependency from CLI
ZSTD_cycleLog() is a very short function,
creating a rather large dependency onto libzstd's internal just for it is overkill.
Prefer duplicating this 2-lines function.

This PR makes the zstd CLI one step closer to being linkable to the dynamic library (see #2450)
More steps are still needed to reach this goal.
2021-01-06 01:35:52 -08:00
Nick Terrell 66e811d782 [license] Update year to 2021 2021-01-04 17:53:52 -05:00
Yann Collet f2ac2b7bcf try to fix cross-compiler tests 2020-12-21 15:43:14 -08:00
Yann Collet 9a9d3f76c4 fixed zstd+sanitizer build 2020-12-20 17:53:04 -08:00
Yann Collet f9884036c2 fixed zstd recipe 2020-12-20 17:19:23 -08:00
Yann Collet 9648bf027b try to keep libzstd.a "as is" once created
to be compatible with scenarios such as
`make -j allmost`
2020-12-20 17:10:57 -08:00
Yann Collet 7c495e8ea2 updated version number to v1.4.8 2020-12-18 15:52:11 -08:00
Yann Collet f647a759fe updated version number to v1.4.7
and updated doc
2020-12-15 20:53:05 -08:00
ihsinme f37896db02
fix Integer Overflow 2020-12-13 12:08:31 +03:00
Yann Collet c2939b96be refinement : only copy if binary is different 2020-12-10 14:31:35 -08:00
Yann Collet c56723ab03 replace final links by direct copy
link can behave slightly differently from real binaries,
breaking a few scripts relying on "real binary" assumption.
2020-12-10 13:25:08 -08:00
Yann Collet fed1c62571 fix gcc10 warnings
gcc10 doesn't like its own strncpy
2020-11-30 04:44:37 -08:00
sen f54109c589
Merge pull request #2378 from senhuang42/free_cress_ptr
[minor] Pass cRess_t by const ptr instead of by value
2020-11-01 20:33:25 -05:00
Yann Collet 0adce4631d Merge branch 'libzstd_autoconf_full' of github.com:facebook/zstd into libzstd_autoconf_full 2020-10-28 10:25:55 -07:00
Yann Collet f6ecf1568f minor Makefile refactor
hopefully improving readability
2020-10-28 09:39:15 -07:00
Yann Collet ceccd7ae2d Merge branch 'dev' into libzstd_autoconf_full 2020-10-27 15:45:30 -07:00
Björn Ketelaars 1f661b5f6b 'head -c BYTES' is non-portable 2020-10-27 16:55:23 +01:00
Yann Collet 456db0c377 make install only rebuild binaries if they don't exist
Now `make` followed by `make install` doesn't rebuild binaries

also : only generated target directories if they don't already exist
2020-10-23 16:46:49 -07:00
Yann Collet a6ee614a44 make zstd is now differentiated from zstd-nomt
avoids mixing object files using different flags
2020-10-23 16:08:21 -07:00
Yann Collet 89b961ea46 simplified silent mode maintenance 2020-10-23 10:41:17 -07:00
Yann Collet ce6cd07c33 updated build documentation 2020-10-22 12:31:23 -07:00
Yann Collet e3867fb735 fixed libzstd.dll compilation on mingw
and zstd linking
2020-10-22 11:52:19 -07:00
Yann Collet dd24496951 programs/zstd also automatically generate object dir per conf
same rules as lib/libzstd
can also be controlled via HASH and BUILD_DIR
2020-10-21 23:38:33 -07:00
Yann Collet 01ecad2326 zstd in programs/ can also receive a DEBUGLEVEL value
simplify tests : only set DEBUGLEVEL, not the flags directly
2020-10-21 23:13:46 -07:00
Yann Collet 2224ec33ed
Merge pull request #2367 from facebook/progressive_build
faster rebuild of zstd
2020-10-21 15:43:14 -07:00
Yann Collet 684f8ea970 decouple zstd object files from lib/
That was a subtle one :
VPATH is affecting search for both %.c source and %.o object files.
This meant that, when an object file already exists in lib/,
it's used in programs/,
even though programs/ is supposed to generate its own %.o object files.

With the new vpath directive, this is no longer the case :
the search is only activated for %.c source files.
Now, local programs/%.o are always generated
even if equivalent ones are already created in lib/.

It more clearly guarantees that lib/ and programs/ can use different compilation directives
without mixing resulting %.o object files.
2020-10-20 18:16:11 -07:00