Commit Graph

123 Commits (9a22140ef46195a6c7f15ce66f5fe98320ed4727)

Author SHA1 Message Date
Yann Collet 9a22140ef4 created UTIL_chmod()
protecting "/dev/null" from having its permissions changed.

also : minor : improved consistency of util.h API
2019-11-25 13:45:22 -08:00
Yann Collet 47034cd6c3
Merge pull request #1850 from bimbashrestha/named_pipes
Gating named pipe support on windows
2019-10-30 12:18:58 -07:00
Shashank Tavildar 9ab6a747d4 Created utility function to extract extension from filename, fixed tests 2019-10-29 12:27:54 -07:00
Bimba Shrestha da73b11b14
Merge branch 'dev' into named_pipes 2019-10-29 01:17:13 -04:00
Shashank Tavildar 0f2bff2faf Addressing comments, removing cyclic dependency with header file, updating tests 2019-10-28 18:21:47 -07:00
Shashank Tavildar 02433e0b15 Addressing comments: -Created a list of extensions defined in fileio.h, -Updated test 2019-10-28 14:54:54 -07:00
Shashank Tavildar 48f856640e Added --exclude-compressed flag feature that skips compression of precompressed files 2019-10-28 13:57:40 -07:00
Bimba Shrestha 5e59d51991 Merge branch 'master' of https://github.com/bimbashrestha/zstd into named_pipes 2019-10-25 15:43:41 -07:00
Bimba Shrestha 857268b32c Gating named pipe support in hedaer file 2019-10-25 15:15:28 -07:00
Yann Collet 37f90930da
Merge pull request #1839 from bimbashrestha/named_pipes
Allowing named pipes to go through in zstdcli
2019-10-24 10:53:12 -07:00
Bimba Shrestha 8a3974807e Allowing named pipes to go through zstdcli 2019-10-22 15:23:22 -07:00
Yann Collet b3e92ad346
Merge pull request #1812 from senhuang42/output-dir-flat
Support for --output-dir-flat: output into a directory
2019-10-14 09:25:05 -07:00
Sen Huang 64bc441d7d Now constructs final destination path without allocating new table 2019-10-03 13:53:04 -04:00
Sen Huang c763457e0a Static analyze fix 2019-10-02 15:30:24 -04:00
Sen Huang f80437c586 Add support for --output-dir-flat
New flag to specify output directory destination for multiple files.
2019-10-02 11:08:20 -04:00
Sen Huang 30bff50e06 fixes for tests and segfault 2019-09-06 11:10:53 -07:00
Sen Huang 7f98b46876 adding support for -O flag: multiple files into one directory for compressions (decompression to come) 2019-09-05 16:03:35 -07:00
Rosen Penev 41e90653fe
zstd: Don't use utime on Linux
utime is deprecated by POSIX 2008 and optionally not available with
uClibc-ng.

Got rid of a few useless headers in timefn.h.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2019-07-31 12:29:13 -07:00
Yann Collet 59a7116cc2 benchfn dependencies reduced to only timefn
benchfn used to rely on mem.h, and util,
which in turn relied on platform.h.
Using benchfn outside of zstd required to bring all these dependencies.

Now, dependency is reduced to timefn only.
This required to create a separate timefn from util,
and rewrite benchfn and timefn to no longer need mem.h.

Separating timefn from util has a wide effect accross the code base,
as usage of time functions is widespread.
A lot of build scripts had to be updated to also include timefn.
2019-04-10 12:37:03 -07:00
shakeelrao e5811e5520 Extract file comparison into utility func 2019-03-23 19:04:56 -07:00
Yann Collet 65a441a8f0 fixed stdlib implementation of time functions
generated redefinitions
2018-12-20 14:02:50 -08:00
Yann Collet 173ef9dea2 fixed : detection of non-existing file
better error message
with test
2018-12-19 18:30:57 -08:00
Rohit Jain 91b2fed8f8 Moving more function to the new C file 2018-10-11 17:34:47 -07:00
Rohit Jain a47f6e6f2d Removing static and making extern 2018-10-11 16:51:29 -07:00
Rohit Jain 705e0b18ab Making changes to make it compile on my laptop 2018-10-11 15:51:57 -07:00
Rohit Jain b19140bc13 Fixed makefile and removed multiple definitions from util.h (UTIL_getTime) 2018-10-11 15:17:40 -07:00
Rohit Jain d6d240f016 Removed UTIL_setFileStat from util.h and move it to util.c 2018-10-11 15:07:12 -07:00
Rohit Jain f881ee8c48 Moving code to util.c 2018-10-11 14:38:41 -07:00
Yann Collet 4a85b126d9 changed ZSTD_NANOSLEEP_SUPPORT definition
to please `-Wexpansion-to-defined`
2018-10-03 15:34:41 -07:00
Yann Collet 549c19b42e portability macro flags updates, for Haiku
some non-trivial changes to platform.h and util.h,
initially related to compilation for Haiku,
but I used this opportunity to make them cleaner
and add some documentation.

Noticed several tests that could be improved
(too harsh conditions, useless exception, etc.)
but I did not dare modifying too many tests just before release.
2018-10-03 14:56:46 -07:00
Nick Terrell 09231dad4d [util] Fix lstat feature test macro 2018-09-28 15:23:09 -07:00
Yann Collet 54001f3dbc fix mingw compatibility
only enable backtraces for platforms we know support it
aka mac OS-X and Linux.
can be extended later.
2018-09-21 14:46:09 -07:00
Yann Collet deb03d8a26 fixed UTIL_isLink() for Mac OS-X 2018-09-06 17:32:16 -07:00
Yann Collet 39c55a118f fixed minor compatibility issues with older compilers 2018-08-30 16:00:57 -07:00
Yann Collet 4086b2871b largeNbDicts compatible with multiple source files
splitting is disabled by default, but can be re-enabled using usual command -B#
update commands to look like zstd ones
2018-08-30 14:38:49 -07:00
cyan4973 f741fb8fcd minor fixes for MSYS2 compilation 2018-06-26 01:22:45 -07:00
Ryan Schmidt b567ce9d68 Fix name of macOS 2018-06-09 14:31:17 -05:00
Yann Collet d3364aa39e improve benchmark measurement for small inputs
by invoking time() once per batch, instead of once per compression / decompression.
Batch is dynamically resized so that each round lasts approximately 1 second.

Also : increases time accuracy to nanosecond
2018-02-20 14:58:40 -08:00
Nick Terrell fd63140e1c [util] Refuse to set file stat on non-regular file 2018-01-05 11:44:25 -08:00
Yann Collet 31293330d0 It's still necessary to check PLATFORM_POSIX_VERSION for clock_gettime()
glibc/uclibc is not enough
2017-12-04 16:31:59 -08:00
Yann Collet 0097469238 removed a few redundant #include 2017-12-04 16:02:42 -08:00
Yann Collet e46194bbf9 fix #911 : changed detection macro for clock_gettime()
The new macro might be a bit too restrictive.
Systems which do not support new test will simply default to <time.h>'s `clock_t clock()`,
suffering lesser benchmark accuracy.
Should it matter, the detection macro will have to be upgraded.
2017-12-04 15:57:01 -08:00
Nick Terrell dab8cfa3c7 Combine definitions of SEC_TO_MICRO 2017-11-30 19:40:53 -08:00
Nick Terrell 9a2f6f477b Use util.h for timing 2017-11-30 14:57:25 -08:00
Yann Collet d898fb7ba6 bench: added cli command `-S` to benchmark multiple files separately
Currently, all files are joined by default,
they are compressed separately but benchmarked together,
providing a single final result.

Benchmarking files separately make it possible to accurately measure difference for each file.
This is expected to be useful while tuning optimal parser.
2017-11-17 00:22:55 -08:00
Yann Collet 18b795374a UTIL_getFileSize() returns UTIL_FILESIZE_UNKNOWN on failure
UTIL_getFileSize() used to return zero on failure.
This made it impossible to distinguish a failure from a genuine empty file.
Both cases where coalesced.

Adding UTIL_FILESIZE_UNKNOWN constant has many consequences on user code,
since in many places, the `0` was assumed to mean "error".
This is no longer the case, and the error code must be actively checked.
2017-10-17 16:14:25 -07:00
Yann Collet 8f26dc3f9c blindfix for Visual
LARGE_INTEGER is not an integer :
https://msdn.microsoft.com/en-us/library/windows/desktop/aa383713(v=vs.85).aspx

Do not take any risk with the structure definition :
use int init = 0; like Mac code
2017-09-12 21:21:17 -07:00
Yann Collet bc41c7f0eb fixed minor prototype warning 2017-09-12 19:32:26 -07:00
Yann Collet c95c0c9725 modified util::time API
for easier invocation.
- no longer expose frequency timer :
it's either useless, or stored internally in a static variable (init is only necessary once).
- UTIL_getTime() provides result by function return.
2017-09-12 18:12:46 -07:00
Nick Terrell 6ab4d5e904 [bench] Use higher resolution timer on POSIX
The timer used was only accurate up to 0.01 seconds. This timer is accurate up to 1 ns.
It is a monotonic timer that measures the real time difference, not on CPU time.
2017-09-12 16:46:44 -07:00