1508 Commits

Author SHA1 Message Date
Mike Swanson
af80f6dfac [programs] Don’t try to chmod a dst file if it can’t be opened
Repairs an oversight in my last commit, thanks @Cyan4973
2019-06-09 01:52:45 -07:00
Mike Swanson
3968160a91 [programs] set chmod 600 after opening destination file
This resolves a race condition where zstd or unzstd may expose read
permissions beyond the original file allowed.  Mode 600 is used
temporarily during the compression and decompression write stage
and the new file inherits the original file’s mode at the end.

Fixes #1630
2019-06-08 21:54:02 -07:00
Yosuke Tanigawa
f46a3603c9
Update README.md
Fix minor typo in the documentation.
2019-06-05 09:11:21 -07:00
Ephraim Park
e498bb64b6 zstdcli : align output message with previous message 2019-06-04 09:42:18 -07:00
Ephraim Park
a38601f05a zstdcli : remove extra semicolon 2019-06-04 09:25:16 -07:00
Ephraim Park
ef2c836955 zstdcli : trying to fix declaration after statement 2019-06-04 09:23:49 -07:00
Ephraim Park
5fe974261b zstdcli : moving cpu load calculation from FIO_compressFilename_dstFile to FIO_compressFilename_internal 2019-06-04 09:04:35 -07:00
Ephraim Park
2841c79e87 zstdcli : fixing mixed declarations and code error 2019-06-03 17:36:09 -07:00
Ephraim Park
43d463eccb zstdcli : moving start time and cpu initialization after potential prompt 2019-06-03 17:31:14 -07:00
Ephraim Park
5029ebce2f zstdcli : exposing cpu load indicator for each file on -vv mode 2019-06-03 14:38:40 -07:00
W. Felix Handte
61025d5b7d zstdgrep: Handle -f Flag 2019-05-24 16:55:59 -04:00
Rosen Penev
23b5ee1236
fileio: Disable BACKTRACE under uClibc
uClibc does not support BACKTRACE. It also defines __GLIBC__ and minor as both 2.
This fixes compilation under such a setup.
2019-05-06 15:02:47 -07:00
cev
c4a40dbf65 set followLinks option true to cat, zcat and gzcat programs 2019-04-27 19:06:29 -04:00
myzhang1029
f837326f11
Fix #1591 - Not building on MinGW-W64
Add a static function LONG_TELL for the forth #if branch
2019-04-24 20:06:05 +08:00
Josh Soref
a880ca239b Spelling (#1582)
* spelling: accidentally

* spelling: across

* spelling: additionally

* spelling: addresses

* spelling: appropriate

* spelling: assumed

* spelling: available

* spelling: builder

* spelling: capacity

* spelling: compiler

* spelling: compressibility

* spelling: compressor

* spelling: compression

* spelling: contract

* spelling: convenience

* spelling: decompress

* spelling: description

* spelling: deflate

* spelling: deterministically

* spelling: dictionary

* spelling: display

* spelling: eliminate

* spelling: preemptively

* spelling: exclude

* spelling: failure

* spelling: independence

* spelling: independent

* spelling: intentionally

* spelling: matching

* spelling: maximum

* spelling: meaning

* spelling: mishandled

* spelling: memory

* spelling: occasionally

* spelling: occurrence

* spelling: official

* spelling: offsets

* spelling: original

* spelling: output

* spelling: overflow

* spelling: overridden

* spelling: parameter

* spelling: performance

* spelling: probability

* spelling: receives

* spelling: redundant

* spelling: recompression

* spelling: resources

* spelling: sanity

* spelling: segment

* spelling: series

* spelling: specified

* spelling: specify

* spelling: subtracted

* spelling: successful

* spelling: return

* spelling: translation

* spelling: update

* spelling: unrelated

* spelling: useless

* spelling: variables

* spelling: variety

* spelling: verbatim

* spelling: verification

* spelling: visited

* spelling: warming

* spelling: workers

* spelling: with
2019-04-12 11:18:11 -07:00
Yann Collet
1e01560b83 fixed timespec_get() initialization bug on some targets
not sure why, but msan fires an "unitialized variable" error
when time gets properly initialized by timespec_get().
Maybe in some cases, not all bytes of the structure are initialized ?
Or maybe msan fails to detect the initialization ?

Anyway, pre-initializing the variable before passing it to timespec_get() works.
2019-04-11 13:46:30 -07:00
Yann Collet
30c26ab726 fixed minor warning
unused variable when assert() turned off in fileio.c
2019-04-10 17:48:56 -07:00
Yann Collet
fbdd30d68e fixed cmake build script for test programs 2019-04-10 17:47:01 -07:00
Yann Collet
526ec646b7 alternate static assert
to circumvent Visual's C4804 warning
2019-04-10 16:05:02 -07:00
Yann Collet
9703a59121 fixed minor conversion warning 2019-04-10 15:54:55 -07:00
Yann Collet
885476fb5b FreeBSD_11 specific fix
C11 mandates the definition of timespec_get() and TIME_UTC.
However, FreeBSD11 announce C11 compliance, but does not provifr timespec_get(),
breaking link stage for benchfn.
Since it does not provide TIME_UTC either, which is also required by C11,
test this macro: this will automatically rule out FreeBSD 11 for this code path
(it will use the backup C90 path instead, based on clock_t).

The issue seeems fixed in FreeBSD 12.
2019-04-10 15:22:18 -07:00
Yann Collet
2c6b14ed22 fixed Windows header
cmake build script: added timefn
2019-04-10 14:54:13 -07:00
Yann Collet
3d346579d8 no more need for CLOCK_MONOTONIC 2019-04-10 14:16:39 -07:00
Yann Collet
36d2dfd846 moved C11 code path to timespec_get 2019-04-10 14:15:11 -07:00
Yann Collet
4765929271 fixed perror include 2019-04-10 14:04:11 -07:00
Yann Collet
70802cde6d fixed error message
using stdlib's perror()
2019-04-10 14:01:18 -07:00
Yann Collet
4b8185c7fc tried a blindfix for unix + c11 2019-04-10 13:26:27 -07:00
Yann Collet
59a7116cc2 benchfn dependencies reduced to only timefn
benchfn used to rely on mem.h, and util,
which in turn relied on platform.h.
Using benchfn outside of zstd required to bring all these dependencies.

Now, dependency is reduced to timefn only.
This required to create a separate timefn from util,
and rewrite benchfn and timefn to no longer need mem.h.

Separating timefn from util has a wide effect accross the code base,
as usage of time functions is widespread.
A lot of build scripts had to be updated to also include timefn.
2019-04-10 12:37:03 -07:00
Yann Collet
094c000904 Merge branch 'dev' into benchfn 2019-04-10 11:57:05 -07:00
Yann Collet
90c0462d63 minor presentation refactoring
and removed some // comment style
2019-04-10 10:03:06 -07:00
Ori Peleg
bdeb4786b5 Fixed -r typo 2019-04-10 13:37:41 +03:00
shakeelrao
dca73db30c fix srcSize typo and add new UTIL func to comment 2019-03-28 17:50:34 -07:00
shakeelrao
5333e41ab3 Add NULL check for dict 2019-03-24 00:23:50 -07:00
shakeelrao
8ea219d8c6 Modify error msg 2019-03-23 21:59:30 -07:00
shakeelrao
1290933d19 Implement file check 2019-03-23 21:53:13 -07:00
shakeelrao
e5811e5520 Extract file comparison into utility func 2019-03-23 19:04:56 -07:00
Nick Terrell
19ca3fbc03 [zstdcli] Respect --[no-]compress-literals in benchmark mode 2019-02-15 16:27:39 -08:00
Nick Terrell
0c53c5ad4a [zstdcli] Add a flag to control literals compression 2019-02-15 15:00:45 -08:00
Nick Terrell
e1a799e5de [util] Fix UTIL_isLink() for FreeBSD 2019-02-11 17:44:10 -08:00
Björn Ketelaars
482b84f07b Make detection of symbolic links more consistent
While fixing the detection of symbolic links on OpenBSD I noticed
inconsistent behaviour:

$ echo hello > hello
$ ln -s hello world
$ zstd hello world
Warning : world is a symbolic link, ignoring
hello                :316.67%   (     6 =>     19 bytes, hello.zst
$ ls *.zst
hello.zst
$ zstd world
world                :316.67%   (     6 =>     19 bytes, world.zst)
$ ls *.zst
hello.zst world.zst
2019-02-12 01:56:05 +01:00
Björn Ketelaars
1e4dc2e5f1 Detect symbolic links on OpenBSD
In #1520 it is described that FreeBSD doesn't detect symbolic links. The
same is true for OpenBSD. This diff fixes this issue for OpenBSD. I'm
guessing that something similar works for FreeBSD as well. However, I'm
unable to test this.
2019-02-12 01:56:05 +01:00
Peter (Stig) Edwards
1951c1b3a0
-Wformat-security not needed with -Wformat=2 2019-02-01 09:31:46 +00:00
Yann Collet
fb756156f5 fixed static size of benchFnState
added a static assert to ensure condition is respected on target platform
2019-01-30 17:59:07 -08:00
Yann Collet
b8701102e0 fixed benchzstd to use new version of benchfn
returning a double type
2019-01-25 15:11:50 -08:00
Yann Collet
8c3ba44ada Merge branch 'fileio_lz4' into benchfn 2019-01-25 14:53:56 -08:00
Yann Collet
07e047873e fixed fileio.c compilation with LZ4 enabled
was broken by #1505.

I'm surprised it passed CI tests.
LZ4 tests are part of the "Extended" tests on Travis CI,
which are run on "master" and in "cron" jobs.

Since latest cron job did not failed,
especially this one : https://travis-ci.org/facebook/zstd/jobs/484365040
it suggests cron jobs are no longer using `dev` branch.

To be investigated
2019-01-25 14:42:44 -08:00
Yann Collet
f75ad2edcf added ability to create timedFnState on stack 2019-01-25 14:22:25 -08:00
Yann Collet
6b2f26791e updated benchfn.h code comments 2019-01-25 12:18:33 -08:00
Karl Ostmo
5e220bf4b5 Remove global parameters, pass into public functions instead 2019-01-23 16:06:27 -08:00
Karl Ostmo
4fa585aee3 fix --list on truncated files
fseek() doesn't indicate when it moves past the end of a file.
Consequently, if a file is truncated within its last block, the error would't be detected.

This PR adds a test scenario that induces this situation using a small compressed file of only one block in size.
This test is added to tests/playTests.sh

Check is implemented by ensuring that the filehandle position is equal to the filesize upon exit.
2019-01-15 19:03:29 -08:00