facebook/zstd
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,534 Commits 2 Branches 0 Tags
Commit Graph

482 Commits

Author SHA1 Message Date
Julian Fessard
0ea286f740 Merge branch 'backtrace' of github.com:facebook/zstd into backtrace 2018-10-09 17:24:48 -07:00
Julian Fessard
70d8c2a031 fileio.c: Disable backtrace when built with address sanitizer 
Covers clang and gcc's sanitizer flags.
Can still be overridden through CFLAGS on commandline.
 2018-10-09 17:14:57 -07:00
Yann Collet
e0ab6b61b7 fixed explicit BACKTRACE order 
and automatic linux backtrace detection :
__GLIBC__ must be tested after #include <features.h>
 2018-10-09 17:12:21 -07:00
Yann Collet
1e0c5466c5 fixed BACKTRACE_ENABLE macro test 2018-10-09 16:36:57 -07:00
Yann Collet
f17c1df1ac backtrace support compiled with more conditions 
following #1356,
only enable backtrace compilation on linux+glibc.

Also, disable backtrace by default from "release" compilation,
so that less platforms get impacted by the new requirements.
Can be manually enabled/disabled using BACKTRACE=1/0.
 2018-10-08 17:03:06 -07:00
maxice8
1d75697d9b
don't assume __linux__ means __GLIBC__ on BACKTRACES_ENABLE 
check for __GLIBC__ instead of __linux__ as musl libc doesn't provide
execinfo.h

fixes compilation on Alpine Linux and Void Linux musl arches.
 2018-10-05 21:23:45 -03:00
Yann Collet
b1407f9acd fixed wrong assert() position 
could fire on invalid input.
blocking for afl tests.
 2018-10-03 12:43:59 -07:00
Yann Collet
3ca6261223 fixed static analyzer warnings 
note : for some reason,
scan-build version on my laptop found problems within fastcover.c
that scan-build on travisCI does not flag.

They are, as usual, false positive :
the analyzer does not understand that a table (`offset`) is correctly filled before usage.
 2018-10-02 15:59:11 -07:00
Yann Collet
9012b6cba0 ./zstd -f do no longer overwrite destination file 
if source file does not exist (#1082)
 2018-10-01 17:16:34 -07:00
Yann Collet
c7bd6a41ab zstd -d -f do no longer erase destination file 
when source file does not exist (#1082)
 2018-10-01 14:04:00 -07:00
Yann Collet
1ab71a8e72 regroup name creation logic into its own function 
for a cleaner main file decompression loop
 2018-09-28 18:19:23 -07:00
Yann Collet
ec1cb8e996 changed macro name 
from EXIT_IF() to RETURN_IF()

EXIT could be misunderstood as exit(), which terminates program execution.
But the macro only leaves the function, not the program.
 2018-09-28 16:04:00 -07:00
Yann Collet
05c0a072b7 minor improvement in the multi-format suffix selection 2018-09-28 15:57:35 -07:00
Yann Collet
d987ab5983 fixed unreachable section warning on Visual 2018-09-28 09:34:16 -07:00
Yann Collet
913a0365b6 Merge branch 'dev' into donotdelete 2018-09-27 19:23:55 -07:00
Yann Collet
ef1272737b fixed minor Visual conversion warnings 2018-09-27 18:29:15 -07:00
Yann Collet
9b45db7fa6 minor refactoring of --list 
trying to reduce recurrent patterns.
 2018-09-27 16:49:08 -07:00
Nick Terrell
f2d6db45cd [zstd] Add -Wmissing-prototypes 2018-09-27 15:24:48 -07:00
Yann Collet
6c51bf420c bounds for --adapt mode 
can supply min and max compression level through advanced command :
--adapt=min=#,max=#
 2018-09-25 16:03:28 -07:00
Yann Collet
04f47bbdd2 Merge branch 'dev' into adapt 2018-09-24 16:56:45 -07:00
Yann Collet
0250ac74ce fixed minor scan-build warnings 2018-09-24 00:52:19 -07:00
Yann Collet
00c18c0c88 simplified "slows down when compression blocked" 2018-09-21 16:35:43 -07:00
Yann Collet
c484345a82 Merge branch 'mingw' into adapt 2018-09-21 16:00:46 -07:00
Yann Collet
bfff4f4809 ensure all writes to job->cSize are mutex protected 
even when reporting errors,
using a macro for code brevity, as suggested by @terrelln,
 2018-09-21 16:00:39 -07:00
Yann Collet
54001f3dbc fix mingw compatibility 
only enable backtraces for platforms we know support it
aka mac OS-X and Linux.
can be extended later.
 2018-09-21 14:46:09 -07:00
Yann Collet
ca02ebee07 removed static variables 
so that --adapt can work on multiple input files too
 2018-09-19 15:25:50 -07:00
Yann Collet
89bc309d90 error out when --adapt is associated with --single-thread 
since they are not compatible
 2018-09-19 14:49:13 -07:00
Yann Collet
2f78228f65 Merge branch 'dev' into adapt 2018-09-19 12:43:42 -07:00
Casey McGinty
b9118ecdab
Update comments, and LD flag usage in Make 2018-09-11 14:49:47 -07:00
Casey McGinty
d4337b6f1d
Move ABRThandler func out of internal lib 2018-09-11 11:39:49 -07:00
Casey McGinty
a06574fc97
Print a stack trace on unexpected term signal (e.g. SIGABRT) 
For OSX and Linux, add a signal handler to SIGABRT, SGIFPE, SIGILL,
SIGSEGV, and SIGBUS. When the program terminates unexpectedly the
handler will print the current stack to the terminal to help determine
the location of the failure.

On OSX the output will look like:

```
Stack trace:
4   zstd                                0x000000010927ed96 main + 16886
5   libdyld.dylib                       0x00007fff767d1015 start + 1
6   ???                                 0x0000000000000001 0x0 + 1
```

On Linux the output will look like:

```
Stack trace:
./zstd() [0x4b8e1b]
./zstd() [0x4b928a]
./zstd() [0x403dc2]
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7f5e0fbb0445]
./zstd() [0x405754]
```

As is, the code does not function on WIN32.

See also: https://oroboro.com/stack-trace-on-crash/
 2018-09-06 18:46:52 -07:00
Eden Zik
78af534f82 Fixed unsafe string copy and concat in fileio.c. 
Per warnings from flawfinder: "Does not check for buffer overflows when
copying to destination [MS-banned] (CWE-120). Consider using snprintf,
strcpy_s, or strlcpy (warning: strncpy easily misused).".

Replaced called to strcpy and strcat in `fileio.c` to calls with a
specified size (`strncpy` and `strncat`).

Tested the changes on OSX, Linux, Windows.
On OSX + Linux, changes were tested with ASAN. The following flags were
used: 'check_initialization_order=1:strict_init_order=1:detect_odr_violation=1:detect_stack_use_after_return=1'

To reproduce warning:
./flawfinder.py ./programs/fileio.c
 2018-08-20 22:15:24 -04:00
Yann Collet
105677c6db created ZSTDMT_toFlushNow() 
tells in a non-blocking way if there is something ready to flush right now.
only works with multi-threading for the time being.

Useful to know if flush speed will be limited by lack of production.
 2018-08-17 18:11:54 -07:00
Yann Collet
09e63c58ac fix : no longer slow down on input saturation 
only slows down when all buffers are full
 2018-08-17 16:27:43 -07:00
Yann Collet
8b674d7dc7 ensured compression level is maxed at ZSTD_maxCLevel() 2018-08-17 16:01:56 -07:00
Yann Collet
3e4617ef54 frameProgression reports nbActiveWorkers and output flushed 2018-08-14 11:49:25 -07:00
Yann Collet
0853f86044 adaptive mode uses default window size of 8 MB 2018-08-13 13:13:22 -07:00
Yann Collet
33f7709c71 fileio: changed parameter type from ptr to plain structure 
safer : this parameter is read-only,
we don't want original structure to be modified
 2018-08-13 13:02:03 -07:00
Yann Collet
f3aa510738 rateLimiter does not "catch up" when input speed is slow 2018-08-13 11:38:55 -07:00
Yann Collet
e7a49c6683 introduced command --adapt 2018-08-11 20:48:06 -07:00
Yann Collet
9d26cb6a75 slow down faster when output speed is limited 2018-08-09 17:44:30 -07:00
Yann Collet
3d7b533f68 Merge branch 'dev' into adapt 2018-08-09 15:57:36 -07:00
Yann Collet
754942cb79 fixed assert() condition 2018-08-09 15:57:19 -07:00
Yann Collet
2dd76037be zstd cli can increase level when input is too slow 2018-08-09 15:51:30 -07:00
Yann Collet
79a35ac20d minor code comments improvements 2018-08-09 15:16:31 -07:00
Nick Terrell
4e706d7f2c fileio: Error in compression on read errors 
We can write a corrupted file if the input file errors during a read.
We should return a non-zero error code in this case.
 2018-07-17 15:26:30 -07:00
W. Felix Handte
712a9fd972 Allow Invoking zstd --list When stdin is not a tty 
Also now returns an error when no inputs are given.

New proposed behavior:

```
felix@odin:~/prog/zstd (list-stdin-check)$ ./zstd -l; echo $?
No files given
1
felix@odin:~/prog/zstd (list-stdin-check)$ ./zstd -l Makefile.zst; echo $?
Frames  Skips  Compressed  Uncompressed  Ratio  Check  Filename
     1      0     3.08 KB      10.92 KB  3.544  XXH64  Makefile.zst
0
felix@odin:~/prog/zstd (list-stdin-check)$ ./zstd -l <Makefile.zst; echo $?
zstd: --list does not support reading from standard input
No files given
1
felix@odin:~/prog/zstd (list-stdin-check)$ ./zstd -l Makefile.zst <Makefile.zst; echo $?
Frames  Skips  Compressed  Uncompressed  Ratio  Check  Filename
     1      0     3.08 KB      10.92 KB  3.544  XXH64  Makefile.zst
0
felix@odin:~/prog/zstd (list-stdin-check)$
```
 2018-06-29 15:33:44 -04:00
Yann Collet
6768cf53fd
Merge pull request #1190 from terrelln/ldm-adjust 
Adjust advanced parameters to source size
 2018-06-19 14:40:56 -07:00
Yann Collet
c0b6ce95b1
Merge pull request #1179 from supertopher/dev 
Improves UX for --list command's lack of support for pipes
 2018-06-19 14:36:30 -07:00
Nick Terrell
1d0fcde45d Use debug.h in fileio.c 2018-06-18 15:51:21 -07:00