Commit Graph

7799 Commits (8f8bd2d1ac555ac7b6f6bcfd67ba0a426f5a8309)

Author SHA1 Message Date
W. Felix Handte 87c541c5f9 Only Trigger libzstd.pc Build on Unix-Like Platforms
We don't even define the rule on unsupported platforms.
2020-05-08 16:11:32 -04:00
W. Felix Handte 85801b99d7 Test libzstd.pc Build on Travis OS X 2020-05-08 16:11:32 -04:00
W. Felix Handte 78aa9373cb Add libzstd.pc Build to More Aggregate Targets in Makefiles 2020-05-08 16:11:32 -04:00
W. Felix Handte 15561bcf74 Fix pkg-config File Generation Again Again
Resubmission of #2001. This switches the `sed` invocations to use `-E`,
extended regex syntax, which is better standardized across platforms.
I guess.

Same test plan:

```
make -C lib clean libzstd.pc
cat lib/libzstd.pc

echo # should fail
make -C lib clean libzstd.pc     LIBDIR=/foo
make -C lib clean libzstd.pc INCLUDEDIR=/foo
make -C lib clean libzstd.pc     LIBDIR=/usr/localfoo
make -C lib clean libzstd.pc INCLUDEDIR=/usr/localfoo
make -C lib clean libzstd.pc     LIBDIR=/usr/local/lib     prefix=/foo
make -C lib clean libzstd.pc INCLUDEDIR=/usr/local/include prefix=/foo

echo # should succeed
make -C lib clean libzstd.pc     LIBDIR=/usr/local/foo
make -C lib clean libzstd.pc INCLUDEDIR=/usr/local/foo
make -C lib clean libzstd.pc     LIBDIR=/usr/local/
make -C lib clean libzstd.pc INCLUDEDIR=/usr/local/
make -C lib clean libzstd.pc     LIBDIR=/usr/local
make -C lib clean libzstd.pc INCLUDEDIR=/usr/local
make -C lib clean libzstd.pc     LIBDIR=/tmp/foo prefix=/tmp
make -C lib clean libzstd.pc INCLUDEDIR=/tmp/foo prefix=/tmp
make -C lib clean libzstd.pc     LIBDIR=/tmp/foo prefix=/tmp/foo
make -C lib clean libzstd.pc INCLUDEDIR=/tmp/foo prefix=/tmp/foo

echo # should also succeed
make -C lib clean libzstd.pc prefix=/foo LIBDIR=/foo/bar INCLUDEDIR=/foo/
cat lib/libzstd.pc

mkdir out
cd out
cmake ../build/cmake
make
cat lib/libzstd.pc
```
2020-05-08 16:11:32 -04:00
Felix Handte a036f60255
Merge pull request #2078 from nocnokneo/fix-destdir-installs
CMake: Fix DESTDIR installs and the Package Config file
2020-05-08 16:09:44 -04:00
Yann Collet 4f2cb944a8 Merge branch 'dev' into clevel_doc 2020-05-08 11:59:45 -07:00
Nick Terrell 32b5a8ec41
Merge pull request #2118 from Artoria2e5/patch-1
programs/README.md: update help text
2020-05-08 11:13:44 -07:00
Yann Collet e873697625 updated documentation regarding ZSTD_CLEVEL
make it clearer that it's useful for `tar --zstd`.
2020-05-08 10:51:37 -07:00
Bimba Shrestha 602dae3509
Merge pull request #2119 from bimbashrestha/mingw
[build] Adding $TARGET_SYSTEM for cross-compilation Fedora fix
2020-05-08 10:54:06 -05:00
Bimba Shrestha df9e5b6f4c adding 2020-05-07 22:07:40 -05:00
Yann Collet efc656c9a6
Merge pull request #2114 from facebook/verbose
support for verbose make
2020-05-07 13:16:33 -07:00
Yann Collet 1afe57cff7
Merge pull request #2112 from facebook/cfast
small speed improvement for strategy fast
2020-05-07 13:13:34 -07:00
caoyzh 969ba4f2b9 Change the modification of ZSTD_wildcopy() 2020-05-07 13:10:46 -07:00
caoyzh a7e34ff693 revert ZSTD_reduceTable_internal()'s modificatiion 2020-05-07 13:10:46 -07:00
caoyzh 9e802ede9c Modify indent of comments 2020-05-07 13:10:46 -07:00
caoyzh 7f75f05e84 Change "arm_neon.h" to system include <arm_neon.h> 2020-05-07 13:10:46 -07:00
caoyzh b2e56f7f7f Optimize compression by using neon function. 2020-05-07 13:10:46 -07:00
Mingye Wang 37d10b6c75
programs/README.md: update help text
I noticed that the help is outdated as there is no mention of --adapt. Y'all should probably put "update help text" into your release checklist.
2020-05-08 02:52:18 +08:00
Nick Terrell 45c66dd298 [zdict] Stabilize ZDICT_finalizeDictionary() 2020-05-07 10:37:01 -07:00
Bimba Shrestha 625924774e
Merge pull request #2115 from bimbashrestha/docs
[doc] Adding static analysis section and note
2020-05-07 12:03:33 -05:00
Bimba Shrestha 1b1cc3d1b7 adding some static-anslysis docs 2020-05-07 10:38:06 -05:00
Yann Collet cf854f4660 support for verbose make
A commonly accepted makefile idiom is V=1 or VERBOSE=1
to request the printing of all commands.

This is not "default" though, and must be manually added.

Example :
Before :
```
make libzstd
compiling dynamic library 1.4.5
creating versioned links

make libzstd V=1
compiling dynamic library 1.4.5
creating versioned links
```

After :
```
make libzstd
compiling dynamic library 1.4.5
creating versioned links

make libzstd V=1
compiling dynamic library 1.4.5
cc -DXXH_NAMESPACE=ZSTD_ -DZSTD_LEGACY_SUPPORT=5 -Wall -Wextra -Wcast-qual -Wcast-align -Wshadow -Wstrict-aliasing=1 -Wswitch-enum -Wdeclaration-after-statement -Wstrict-prototypes -Wundef -Wpointer-arith -Wvla -Wformat=2 -Winit-self -Wfloat-equal -Wwrite-strings -Wredundant-decls -Wmissing-prototypes -Wc++-compat  -O3 common/debug.c common/entropy_common.c common/error_private.c common/fse_decompress.c common/pool.c common/threading.c common/xxhash.c common/zstd_common.c compress/fse_compress.c compress/hist.c compress/huf_compress.c compress/zstd_compress.c compress/zstd_compress_literals.c compress/zstd_compress_sequences.c compress/zstd_compress_superblock.c compress/zstd_double_fast.c compress/zstd_fast.c compress/zstd_lazy.c compress/zstd_ldm.c compress/zstd_opt.c compress/zstdmt_compress.c decompress/huf_decompress.c decompress/zstd_ddict.c decompress/zstd_decompress.c decompress/zstd_decompress_block.c deprecated/zbuff_common.c deprecated/zbuff_compress.c deprecated/zbuff_decompress.c dictBuilder/cover.c dictBuilder/divsufsort.c dictBuilder/fastcover.c dictBuilder/zdict.c legacy/zstd_v05.c legacy/zstd_v06.c legacy/zstd_v07.c -shared -fPIC -fvisibility=hidden -Wl,-soname=libzstd.so.1 -o libzstd.so.1.4.5
creating versioned links
ln -sf libzstd.so.1.4.5 libzstd.so.1
ln -sf libzstd.so.1.4.5 libzstd.so
```
2020-05-07 08:04:10 -07:00
Yann Collet 54144285fd small speed improvement for strategy fast
gcc 9.3.0 :
kennedy : 459 -> 466
silesia : 360 -> 365
enwik8  : 267 -> 269

clang 10.0.0 :
kennedy : 436 -> 441
silesia : 364 -> 366
enwik8  : 271 -> 272
2020-05-07 06:15:58 -07:00
Nick Terrell 5717bd39ee [lib] Fix NULL pointer dereference
When the output buffer is `NULL` with size 0, but the frame content size
is non-zero, we will write to the NULL pointer because our bounds check
underflowed.

This was exposed by a recent PR that allowed an empty frame into the
single-pass shortcut in streaming mode.

* Fix the bug.
* Fix another NULL dereference in zstd-v1.
* Overflow checks in 32-bit mode.
* Add a dedicated test.
* Expose the bug in the dedicated simple_decompress fuzzer.
* Switch all mallocs in fuzzers to return NULL for size=0.
* Fix a new timeout in a fuzzer.

Neither clang nor gcc show a decompression speed regression on x86-64.
On x86-32 clang is slightly positive and gcc loses 2.5% of speed.

Credit to OSS-Fuzz.
2020-05-06 12:09:02 -07:00
Felix Handte ad8dbae1b7
Merge pull request #2103 from felixhandte/relative-includes
Migrate Includes to Relative Paths
2020-05-06 09:42:23 -07:00
Bimba Shrestha e7df0d41bb
Merge pull request #2095 from bimbashrestha/grep
[bugs] zstdgrep/grep inconsistencies
2020-05-06 11:18:15 -05:00
Yann Collet d116f24070
Merge pull request #2109 from facebook/fasterCygwin
faster cygwin test
2020-05-05 20:50:40 -07:00
Bimba Shrestha 250184adf6 adding tests back 2020-05-05 16:51:06 -07:00
Yann Collet e09cdf9080
Merge pull request #2108 from facebook/hufc_wconv
some more conversion fixes …
2020-05-05 16:38:17 -07:00
Yann Collet 4f755bdafc faster cygwin test
The cygwin test on Appveyor is way too long (>30 mn).
The main issue is the very long zstreamtest, because no time out has been programmed.
Ensures that both fuzzer and zstreamtest receive a 30sec slot.
Should reduce cygwin tests duration by -30mn.
2020-05-05 14:08:35 -07:00
W. Felix Handte 5163778c02 Yay It Works, Clean Up Logging A Bit 2020-05-05 15:27:27 -04:00
W. Felix Handte 005ffd3bdb Fix Sed Regex Escaping for Extended Regex Rules...... 2020-05-05 14:54:04 -04:00
W. Felix Handte bdfb50f7c4 Uhhh It's Still Failing, Let's Try Printing Debug Info? 2020-05-05 13:58:22 -04:00
Yann Collet ef7d98e55a Merge branch 'dev' into hufc_wconv 2020-05-05 10:23:00 -07:00
W. Felix Handte 229cbd056e Fix Fallbacks to Work on FreeBSD 2020-05-05 13:18:45 -04:00
Yann Collet c29fd7cd8b some more conversion warnings
hunting down some static analyzer warnings
2020-05-05 10:16:59 -07:00
Bimba Shrestha 6ff16b4246 adding help text 2020-05-05 10:03:57 -07:00
Bimba Shrestha a06a508bf3 fixing inconsistencies 2020-05-05 09:56:15 -07:00
Yann Collet 7120d97cb3
fix minor conversion warnings (#2106) 2020-05-05 09:08:05 -07:00
W. Felix Handte 6381b7ee3f Add Fallback When realpath Doesn't Support --relative-to 2020-05-04 18:59:47 -04:00
W. Felix Handte c7cba8e87c Mark zstddeclib-in.c Not Executable 2020-05-04 18:59:26 -04:00
Yann Collet c1b836f4c3 fix minor conversion warnings 2020-05-04 14:43:09 -07:00
Felix Handte 8b327149a8
Merge pull request #1976 from felixhandte/minimal-lib-target
Add Minification Variable to `lib/Makefile`
2020-05-04 12:42:56 -07:00
W. Felix Handte b48f6c7d26 Try to Fix Single File Library Combiner Script to Handle Relative Includes 2020-05-04 15:20:26 -04:00
W. Felix Handte 7b75d772b1 Remove Useless Assignment in Makefile 2020-05-04 15:20:26 -04:00
W. Felix Handte 7dcca6bc64 Also Move programs/ Directory to Relative Includes 2020-05-04 15:20:26 -04:00
W. Felix Handte 6028827fee Rewrite Include Paths to be Relative
Addresses #1998.
2020-05-04 15:20:26 -04:00
Bimba Shrestha 0301ef5d04
[bench] Extending largeNbDicts to compression (#2089)
* adding cdict_collection_t

* adding shuffleCDictionaries()

* adding compressInstructions

* adding compress()

* integrating compression into bench()

* copy paste error fix

* static analyzer uninit value complaint fix

* changing to control

* removing assert

* changing to control

* moving memcpy to seperate function

* fixing static analyzer complaint

* another hacky solution attempt

* Copying createbuffer logic
2020-05-04 10:42:22 -07:00
Felix Handte 7e9aabd652
Merge pull request #2099 from felixhandte/compile-under-pedantic
Compile Under `-pedantic -Werror` and `-std=c90`
2020-05-04 10:07:13 -07:00
W. Felix Handte 0b81035101 Test Flag in Travis CI 2020-05-04 11:22:07 -04:00