Yann Collet
be82a0ab8f
Merge pull request #2746 from eli-schwartz/meson-fixup
...
meson fixups
2021-08-23 15:57:47 -07:00
Yann Collet
18a20b3ad7
Merge pull request #2752 from facebook/hashLog3max
...
make ZSTD_HASHLOG3_MAX private
2021-08-20 12:51:17 -07:00
Yann Collet
2de42174bb
make ZSTD_HASHLOG3_MAX private
...
This is an implementation detail,
it doesn't belong to public space (zstd.h).
2021-08-20 09:52:42 -07:00
sen
ae998544de
Merge pull request #2750 from senhuang42/sb_compress
...
Improve branch misses on FSE symbol spreading
2021-08-20 12:47:24 -04:00
senhuang42
da095ed899
Improve branch misses on FSE symbol spreading
2021-08-18 10:22:22 -07:00
Eli Schwartz
9748608aeb
meson: set the symbol visibility of the shared library to hidden
...
This matches the Makefile build. Due to one private xxhash symbol in use
by the program, it recompiles a private copy of xxhash.
Due to the test binaries making extensive (?) use of private symbols, it
doesn't even attempt to link to shared libzstd, and instead, all of the
original object files are added to libtestcommon itself for private
linkage. This, too, matches the Makefile build.
Ref. #2261
2021-08-09 22:03:17 -04:00
Eli Schwartz
d95a3f5097
meson: fix warnings in build files
...
meson prefers that project-level options for Wall/Wextra/pedantic be
used, rather than hardcoding raw flags in add_project_arguments. If you
do the latter anyway, it raises a meson warning.
Set the default options for the project to use all this.
Also move the -Werror comment to the project default options with
appropriate format, but leave it commented out since it does not work.
2021-08-09 20:17:00 -04:00
Eli Schwartz
ee3355df30
meson: link tests with a convenience library of sources used by multiple binaries
...
This saves repeatedly compiling them, resulting in 16 fewer compile
steps, in exchange for one `ar` step.
2021-08-09 20:15:44 -04:00
sen
6a258043f5
Merge pull request #2692 from senhuang42/rebalance_clevel
...
[RFC] Rebalance compression levels
2021-08-06 12:51:31 -04:00
Sen Huang
539b3aab9b
Optimize 32-bit VecMask_next()
2021-08-04 17:14:58 -04:00
Felix Handte
ae131282f5
Merge pull request #2742 from felixhandte/set-mtime
...
Set mtime on Output Files
2021-08-04 16:44:46 -04:00
senhuang42
e411040ea1
Add 64 row entry support for lazy
2021-08-04 16:19:12 -04:00
senhuang42
31820e032c
Rebalance clevels for lazy
2021-08-04 16:18:52 -04:00
sen
f9b034086a
Merge pull request #2732 from senhuang42/huf_sort_improvement
...
[HUF] Improve Huffman sorting algorithm
2021-08-04 16:07:56 -04:00
W. Felix Handte
a8f4612eab
Remove sleep()s in Test; Replace with Artificial mtime
...
This behavior of `touch` is standardized in POSIX, so it should be available.
2021-08-04 15:56:46 -04:00
W. Felix Handte
9cd6c1ff4d
Update mtime and atime for Written Files
2021-08-04 14:49:56 -04:00
W. Felix Handte
a719edbbc3
Pull utime() Call into Helper
2021-08-04 14:49:00 -04:00
senhuang42
aa1957477b
Improve Huffman sorting algorithm
2021-08-04 12:43:34 -04:00
W. Felix Handte
ead41bcb4e
Add Test to Verify mtime is Copied to Destination
2021-08-03 17:22:58 -04:00
Nick Terrell
6ee70bae46
Merge pull request #2733 from terrelln/huf-cspeed
...
[HUF] Improve Huffman encoding speed
2021-08-03 12:59:54 -04:00
Nick Terrell
d8a0797268
[fuzz] Add Huffman round trip fuzzer
...
* Add a Huffman round trip fuzzer
* Fix two minor bugs in Huffman that aren't exposed in zstd
- Incorrect weight comparison (weights are allowed to be equal to
table log).
- HUF_compress1X_usingCTable_internal() can return compressed
size >= source size, so the assert that `cSize <= 65535` isn't
correct, and it needs to be checked instead.
2021-08-03 08:10:06 -07:00
sen
5c46f62006
Merge pull request #2677 from senhuang42/ci_overhaul_2
...
[CI][2/2] Migrate CI tests which (currently) fail
2021-08-02 09:55:49 -04:00
Sen Huang
5ec7897a26
Fix static analyzer warnings
2021-07-29 09:11:12 -07:00
Sen Huang
43aad811db
Add VS2019 test, migrate minimal decompressor macros to GHActions
2021-07-29 08:49:36 -07:00
Nick Terrell
32cfe15edc
Merge pull request #2724 from jonringer/fix-pzstd-static-build
...
pzstd: fix linking for static builds
2021-07-28 14:53:01 -07:00
Nick Terrell
d69e007cca
Merge pull request #2707 from koalabearguo/dev
...
Z_PREFIX zError function
2021-07-28 14:52:15 -07:00
Felix Handte
495597699c
Merge pull request #2729 from felixhandte/fix-ddss-load
...
Fix DDSS Load
2021-07-28 13:12:50 -04:00
Nick Terrell
46f2710562
[HUF] Improve Huffman encoding speed
...
Improve Huffman encoding speed by 20% for gcc and 10% for clang.
| Compiler | Benchmark | Config | Dataset | Ratio | Speed MB/s (dev) | Speed MB/s (huf-cspeed) | Speed MB/s (huf-cspeed - dev) |
|----------|-------------------|---------|-------------|-------|------------------|-------------------------|-------------------------------|
| gcc | compress | level_1 | enwik7 | 2.43 | 253.70 | 258.72 | 2.0% |
| gcc | compress | level_1 | silesia | 2.88 | 341.90 | 348.15 | 1.8% |
| gcc | compress_literals | level_1 | enwik7 | 1.49 | 761.83 | 912.76 | 19.8% |
| gcc | compress_literals | level_1 | silesia | 1.28 | 754.83 | 902.37 | 19.5% |
| gcc | compress_literals | level_7 | enwik7 | 1.29 | 502.81 | 552.79 | 9.9% |
| gcc | compress_literals | level_7 | silesia | 1.11 | 675.97 | 776.44 | 14.9% |
| clang | compress | level_1 | enwik7 | 2.43 | 277.54 | 280.98 | 1.2% |
| clang | compress | level_1 | silesia | 2.88 | 369.98 | 375.46 | 1.5% |
| clang | compress_literals | level_1 | enwik7 | 1.49 | 828.83 | 918.41 | 10.8% |
| clang | compress_literals | level_1 | silesia | 1.28 | 815.81 | 905.41 | 11.0% |
| clang | compress_literals | level_7 | enwik7 | 1.29 | 533.13 | 553.30 | 3.8% |
| clang | compress_literals | level_7 | silesia | 1.11 | 714.52 | 775.38 | 8.5% |
2021-07-27 15:10:35 -07:00
W. Felix Handte
da58821ff2
Fix DDSS Load
...
This PR fixes an incorrect comparison in figuring out `minChain` in
`ZSTD_dedicatedDictSearch_lazy_loadDictionary()`. This incorrect comparison
had been masked by the fact that `idx` was always 1, until @terrelln changed
that in #2726 .
Credit-to: OSS-Fuzz
2021-07-27 11:49:44 -04:00
Nick Terrell
d2b5e5496d
Merge pull request #2726 from terrelln/oss-fuzz-fix
...
[bug-fix] Fix a determinism bug with the DUBT
2021-07-21 13:46:21 -07:00
Nick Terrell
ba044bd6f1
[bug-fix] Fix a determinism bug with the DUBT
...
The DUBT can be non-deterministic if an index is equal to
`ZSTD_DUBT_UNSORTED_MARK`. Ensure that never happens by starting the
indices at 2.
This bug was found by the OSS-Fuzz determinism fuzzer. With this change
the fuzzer test passes. And I've confirmed that this is the root cause,
not just hiding the problem.
Aside: This took me a long time to figure out, because I thought I had
tried this first thing. But, apparantly I messed it up, because when I
was going through it again with @felixhandte, I was pointing out that it
wasn't the case, but it turns out it was.
Credit to: OSS-Fuzz
2021-07-15 13:02:49 -07:00
Yann Collet
b18febe52c
Merge pull request #2725 from makise-homura/mcst-lcc-support
...
Add support for MCST LCC compiler
2021-07-15 00:09:12 +02:00
makise-homura
3cd085cec3
Clarify no-tree-vectorize usage for ICC and LCC
2021-07-14 20:00:44 +03:00
makise-homura
a5f518ae27
Change zstdcli's main() declaration due to -Wmain on some compilers
2021-07-14 19:55:47 +03:00
makise-homura
d4ad02c721
Add support for MCST LCC compiler
2021-07-10 03:57:06 +03:00
Jonathan Ringer
e1f85dbca3
pzstd: fix linking for static builds
2021-07-09 13:21:15 -07:00
binhdvo
b3e372c171
Merge pull request #2717 from binhdvo/bootcamp
...
Proactively skip huffman compression based on sampling where non-comp…
2021-07-01 10:39:58 -04:00
sen
d90bc0e0b6
Merge pull request #2720 from senhuang42/remove_folder
...
Remove folder when done with test
2021-06-30 14:37:21 -04:00
senhuang42
f5f6cc2e48
Remove folder when done with test
2021-06-30 13:09:00 -04:00
Binh Vo
dc5b693f1e
Proactively skip huffman compression based on sampling where non-compressibility is suspected
2021-06-30 11:02:47 -04:00
Nick Terrell
7b43e69b45
Merge pull request #2655 from heitbaum/patch-1
...
[trace] remove zstd_trace.c reference from freestanding
2021-06-29 18:33:40 -07:00
Nick Terrell
609be382ac
Merge pull request #2719 from danlark1/danlark_iwyu
...
Include what you use in zstd_ldm_geartab
2021-06-29 16:53:10 -07:00
Nick Terrell
094b26081f
Merge pull request #2689 from danlark1/dev
...
Optimize zstd decompression by another x%
2021-06-29 11:34:36 -07:00
Danila Kutenin
e855b78be6
Include what you use in zstd_ldm_geartab
2021-06-29 17:57:53 +01:00
Danila Kutenin
2c2c9e7dfd
Add possible improvements for gcc-11
2021-06-29 09:06:47 +01:00
Yann Collet
88f107b5b9
Merge pull request #2716 from facebook/fix2701
...
remove invalid test
2021-06-24 10:57:18 -07:00
sen
45d707e908
Merge pull request #2715 from senhuang42/sequence_api_3
...
[RFC] Add internal API for converting ZSTD_Sequence into seqStore
2021-06-24 13:02:11 -04:00
Yann Collet
139a2f1e0c
remove invalid test
...
`--mt` is no longer supported by `zstreamtest`
(relevant API entry point has been removed from `libzstd`).
fix #2701
2021-06-24 09:07:55 -07:00
senhuang42
76466dfadf
Add simple API for converting ZSTD_Sequence into seqStore
2021-06-23 12:10:48 -04:00
Felix Handte
628f65b79c
Merge pull request #2714 from luisdallos/build-macros-typos
...
lib/Makefile: Fix small typo in ZSTD_FORCE_DECOMPRESS_* build macros
2021-06-18 16:16:44 -04:00
