Commit Graph

5021 Commits (b824d213cbdeeed846e4085da33ab0e47ab7d227)

Author SHA1 Message Date
Yann Collet b824d213cb fix #1115 2018-05-12 10:21:30 -07:00
Yann Collet 66b81817b5
Merge pull request #1128 from facebook/libdir
minor Makefile patch
2018-05-11 11:47:59 -07:00
Yann Collet 3193d692c2 minor patch, ensuring LIBDIR is created before installation
follow-up from #1123
2018-05-11 11:31:48 -07:00
Yann Collet 498ab7bb12
Merge pull request #1123 from baruchsiach/fix-install-pc
lib/Makefile: create include directory before headers installation
2018-05-10 10:38:19 -07:00
Yann Collet 33dfc54069
Merge pull request #1124 from terrelln/playtests-gnu
Write to /dev/random for test
2018-05-09 09:39:02 -07:00
Nick Terrell 2dde9d5aba Write to /dev/random for test 2018-05-08 11:06:01 -07:00
Baruch Siach 9a0643b633 lib/Makefile: create include directory before headers installation
Make sure that $(INCLUDEDIR) exists before copying the headers there.
Otherwise, the contest of header files is copied over
$(DESTDIR)$(INCLUDEDIR), making it a regular file.

While at it, remove $(DESTDIR)$(INCLUDEDIR) from the list of directories
to create in the install-pc target. The install-pc target does not need
this directory.
2018-05-08 20:59:44 +03:00
Yann Collet 87125c2c01
Merge pull request #1122 from facebook/fixCDictBlock
fix ZSTD_compressBlock() associated with ZSTD_CDict
2018-05-07 17:13:59 -07:00
Yann Collet ad4524d605 fix ZSTD_compressBlock() associated with CDict
reported by @let-def.

It's actually a bug in ZSTD_compressBegin_usingCDict()
which would pass a wrong pledgedSrcSize value (0 instead of ZSTD_CONTENTSIZE_UNKNOWN)
resulting in wrong window size, resulting in downsized seqStore,
resulting in segfault when writing into the seqStore later in the process.

Added a test in fuzzer to cover this use case (fails before the patch).
2018-05-07 12:54:13 -07:00
Yann Collet ef1abd3c07
Merge pull request #1120 from lamby/897904-libzstd-please-make-the-build-reproducible
Make the build reproducible
2018-05-04 12:39:40 -07:00
Chris Lamb 2dbe408a49 Make the build reproducible
Whilst working on the Reproducible Builds effort [0], we noticed
that zstd could not be built reproducibly.

This is due to the manual page encoding the number of CPUs from the
build machine and thus varies across builds.

This was originally filed in Debian as #897904 [1].

 [0] https://reproducible-builds.org/
 [1] https://bugs.debian.org/897904

Signed-off-by: Chris Lamb <lamby@debian.org>
2018-05-04 08:39:51 -07:00
Yann Collet 5ec92c4d96
Merge pull request #1118 from terrelln/playtests-typo
Fix playTests.sh typo
2018-05-03 14:26:26 -07:00
Nick Terrell 263134ce66 Fix playTests.sh typo
Thanks to @mestia for reporting and finding the bug!
Fixes #1116.
2018-05-03 13:03:08 -07:00
cyan4973 b095bffac6 ignore Windows build/test artefacts 2018-05-01 05:45:46 -07:00
Yann Collet 1fc15dfd9d
Merge pull request #1114 from pseiderer/ps-devel-001
Split library install target into pc, static, shared and include only target
2018-04-30 14:59:59 -07:00
Yann Collet 82ad249645 Clarifications of Zstandard format specification
from IETF RFC review
2018-04-30 12:36:55 -07:00
Peter Seiderer 64bfdca5b9 Split library install target into pc, static, shared and include only target
Signed-off-by: Peter Seiderer <ps.report@gmx.net>
2018-04-30 20:32:32 +02:00
Yann Collet a11f0c67bb
Merge pull request #1113 from terrelln/dict-fix
Fix parameter adjustment with dictionary
2018-04-26 10:31:25 -07:00
Nick Terrell ca77822ddf Fix parameter adjustment with dictionary
The new advanced API basically set `requestedParams = appliedParams` when
using a dictionary. This halted all parameter adjustment, which can hurt
compression ratio if, for example, the window log is small for the first
call, but the rest of the files are large.

This patch fixes the bug, and checks that the `requestedParams` don't change
in the new advanced API when using a dictionary, and generally in the fuzzer.
2018-04-25 16:32:29 -07:00
Yann Collet 12f60b8c98 clarified documentation related to refPrefix() 2018-04-25 10:17:06 -07:00
Yann Collet 449cccf6c9
Merge pull request #1112 from facebook/lz4dec
fixed decoding bogus lz4 frame
2018-04-24 16:50:54 -07:00
Yann Collet ace856a835 updated documentation of streaming compression api 2018-04-24 14:44:27 -07:00
cyan4973 62487b5e76 fixed decoding bogus lz4 frame
FIO would keep presenting data after an LZ4F decoding error
resulting in a NULL pointer dereference
when associated with older liblz4 version (< v1.8.1.2)
2018-04-23 18:50:16 -07:00
taigacon 2c3ad05812 Fix the problem that enables DYNAMIC_BMI2 macro by mistake on ARM architecture with Clang (#1110) 2018-04-23 15:41:50 -07:00
Yann Collet 9de03cd748
Merge pull request #1107 from terrelln/reset-params
Add ZSTD_CCtx_resetParameters() function
2018-04-13 13:23:47 -07:00
Nick Terrell e8c9dc5cea Fix documentation 2018-04-13 12:43:38 -07:00
Nick Terrell c0987986e5 Only reset CDict in ZSTD_CCtx_resetParameters() 2018-04-13 11:26:40 -07:00
Nick Terrell 9f76eebd17 Add ZSTD_CCtx_resetParameters() function
* Fix docs for `ZSTD_CCtx_reset()`.
* Add `ZSTD_CCtx_resetParameters()`.

Fixes #1094.
2018-04-12 16:54:07 -07:00
Nick Terrell 3c3f59e68f
Enforce pledgeSrcSize whenever known (#1106)
The test fails before the patch and passes after.

Fixes #1095.
2018-04-12 16:02:03 -07:00
Yann Collet 1f25b17c7d
Merge pull request #1105 from terrelln/get-param
Add ZSTD_CCtx(Param)?_getParameter() function
2018-04-12 13:25:57 -07:00
Nick Terrell 280a236e9e Add ZSTD_CCtx(Param)?_getParameter() function
Closes #1096.
2018-04-12 11:50:12 -07:00
Yann Collet 04212178b5 doc : clarified advanced API usage
sticky parameters only work with `ZSTD_compress_generic()`
2018-04-10 11:40:36 -07:00
Yann Collet ad5ba6cdcf updated comment on parameters that can be changed during compression 2018-04-09 17:39:07 -07:00
Yann Collet 1da629f2ad
Merge pull request #1104 from terrelln/fast-train
Allow negative compression levels in training
2018-04-09 14:16:20 -07:00
Nick Terrell 569e2abccd Allow negative compression levels in training
* Set `dictCLevel` in `zstdcli.c`.
* Only set to default level if the compression level `== 0`, not `<= 0`.
2018-04-09 12:12:03 -07:00
Yann Collet 77e373bff5
Merge pull request #1103 from kilobyte/dev
Fix riscv.
2018-04-09 11:12:51 -07:00
Adam Borowski 00d52282d0 Use -pthread rather than -lpthread.
It can have other effects, such as pulling in extra libraries.  Without it,
riscv build fails with undefined reference to `__atomic_compare_exchange_1'.
2018-04-09 01:50:49 +02:00
Yann Collet 4195b36dd7
Merge pull request #1100 from bket/stable_sort
zstd requires a stable sort.
2018-04-05 11:39:27 -07:00
Yann Collet f35b8ba9da updated ZSTD_p_chainLog description 2018-04-05 11:05:11 -07:00
Yann Collet b1e4d891cf
Merge pull request #1099 from bket/fix_typo
fix typo in programs/zstd.{1,1.md}
2018-04-05 10:30:35 -07:00
Björn Ketelaars 462aed6811 zstd requires a stable sort.
On OpenBSD qsort() is not guaranteed to be stable, their mergesort() is.
This fixes issue #1088. All the hard work has been done by @terrelln.
2018-04-05 07:59:16 +02:00
Björn Ketelaars e5ea8d272a fix typo in programs/zstd.{1,1.md}
s/nodictID/no-dictID/g
2018-04-05 06:44:46 +02:00
Yann Collet 55f67502f4
Merge pull request #1098 from terrelln/nd-mt
Only load extra table positions for CDicts
2018-04-02 15:38:20 -07:00
Nick Terrell 295ab0dbfa Only load extra table positions for CDicts
Zstdmt uses prefixes to load the overlap between segments. Loading extra
positions makes compression non-deterministic, depending on the previous
job the context was used for. Since loading extra position takes extra
time as well, only do it when creating a `ZSTD_CDict`.

Fixes #1077.
2018-04-02 14:41:30 -07:00
Yann Collet e866f0919f
Merge pull request #1089 from bket/portability
examples/Makefile assumes GNU make.
2018-04-02 14:16:16 -07:00
Yann Collet 5b616fa269
Merge pull request #1090 from bket/openbsd
Fix building zstd on OpenBSD.
2018-04-02 14:15:26 -07:00
bket daf3ab0aad Make sure that make check runs on OpenBSD. (#1091)
OpenBSD uses md5 instead of md5sum, and has no device called full.

With this patch, make check runs until #1088. With the assumption made
in the issue make check runs succesfully.
2018-04-02 14:12:18 -07:00
Yann Collet 754a821f17
Merge pull request #1080 from facebook/getFrameHeader
added ZSTD_getFrameHeader_advanced()
2018-04-01 14:27:46 -07:00
Björn Ketelaars 9d3048346d Fix building zstd on OpenBSD. 2018-03-31 10:46:20 +02:00
Björn Ketelaars b7e2c8e723 examples/Makefile assumes GNU make.
This fixes a building issue on systems that default to a different
flavour of make (as is the case with OpenBSD).
2018-03-31 10:25:12 +02:00