Commit Graph

6428 Commits (4765929271fbd3a2e60b3558225d0b7fe6527095)

Author SHA1 Message Date
Nick Terrell 7186a50775
Merge pull request #1559 from shakeelrao/reject-dict
[CLI] ensure dictionary and input file are different
2019-03-25 15:06:58 -07:00
shakeelrao 44f77b5c71 Add whitespace to test case 2019-03-24 03:42:11 -07:00
shakeelrao b25d7eacf2 Rename test 2019-03-24 03:40:03 -07:00
shakeelrao 2b4491d81a Add CLI test to validate error 2019-03-24 00:47:13 -07:00
shakeelrao 5333e41ab3 Add NULL check for dict 2019-03-24 00:23:50 -07:00
shakeelrao 8ea219d8c6 Modify error msg 2019-03-23 21:59:30 -07:00
shakeelrao 1290933d19 Implement file check 2019-03-23 21:53:13 -07:00
shakeelrao e5811e5520 Extract file comparison into utility func 2019-03-23 19:04:56 -07:00
Nick Terrell f5cbee988b [examples] Update streaming_compression to the new API 2019-03-23 15:59:26 -07:00
Nick Terrell d97605ad85
Merge pull request #1558 from nehaljwani/fix-version-soversion-libzstd
[libzstd] Specify soversion and version corectly for CMake build
2019-03-23 13:32:39 -07:00
Nehal J Wani 7ac2052dbc
[libzstd] Specify soversion and version correctly for CMake build
Fixes #1512
2019-03-23 17:37:37 +05:30
Nick Terrell d0f5ba36fb [cover] Improvements for small or homogeneous data
* The algorithm would bail as soon as it found one epoch that
  contained no new segments. Change it so it now has to fail
  >= 10 times in a row (10 for fastcover, 10-100 for cover).
* The algorithm uses the `maxDict` size to decide the epoch size.
  When this size is absurdly large, it causes tiny epochs. Lower
  bound the epoch size at 10x the segment size, and warn the user
  that their training set is too small.

Fixes #1554
2019-03-22 14:14:46 -07:00
Nick Terrell 0c7668cd06
Merge pull request #1555 from terrelln/load-dict
[lib] Allow ZSTD_CCtx_loadDictionary() to be called before parameters are set
2019-03-21 17:52:57 -07:00
Nick Terrell 6b053b9f60 [lib] Allow ZSTD_CCtx_loadDictionary() to be called before parameters are set
* After loading a dictionary only create the cdict once we've started the
  compression job. This allows the user to pass the dictionary before they
  set other settings, and is in line with the rest of the API.
* Add tests that mix the 3 dictionary loading APIs.
* Add extra tests for `ZSTD_CCtx_loadDictionary()`.
* The first 2 tests added fail before this patch.
* Run the regression test suite.
2019-03-21 16:13:53 -07:00
Nick Terrell 041bc0bdb7
Merge pull request #1551 from terrelln/stream-wrap
Refactor old stream API to completely wrap the advanced API
2019-03-21 16:10:00 -07:00
Nick Terrell 20f9ff7e53 Update documentation to tell how to replace the old streaming API with the new one. 2019-03-21 16:08:58 -07:00
Nick Terrell e55da9e963 Wrap the new advanced api completely 2019-03-21 10:54:40 -07:00
Nick Terrell 11e73576bb [regression] Add more streaming tests
* Test all of the `ZSTD_initCStream*()` variants.
* Fix a typo in the zstdcli method.
2019-03-21 10:54:18 -07:00
Nick Terrell 0dd3588acc
Merge pull request #1553 from shakeelrao/legacy-bound
Add legacy support to decompressBound
2019-03-19 10:53:12 -07:00
shakeelrao 186ded6d91 Fix typo in legacy documentation 2019-03-19 01:44:08 -07:00
shakeelrao 5740eb6769 Remove extraneous spacing in comments 2019-03-18 21:05:35 -07:00
shakeelrao 0a3fa6f909 Add legacy mode in documentation 2019-03-18 20:33:15 -07:00
shakeelrao 20aa1b455c Stylistic changes 2019-03-17 19:35:43 -07:00
shakeelrao 0033bb4785 Update documentation for ZSTD_frameSizeInfo 2019-03-17 17:41:27 -07:00
shakeelrao 19b75b6ecb Test new ZSTD_findFrameCompressedSize and update documentation 2019-03-15 18:04:19 -07:00
shakeelrao 8cd423a659 Reorder declaration in ZSTD_findFrameSizeInfoLegacy 2019-03-15 16:20:34 -07:00
shakeelrao 60796e76b0 Add legacy support to decompressBound 2019-03-15 16:10:37 -07:00
Nick Terrell f52a7d8faa
Merge pull request #1547 from shakeelrao/fix-error
Fix incorrect error code in ZSTD_errorFrameSizeInfo
2019-03-15 10:57:49 -07:00
shakeelrao 4c0540da1c Add static linking to legacy tests 2019-03-15 05:13:55 -07:00
shakeelrao 91ffc8d256 Add test to validate patch 2019-03-15 03:59:03 -07:00
Nick Terrell 45139e9fd5
Merge pull request #1550 from terrelln/cparams-cdict
[libzstd] Allow compression parameters to be set with a cdict
2019-03-13 19:17:22 -07:00
Nick Terrell 18fbcddd0c [zstreamtest] Remove outdated test 2019-03-13 17:01:23 -07:00
Nick Terrell 787b76904a [libzstd] Allow compression parameters to be set with a cdict
The order you set parameters in the advanced API is not supposed to matter.
However, once you call `ZSTD_CCtx_refCDict()` the compression parameters
cannot be changed. Remove that restriction, and document what parameters
are used when using a CDict.

If the CCtx is in dictionary mode, then the CDict's parameters are used.
If the CCtx is not in dictionary mode, then its requested parameters are
used.
2019-03-13 16:10:05 -07:00
Nick Terrell ea3b81f02c
Merge pull request #1549 from terrelln/ref-cdict-free
[libzstd] Free local cdict when referencing cdict
2019-03-13 16:08:54 -07:00
Nick Terrell 0594e8135b [libzstd] Free local cdict when referencing cdict
We no longer care about the `cdictLocal` after calling
`ZSTD_CCtx_refCDict()`, so we should free it to save some
memory.
2019-03-13 14:54:31 -07:00
shakeelrao 18d3a97d43 Add unit test to validate the error case 2019-03-13 01:43:40 -07:00
shakeelrao 79827a179f Fix incorrectly assigned value in ZSTD_errorFrameSizeInfo
As documented in `zstd.h`, ZSTD_decompressBound returns `ZSTD_CONTENTSIZE_ERROR`
if an error occurs (not `ZSTD_CONTENTSIZE_UNKNOWN`). This is consistent with
the error checking made in ZSTD_decompressBound, particularly line 545.
2019-03-13 01:23:07 -07:00
Nick Terrell 13cc7c610e
Merge pull request #1543 from shakeelrao/dev
Provide a function to estimate decompressed size.
2019-03-04 08:00:47 -08:00
shakeelrao 9ad3f31d33 update documentation for decompressBound 2019-03-02 17:56:10 -08:00
shakeelrao 95dfd48143 update formatting 2019-03-01 23:11:15 -08:00
shakeelrao 3da3dc2f45 add missing size content test 2019-03-01 21:27:30 -08:00
shakeelrao 1e08c49f75 add stylistic changes 2019-03-01 18:29:35 -08:00
shakeelrao 2bb5eec711 update missing error case to CONTENTSIZE_ERROR 2019-03-01 00:12:16 -08:00
shakeelrao 44ae395b3e change nbBlocks to size_t for consistency 2019-03-01 00:05:59 -08:00
shakeelrao 03026c3b1d change compressedBound to ULL 2019-03-01 00:03:50 -08:00
shakeelrao 8930c3c79b implement API-level changes 2019-02-28 22:55:18 -08:00
Nick Terrell 8b7fa49590
Merge pull request #1541 from terrelln/fuzz
[fuzz] Add --enable-fuzzer for clang fuzzing
2019-02-28 11:25:27 -08:00
shakeelrao dce9a09772 initialize local vars in decompressBound 2019-02-28 03:01:21 -08:00
shakeelrao 515c506b4c switch frameBound type to ULL 2019-02-28 02:10:17 -08:00
shakeelrao d0a3f25697 change return type to ULL 2019-02-28 01:52:01 -08:00