659e9f05cf
Fix null pointer addition
2019-11-20 18:36:04 -08:00
a839d6852c
Merge pull request #1888 from senhuang42/superblocks_fixed
...
RLE test and re-enable RLE in main compression loop
2019-11-18 16:09:33 -08:00
bc3e21578d
No margin on RLE test size check
2019-11-18 16:39:16 -05:00
db8efbfe7d
Updated comment to reflect actual compression behavior
2019-11-15 16:11:14 -05:00
75c34684c0
Modified existing RLE test to take compressed size into account
2019-11-15 12:26:48 -05:00
d67742bc5d
Merge pull request #1858 from senhuang42/dictionary_header_size
...
Method to get dictionary header size
2019-11-14 09:44:07 -08:00
97b7f712f3
Change to heap allocation, remove implicit type conversion
2019-11-08 13:57:25 -05:00
e1edc554a3
Added 2 unit tests: one for sanity, one for correctnesson fixed dict
2019-11-08 13:57:25 -05:00
8c474f9845
Fix parameter selection and adjustment with srcSize == 0
2019-11-07 08:58:43 -08:00
b1ec94e63c
Fix ZSTD_f_zstd1_magicless for small data
...
* Fix `ZSTD_FRAMEHEADERSIZE_PREFIX` and `ZSTD_FRAMEHEADERSIZE_MIN` to
take a `format` parameter, so it is impossible to get the wrong size.
* Fix the places that called `ZSTD_FRAMEHEADERSIZE_PREFIX` without
taking the format into account, which is now impossible by design.
* Call `ZSTD_frameHeaderSize_internal()` with `dctx->format`.
* The added tests catch both bugs in `ZSTD_decompressFrame()`.
Fixes #1813 .
2019-10-21 21:16:17 -07:00
6323966e53
updated erroneous comments using ZSTD_dm_*
...
instead of the current ZSTD_dct_*,
reported by @nigeltao (#1822 )
2019-10-16 16:14:04 -07:00
fb77afc626
Merge pull request #1760 from bimbashrestha/extract_sequences_api
...
Adding api for extracting sequences from seqstore
2019-10-10 13:11:18 -07:00
36528b96c4
Manually moving instead of memcpy on decoder and using genBuffer()
2019-10-03 09:26:51 -07:00
b63a1e7ae5
Typo fix
2019-09-27 07:20:20 -07:00
91daee5c06
Fixing appveyor test
2019-09-26 16:21:57 -07:00
75b1286354
Fixing shortest failure
2019-09-26 16:07:34 -07:00
bb27472afc
Adding more realistic test for get sequences
2019-09-26 15:38:31 -07:00
be0bebd24e
Adding test and null check for malloc
2019-09-23 15:08:18 -07:00
f7d9b36835
Update Comment on `ZSTD_estimateCCtxSize()`
2019-09-20 14:11:29 -04:00
3cacc0a30b
Casting void pointer to ZSTD_Sequence pointer
2019-09-17 17:44:08 -07:00
5b038f128f
Merge branch 'extract_sequences_api' of https://github.com/bimbashrestha/zstd into extract_sequences_api
2019-09-16 13:35:49 -07:00
1f93be0f6d
Handling memory leak and potential side effect
2019-09-16 13:35:45 -07:00
a874435478
Merge branch 'dev' into extract_sequences_api
2019-09-16 13:29:59 -07:00
194c542598
Fix Memory Leak in Test
2019-09-11 14:25:30 -04:00
ff67c62458
Fix Compilation Error (`uint32_t` -> `size_t`)
2019-09-11 13:59:09 -04:00
5707c8a9d5
Speed Up Test a Little
2019-09-11 13:23:59 -04:00
ed4c2c60c3
Add Fuzzer Test Case for Index Reduction
2019-09-11 13:17:19 -04:00
9e7bb55e14
Addressing comments
2019-09-09 20:04:46 -07:00
5f8b0f6890
Changing api to get sequences across all blocks
2019-08-30 09:18:44 -07:00
5198347382
Merge pull request #1744 from bimbashrestha/dev
...
Generate RLE blocks in the encoder
2019-08-29 15:19:10 -07:00
e5704bbfdf
Added test for multiple blocks of zeros and fixed nit about comments
2019-08-28 08:32:34 -07:00
b81d7cc6a0
remove extraneous doubled ;s
2019-08-15 21:17:06 -04:00
0b0b83e8f3
fix test 122
...
it's an unsupported scenario.
2019-08-03 16:51:26 +02:00
efe8496755
minor test refactoring
...
just for clarity, for the currently failing unit test
2019-08-02 19:31:19 +02:00
387e20d4f0
fixed minor conversion warning in datagen
2019-08-02 18:02:54 +02:00
37f47e51a8
fixed datagen
...
to produce same content on both 32 and 64-bit platforms
by removing floating from literal table determination.
also : added checksum trace in compression control test,
so that it's easier to determine if test fails
as a consequence of compressing a different sample.
2019-08-02 17:34:53 +02:00
d1927f0b39
regenerate sample to compress
...
to reduce chances of differences between 32 and 64-bit fuzzer tests
2019-08-02 15:31:00 +02:00
5cf1b24aca
fixed strategies greedy, lazy & lazy2
...
restore dictionary compression ratio
2019-08-02 14:21:39 +02:00
2115292616
minor : fixed ptr arithmetic
...
invalid on void ptr
2019-08-01 17:12:26 +02:00
810a9cac08
added efficiency test
...
to detect gross CR variations after a patch.
Tests normal and dictionary compression.
2019-08-01 16:59:22 +02:00
98692c2838
fixed compression ratio regression when dictionary-compressing medium-size inputs at levels 1-3
2019-08-01 15:58:17 +02:00
c55d2e7ba3
Adding shrinking flag for cover and fastcover ( #1656 )
...
* Changed ERROR(GENERIC) excluding inits
* editing git ignore
* Edited init functions to size_t returns
* moved declarations earlier
* resolved issues with changes to init functions
* fixed style and an error check
* attempting to add tests that might trigger changes
* added && die to cases expecting to fail
* resolved no die on expected failed command
* fixed accel to be incorrect value
* Adding an automated shrinking option
* Fixing build
* finalizing fixes
* fix?
* Removing added comment in cover.h
* Styling fixes
* Merging with fb dev
* removing megic number for default regression
* Requested revisions
* fixing support for fast cover
* fixing casting errors
* parenthesis fix
* fixing some build nits
* resolving travis ci syntax
* might resolve all compilation issues
* removed unused variable
* remodeling the selectDict function
* fixing bad memory access
* fixing error checks
* fixed erroring check in selectDict
* fixing mixed declarations
* modify mixed declaration
* fixing nits and adding test cases
* Adding requested changes + fixed bug for error checking
* switched double comparison from != to <
* fixed declaration typing
* refactoring COVER_best_finish() and changing shrinkDict
* removing the const's
* modifying ZDICT_optimizeTrainFromBuffer_cover functions
* fixing potential bad memcpy
* fixing the error function for dict size
2019-06-27 16:26:57 -07:00
ed38b645db
fullbench: pass proper parameters in scenario 43
2019-05-29 15:26:06 -07:00
4baecdf72a
added comments to better understand enforceMaxDist()
2019-05-28 13:15:48 -07:00
a880ca239b
Spelling ( #1582 )
...
* spelling: accidentally
* spelling: across
* spelling: additionally
* spelling: addresses
* spelling: appropriate
* spelling: assumed
* spelling: available
* spelling: builder
* spelling: capacity
* spelling: compiler
* spelling: compressibility
* spelling: compressor
* spelling: compression
* spelling: contract
* spelling: convenience
* spelling: decompress
* spelling: description
* spelling: deflate
* spelling: deterministically
* spelling: dictionary
* spelling: display
* spelling: eliminate
* spelling: preemptively
* spelling: exclude
* spelling: failure
* spelling: independence
* spelling: independent
* spelling: intentionally
* spelling: matching
* spelling: maximum
* spelling: meaning
* spelling: mishandled
* spelling: memory
* spelling: occasionally
* spelling: occurrence
* spelling: official
* spelling: offsets
* spelling: original
* spelling: output
* spelling: overflow
* spelling: overridden
* spelling: parameter
* spelling: performance
* spelling: probability
* spelling: receives
* spelling: redundant
* spelling: recompression
* spelling: resources
* spelling: sanity
* spelling: segment
* spelling: series
* spelling: specified
* spelling: specify
* spelling: subtracted
* spelling: successful
* spelling: return
* spelling: translation
* spelling: update
* spelling: unrelated
* spelling: useless
* spelling: variables
* spelling: variety
* spelling: verbatim
* spelling: verification
* spelling: visited
* spelling: warming
* spelling: workers
* spelling: with
2019-04-12 11:18:11 -07:00
8ac2831f3d
Merge pull request #1581 from facebook/benchfn
...
benchfn's reduced dependencies
2019-04-11 14:23:04 -07:00
50b9c41196
[libzstd] Fix decompression dictionary bugs and clean up initialization
...
Bugs:
* `ZSTD_DCtx_refPrefix()` didn't clear the dictionary after the first
use. Fix and add a test case.
* `ZSTD_DCtx_reset()` always cleared the dictionary. Fix and add a test
case.
* After calling `ZSTD_resetDStream()` you could no longer load a
dictionary, since the stage was set to `zdss_loadHeader`. Fix and add
a test case.
Cleanup:
* Make `ZSTD_initDStream*()` and `ZSTD_resetDStream()` wrap the new
advanced API, and add test cases.
* Document the equivalent of these functions in the advanced API and
document the unstable functions as deprecated.
2019-04-10 12:59:02 -07:00
59a7116cc2
benchfn dependencies reduced to only timefn
...
benchfn used to rely on mem.h, and util,
which in turn relied on platform.h.
Using benchfn outside of zstd required to bring all these dependencies.
Now, dependency is reduced to timefn only.
This required to create a separate timefn from util,
and rewrite benchfn and timefn to no longer need mem.h.
Separating timefn from util has a wide effect accross the code base,
as usage of time functions is widespread.
A lot of build scripts had to be updated to also include timefn.
2019-04-10 12:37:03 -07:00
824aaa695f
[libzstd] Fix ZSTD_decompressDCtx() with a dictionary
...
* `ZSTD_decompressDCtx()` did not use the dictionary loaded by
`ZSTD_DCtx_loadDictionary()`.
* Add a unit test.
* A stacked diff uses `ZSTD_decompressDCtx()` in the
`dictionary_round_trip` and `dictionary_decompress` fuzzers.
2019-04-09 17:59:27 -07:00
48a6427d22
[libzstd] Fix ZSTD_compress2() for multithreaded compression
...
`ZSTD_compress2()` wouldn't wait for multithreaded compression to
finish. We didn't find this because ZSTDMT will block when it can
compress all in one go, but it can't do that if it doesn't have enough
output space, or if `ZSTD_c_rsyncable` is enabled.
Since we will already sometimes block when using `ZSTD_e_end`, I've
changed `ZSTD_e_end` and `ZSTD_e_flush` to guarantee maximum forward
progress. This simplifies the API, and helps users avoid the easy bug
that was made in `ZSTD_compress2()`
* Found by the libfuzzer fuzzers.
* Added a test case that catches the problem.
* I will make the fuzzers sometimes allocate less than
`ZSTD_compressBound()` output space.
2019-04-09 16:24:17 -07:00
Nick Terrell
Nick Terrell
Sen Huang
Yann Collet
Yann Collet
Bimba Shrestha
W. Felix Handte
Bimba Shrestha
Ed Maste
Tyler-Tran
Josh Soref