1be8660eca
Merge pull request #1062 from facebook/advancedDecompress
...
Advanced decompression API
2018-03-21 14:52:13 -07:00
a99c4a3621
Merge branch 'dev' into advancedDecompress
2018-03-21 06:08:28 -07:00
87b0cf05bd
Merge pull request #1057 from facebook/lrmSettings
...
LRM parameters
2018-03-21 05:59:39 -07:00
1292a738d8
Merge pull request #1063 from Varunram/manpage
...
Update man page variables in CMakeLists
2018-03-21 05:36:31 -07:00
0034c759b0
Update man page variables in CMakeLists
2018-03-21 14:10:42 +05:30
453d487807
fixed minor declaration warning
2018-03-20 18:03:56 -07:00
d1bf609abf
Merge pull request #1059 from terrelln/mt-ldm
...
Integrate ldm with zstdmt
2018-03-20 17:50:20 -07:00
e0cb8d19c6
fixed legacy test case
2018-03-20 17:48:22 -07:00
878728dc26
fixed several comments by @terrelln
2018-03-20 16:35:14 -07:00
e1c52faace
Merge pull request #1060 from facebook/compressImpl
...
merge bmi2 implementation of encodeSequence into zstd_compress.c
2018-03-20 16:19:42 -07:00
b18cb7e0b7
zstreamtest : slightly decreased time to build dictionary size
2018-03-20 16:18:48 -07:00
6cda8c932c
added test with ZSTD_decompress_generic() + ZSTD_DCtx_refPrefix()
...
also :
clarified stage condition to accept new parameters,
fixed initializers correspondingly.
2018-03-20 16:16:13 -07:00
2af41592ea
added a test with ZSTD_decompress_generic() + ZSTD_DCtx_loadDictionary_byReference()
2018-03-20 15:59:25 -07:00
0dadb6b70d
implemented ZSTD_DCtx_refPrefix*()
2018-03-20 15:45:56 -07:00
569b8ba4d9
implemented ZSTD_DCtx_refDDict()
2018-03-20 15:43:49 -07:00
a3b76a77ef
Quiet appveyor warnings
2018-03-20 15:34:40 -07:00
6873fec658
changed dictMore for dictContentType
...
which seems clearer to describe what the variable/argument is about.
2018-03-20 15:13:14 -07:00
31b54b6eea
updated ZSTD_initStaticDDict() prototype
...
can also specify dictContentType.
2018-03-20 14:52:02 -07:00
136b9e2392
Fix external sequence corner cases
...
* Clear external sequences when we reset the `ZSTD_CCtx`.
* Skip external sequences when a block is too small to compress.
2018-03-20 14:50:28 -07:00
353117c5d7
implemented ZSTD_DCtx_loadDictionary*()
...
this required updating ZSTD_createDDict_advanced()
to accept a dictContentType parameter (raw, full, auto).
2018-03-20 13:40:29 -07:00
451357f37f
Merge pull request #1058 from facebook/cctxParams
...
updated CCtxParams API
2018-03-20 12:36:12 -07:00
2ed5af0766
merge bmi2 implementation of encodeSequence into zstd_compress.c
2018-03-19 19:10:31 -07:00
d19f803a3b
Fix window size for 1 worker + flushing
2018-03-19 18:56:39 -07:00
24d9edbdd8
Set ldmParams to 0 when disabled
2018-03-19 18:23:54 -07:00
4b92574feb
Fix corner cases exposed by zstreamtest
2018-03-19 17:54:04 -07:00
94c77710a9
Integrate ldm with zstdmt
...
Integrate ldm into zstdmt by running it in serial and in order in the first
step of each job, in the same place as the hash gets updated. The input
buffer is sized to fit the whole LDM window and 2 full buffers of slack.
Input buffers cannot be reused until the LDM step is done with them.
After the LDM step is finished, the jobs don't actually have access to the
full window, only the overlap.
Tested on a few different multi-GB files with and without sanitizers,
and with different numbers of threads.
2018-03-19 16:29:03 -07:00
9e6ba88363
moved snap to /contrib
2018-03-19 16:15:06 -07:00
5ad6cc4780
Merge pull request #1015 from elopio/snapcraft
...
Add the packaging metadata to build the zstd snap
2018-03-19 16:14:42 -07:00
a7b532a347
added docker readme
2018-03-19 16:13:12 -07:00
b06db3b3c5
Merge pull request #1052 from Varunram/dockerfile
...
Add Dockerfile
2018-03-19 16:07:41 -07:00
e5ddfaede8
add a test for unordered parameters
2018-03-19 16:02:51 -07:00
aa4dbd09a1
Pull job/overlap log logic into common function ( #1055 )
...
Prepares for LDM integration by separating the job size and overlap logic
into helper functions.
2018-03-19 15:56:36 -07:00
c8b3d389fd
updated CCtxParams API
...
to respect naming convention :
ZSTD_CCtxParams_*()
2018-03-19 15:07:26 -07:00
6f4d0778a5
make it possible to express compression parameters in any order
2018-03-19 14:41:23 -07:00
dc6a5471eb
Merge pull request #1056 from terrelln/xxh-parallel
...
Move XXH64_update() into worker threads
2018-03-19 14:13:25 -07:00
9f6b7b6ef4
Merge pull request #1049 from facebook/contrib
...
`/contrib` projects as part of CI tests
2018-03-19 12:04:27 -07:00
2253d01b27
Move XXH64_update() into worker threads
...
* Computes the XXH hash in the worker threads.
* Workers get a sequence number and wait until ther number shows up. On
error, ensures that its sequence is finished, so future threads don't
get blocked.
* Sets up for ldm integration, which will go in the same spot.
2018-03-19 11:08:27 -07:00
9618c0c804
make it possible to specify LDM parameters in any order
2018-03-19 11:07:04 -07:00
f2562c02e4
Merge pull request #1053 from Varunram/cmakelists
...
Add missing checks to CMakeLists
2018-03-18 12:00:24 -07:00
d932fb621f
Merge pull request #1045 from terrelln/mt-single
...
Use a single buffer in zstdmt
2018-03-18 11:37:35 -07:00
90c598f089
Add missing checks to CMakeLists;closes #1023
2018-03-18 15:48:58 +05:30
7616200eaf
Add Dockerfile
...
Dockerfile initially proposed by @gyscos at #880
2018-03-18 14:53:48 +05:30
ec0959e701
Merge branch 'dev' into mt-single
2018-03-18 01:06:31 -07:00
d796a78c92
Merge pull request #1050 from terrelln/restore-loadedDictEnd
...
Restore setting loadedDictEnd
2018-03-18 01:03:26 -07:00
4af1fafeb8
Restore setting loadedDictEnd
...
Setting `loadedDictEnd` was accidently removed from `ZSTD_loadDictionaryContent()`,
which means that dictionary compression will only be able to reference the parts of
the dictionary within the window. The spec allows us to reference the entire
dictionary so long as even one byte is in the window.
`ZSTD_enforceMaxDist()` incorrectly always allowed offsets up to `loadedDictEnd`
beyond the window, even once the dictionary was out of range.
When overflow protection kicked in, the check `current > loadedDictEnd + maxDist`
is incorrect if `loadedDictEnd` isn't reset back to zero. `current` could be reset
below the value, which would incorrectly allow references beyond the window. This
bug is present in `master`, but is very hard to trigger, since it requires both
dictionaries and data which triggers overflow correction.
2018-03-16 14:54:06 -07:00
cbc71e40f6
moving LRM parameters out of experimental section
...
into "normal" range,
start pinned at 160.
2018-03-15 17:22:40 -07:00
5373e44ba7
fixed contrib/adaptive-compression
2018-03-15 17:10:15 -07:00
97816400ca
added /contrib projects to `make all`
2018-03-15 16:40:14 -07:00
355cb645bf
fixed seekable format example
2018-03-15 16:29:28 -07:00
38cbcb5f1a
removed LRM exploratory experiment
2018-03-15 16:26:08 -07:00
Yann Collet
Yann Collet
Varunram
Nick Terrell
Nick Terrell