Commit Graph

7811 Commits (5fbc6addb6a2a271d9164631f2fa81f24158a9c1)

Author SHA1 Message Date
Bimba Shrestha 1875f616ce passing dictContentType instead of rawContent every time 2020-04-21 22:29:35 -07:00
Bimba Shrestha f7a7409a49 adding fail test when passing wrong fullDict using refPrefix 2020-04-21 22:26:48 -07:00
Bimba Shrestha dba02245bf bash to shell conversion 2020-04-21 20:31:11 -07:00
Bimba Shrestha 659ff85cf5 changing to cParams.hashLog 2020-04-21 21:12:50 -05:00
Bimba Shrestha 0b107188b8 adding test for long mode trigger 2020-04-21 21:09:49 -05:00
Bimba Shrestha 38a6d2a8f2
Merge pull request #2085 from bimbashrestha/quick2
[doc-bug] Updating the recommended max chainlog for patch-from
2020-04-20 14:09:23 -05:00
Bimba Shrestha b640802ff8 updating chainlog bound 2020-04-20 11:34:24 -07:00
Bimba Shrestha c6311584aa
Merge pull request #2083 from bimbashrestha/quick1
[nit] Adding missing newline at end of patch-from recommendations
2020-04-20 13:24:34 -05:00
Bimba Shrestha 587a20a12a adding newline 2020-04-20 10:25:58 -07:00
Bimba Shrestha 5b0a452cac
Adding --long support for --patch-from (#1959)
* adding long support for patch-from

* adding refPrefix to dictionary_decompress

* adding refPrefix to dictionary_loader

* conversion nit

* triggering log mode on chainLog < fileLog and removing old threshold

* adding refPrefix to dictionary_round_trip

* adding docs

* adding enableldm + forceWindow test for dict

* separate patch-from logic into FIO_adjustParamsForPatchFromMode

* moving memLimit adjustment to outside ifdefs (need for decomp)

* removing refPrefix gate on dictionary_round_trip

* rebase on top of dev refPrefix change

* making sure refPrefx + ldm is < 1% of srcSize

* combining notes for patch-from

* moving memlimit logic inside fileio.c

* adding display for optimal parser and long mode trigger

* conversion nit

* fuzzer found heap-overflow fix

* another conversion nit

* moving FIO_adjustMemLimitForPatchFromMode outside ifndef

* making params immutable

* moving memLimit update before createDictBuffer call

* making maxSrcSize unsigned long long

* making dictSize and maxSrcSize params unsigned long long

* error on files larger than 4gb

* extend refPrefix test to include round trip

* conversion to size_t

* making sure ldm is at least 10x better

* removing break

* including zstd_compress_internal and removing redundant macros

* exposing ZSTD_cycleLog()

* using cycleLog instead of chainLog

* add some more docs about user optimizations

* formatting
2020-04-17 15:58:53 -05:00
Yann Collet 88ecdc939b
Merge pull request #2080 from facebook/minor_fileio_refactor
minor refactor of `fileio.c`
2020-04-13 20:47:34 -07:00
Yann Collet 458a1a1723 minor refactor
- fix a few comments
- reorder some parameters, to enforce "mutable references first"
- simplified fwriteSparse()
2020-04-13 14:09:57 -07:00
Taylor Braun-Jones 105d9b62ae Minor CMake refactoring/simplification 2020-04-09 17:16:12 -04:00
Taylor Braun-Jones b674a1eec3 Fix that installation of the libzstd.pc does not respect DESTDIR installations 2020-04-09 17:16:12 -04:00
Taylor Braun-Jones c22551b903 Generate a better CMake Package Config file 2020-04-09 17:16:12 -04:00
Nick Terrell 5fcbc484c8
Merge pull request #2040 from caoyzh/dev-2
Optimize by prefetching on aarch64
2020-04-08 13:14:47 -07:00
Bimba Shrestha c0d4b2b5a3
Merge pull request #2075 from bimbashrestha/dict_fuzzer_ref
[bug] handling case where prefix is NULL or 0 sized in refPrefix_advanced
2020-04-07 17:37:19 -05:00
Bimba Shrestha 1658ae75cd handling nil case for refprefix 2020-04-07 14:41:53 -07:00
Yann Collet dde98d833d
Merge pull request #2065 from cwoffenden/single-file-lib
Single-file decoder script can now (optionally) create an encoder
2020-04-07 10:17:05 -07:00
Carl Woffenden 3eaa525138 Fixed clash when projects are already using xxHash
Undefing XXH_* macros allows the `.c` to build standalone without clashes. Removing `xxhash.c` and only including the header is the correct usage (according to `XXH_PRIVATE_API`).
2020-04-07 18:17:59 +02:00
Carl Woffenden 4e6027f8ca
Fixed-up single-file libs directory 2020-04-07 13:35:19 +02:00
Carl Woffenden d01e7c2fad Renamed directory 2020-04-07 13:34:19 +02:00
Carl Woffenden 59812afac7 Improved documentation 2020-04-07 13:19:52 +02:00
Carl Woffenden a93fadfcd9 Further replication removed
`CHECK_F` is now in `error_private.h`. Minor tidy.
2020-04-07 11:25:16 +02:00
Carl Woffenden 7af7735fa3 Merge remote-tracking branch 'upstream/dev' into single-file-lib 2020-04-07 11:13:02 +02:00
Carl Woffenden edd9a07322 Code replicated in compression and decompression moved to shared headers
`CHECK_F` macro moved to `error_private.h` (shared between `fse_compress.c` and `fse_decompress.c`). `ZSTD_limitCopy()` moved to `zstd_internal.h` (shared between `zstd_compress.c` and `zstd_decompress.c`). Erroneous build artefact `zstd.h` removed from repo.
2020-04-07 11:02:06 +02:00
Bimba Shrestha 794f03459e adding refPrefix 2020-04-06 22:57:49 -07:00
Bimba Shrestha 2847d1a606
Merge pull request #2069 from bimbashrestha/doversize
[memory opt] Downsize decompression buffers when large for too long
2020-04-06 16:06:37 -05:00
Bimba Shrestha 31e76f1ed4 adding test for dctx size reduction 2020-04-04 08:49:24 -07:00
Bimba Shrestha ebc422b0b7
Merge pull request #2068 from bimbashrestha/quick
[bug] Fixing Incorrect/inconsistent reported compression percentage for empty files
2020-04-03 19:28:42 -05:00
Bimba Shrestha 0154866749 moving consts to zstd_internal and reusing them 2020-04-03 14:26:15 -07:00
Bimba Shrestha 0a172c5e43 converting to if 2020-04-03 14:21:24 -07:00
Bimba Shrestha 3a4c8cc9b3 adding dctx to function name 2020-04-03 14:14:46 -07:00
Bimba Shrestha ae47d50355 only computing sizes once 2020-04-03 14:12:23 -07:00
Bimba Shrestha a4cbe79ccb Using in and out size together 2020-04-03 14:09:21 -07:00
Bimba Shrestha 009a92f749
Merge pull request #2067 from bimbashrestha/res-leak
[bug] adding fclose before return
2020-04-03 15:51:59 -05:00
Bimba Shrestha 1d267dc5d6 returning on null check 2020-04-03 13:39:02 -07:00
Bimba Shrestha d598c88fb3 adding fclose before return 2020-04-03 13:31:47 -07:00
Bimba Shrestha 936aa63ff1 adding oversized check on decompression 2020-04-03 13:25:32 -07:00
Bimba Shrestha 05574ec141 adding oversizeDuration to dctx and macros 2020-04-03 13:08:29 -07:00
Bimba Shrestha d0412f3aba no percentage on readsize == 0 2020-04-03 12:10:02 -07:00
Bimba Shrestha bf1856c26f removing max(1, ..) 2020-04-03 12:07:17 -07:00
Carl Woffenden 88da79b42e Minor typo 2020-04-03 20:50:54 +02:00
Carl Woffenden 0442be5aa4 Added missing -pthread flag (was breaking on BSD) 2020-04-03 20:34:47 +02:00
Carl Woffenden 7c420344d2 Single-file decoder script can now (optionally) create an encoder
To complement the single-file decoder a new script was added to create an amalgamated single-file of all of the Zstd source, along with examples and (simple) tests.
2020-04-03 19:07:46 +02:00
Carl Woffenden 7202184ee0
Fixes decompressor when using -Wshorten-64-to-32 (#2062)
Spotted on iOS when building with `-Wshorten-64-to-32` (since `__builtin_expect` returns a `long`).
2020-04-03 02:55:29 -07:00
Taylor Braun-Jones 5b054d9e90 Add support for running more tests via CTest 2020-03-30 15:14:00 -04:00
Nick Terrell 1665462573
Merge pull request #2054 from terrelln/license-fix
Standardize and fix copyright and licenses
2020-03-27 11:00:01 -07:00
Nick Terrell 8fda220d70
Merge pull request #2053 from terrelln/fix-play-test
[test] Fix playTests.sh with space in binary path
2020-03-26 21:08:10 -07:00
Nick Terrell c7ae28a67d [circleci] Run test-license.py
Run it first because it is very fast (~1 second)
2020-03-26 20:13:16 -07:00