Commit Graph

1393 Commits (c2007388a556613f2e3ce1f810f5ff2cee4dfb9c)

Author SHA1 Message Date
Yann Collet 7cf78f1be7 Protects ZSTD_compressBegin_usingCDict() vs NULL cdict dereference
Will issue an error (GENERIC) is cdict==NULL
2017-04-04 12:38:14 -07:00
Nick Terrell 26b046a7c4 Remove unnecessary dictID store 2017-04-03 21:46:28 -07:00
Nick Terrell 39a6cc5172 Make ZSTD_compress_usingCDict() respect contentSizeFlag 2017-04-03 21:09:55 -07:00
Nick Terrell 62ecad3819 Fix ZSTD_initCStream_usingCDict() to use dictionary 2017-04-03 21:05:59 -07:00
Yann Collet 30c7698970 optimize ZSTDMT_compress() memory usage
does no longer allocate temporary buffers
when there is enough room in dstBuffer to decompress directly there.
(previous method would skip that for 1st chunk only).

Also : fix ZSTD_compressBound() for small srcSize
2017-03-31 18:27:03 -07:00
Yann Collet 3f75d52527 Changed ZSTD_compressBound()
required so that if Total = A+B
compressBound(Total) <= compressBound(A) + compressBound(B)
under condition of a minimum size for A and B

Will help for ZSTDMT_compress() memory allocation
2017-03-31 17:11:38 -07:00
Yann Collet 7b70a1969e Merge branch 'dev' into zstdmt 2017-03-31 16:22:33 -07:00
Yann Collet 53203e7c38 Merge pull request #640 from facebook/memAccess
Changed memory strategy to __packed for gcc
2017-03-31 15:49:12 -07:00
Yann Collet eea7858e2b fixed minor warnings in debug code 2017-03-30 16:47:19 -07:00
Yann Collet 34cc487d05 overlap at full windowSize for max compression level
as it provides max compression ratio
2017-03-30 16:23:22 -07:00
Yann Collet 458e955c23 improved ZSTDMT_compress()
Use a bit more threads by default.
Uses overlap segments to boost compression ratio (like the streaming variant)
2017-03-30 15:51:58 -07:00
Yann Collet 6476c51b86 Merge pull request #637 from facebook/zstdmt
Zstdmt
2017-03-30 14:18:37 -07:00
Yann Collet 274f59919d Changed memory strategy to __packed for gcc
Method 1 __packed is always as good or better than memcpy().
But it's not portable, as it depends on compiler extension.

For gcc, __pakced directive works fine.
Furthermore, gcc has serious performance issues with memcpy() on ARM 32 bits.
See #620
2017-03-30 12:52:14 -07:00
Nick Terrell 5152fb2cb2 Convert all tabs to spaces 2017-03-29 18:51:58 -07:00
Yann Collet ca5a8bbe36 re-added patch ... 2017-03-29 17:15:27 -07:00
Yann Collet 2e2e78de47 removed unnecessary restriction on minmatchLength
it's now transparently translated to nearest value when unsupported
(7->6) (3->4)
2017-03-29 16:02:47 -07:00
Yann Collet 26769d88bc Merge branch 'dev' of github.com:facebook/zstd into dev 2017-03-29 15:21:30 -07:00
Yann Collet 933ce4a1dd fix : minmatch 7 conversion
minmatch 7 now converted to minmatch 6 for strategies which do not support 7
Used to folded into "default", which applied minmatch 4
2017-03-29 14:35:38 -07:00
Sean Purcell 4708394bdd Remove extra 'F' from skippable magic mask 2017-03-29 11:46:57 -07:00
Yann Collet 4cf0093571 restored bonus rule 2017-03-26 14:51:00 -07:00
Yann Collet 69017bf253 Merge branch 'dev' into LegacyDictBuilder 2017-03-26 14:39:13 -07:00
Yann Collet 582760818f minor refactor
add const
changed if for easier to add new conditions
2017-03-26 03:04:56 -07:00
Yann Collet 858f72eeb8 fixed dictBuilder issue
dictionary loading would fail during entropy analysis
2017-03-26 02:50:00 -07:00
Yann Collet ecee9f2ef8 fixed conversion warnings 2017-03-26 00:59:14 -07:00
Yann Collet 0246d5c531 Merge pull request #630 from facebook/advancedCliCommands
changed advanced commands --maxdict= and --dictID=
2017-03-26 00:13:35 -07:00
Yann Collet 4c41d37fcc changed test for new syntax
--dictID= and --maxdict=
2017-03-24 18:36:56 -07:00
Yann Collet d41f707e88 minor improvement : remove duplicates with 1 char prefix difference 2017-03-24 17:56:45 -07:00
Yann Collet b364caf455 Merge pull request #628 from facebook/dictBuilder_limits
Ensure all limits derived from same constants
2017-03-24 17:54:42 -07:00
Yann Collet 2238870eb6 Merge pull request #625 from facebook/loadCDict
limited CDict acceptation criteria to be the same as DDict
2017-03-24 16:06:20 -07:00
Yann Collet 96aa3019b2 changed advanced commands --maxdict= and --dictID=
now works with the `=` variant, which is the recommended one.
Old variant `--dictID #` still works, for compatibility with existing scripts.
Long term objective is to remove the old variant..
2017-03-24 16:04:29 -07:00
Yann Collet 9da3b215ec Ensure all limits derived from same constants
Now uses ZDICT_DICTSIZE_MIN and ZDICT_CONTENTSIZE_MIN
from zdict.h.

Also : reduced values to 256 and 128 respectively
2017-03-24 15:02:09 -07:00
Yann Collet ebe9963cf6 Merge pull request #626 from facebook/stricterDictBuilder
dictBuilder fails to create dictionary on certain input
2017-03-24 14:27:28 -07:00
Yann Collet 16a0b10781 fixed ZSTD_loadZstdDictionary()
forgot to add the dictionary content
(tests were not failing, just compressing less).

Also : added size protections when adding dict content
since hc/bt table filling would fail if size < 8
2017-03-24 12:46:46 -07:00
Yann Collet 23776ce290 fixed ERROR_GENERIC on dstSize_tooSmall
required by users which depends on this error code to size dest buffer
2017-03-23 17:59:50 -07:00
Yann Collet f332ece468 dictBuilder fails to create dictionary on certain input
Properly expressed with an error code (see zstd_errors.h)
and a cli return code != 0
2017-03-23 16:24:02 -07:00
Yann Collet bea78e8fc2 limited CDict acceptation criteria to be the same as DDict 2017-03-23 15:46:06 -07:00
Sean Purcell 042ba122ae Change g_displayLevel to int and fix DISPLAYUPDATE flush 2017-03-23 11:21:59 -07:00
Nick Terrell eaf69b07f0 Zero pointers after freeing 2017-03-21 13:20:59 -07:00
Yann Collet f3dfcdccd1 bump version number 2017-03-21 12:18:28 -07:00
Przemyslaw Skibinski 8086d623ca updated build of Windows packages 2017-03-18 11:19:09 +01:00
Yann Collet 7e35b352c6 Merge pull request #602 from iburinoc/doc
Add functions missing from manual, and fix parameter alignment
2017-03-14 14:08:41 -07:00
Sean Purcell dec2b96536 Add functions missing from manual, and fix parameter alignment 2017-03-14 11:24:09 -07:00
Sean Purcell 9830aeeea6 Fix legacy support=0 case and accidental double include of version headers 2017-03-13 17:19:37 -07:00
Sean Purcell 120df494e9 Update builds to not support legacy v01-v03 2017-03-13 14:44:08 -07:00
Sean Purcell 334cb34edb ZSTD_LEGACY_SUPPORT defines lowest supported version 2017-03-13 14:32:30 -07:00
Sean Purcell 784082f49c Change gotoDict type to uPtrDiff 2017-03-10 10:34:45 -08:00
Sean Purcell 8fe5c6862c Fix undefined behaviour in decompressor 2017-03-10 10:17:42 -08:00
Nick Terrell e65aab8e0f Remove 'mem.h' dependency from ZSTD_WINDOWLOG_MAX 2017-03-08 15:40:13 -08:00
Yann Collet a41a4ed39a Merge pull request #594 from terrelln/bugs
Small fixes
2017-03-08 14:56:07 -08:00
Nick Terrell 81512e9ebe Avoid '#define inline /* ... */'
Take definition of `FORCE_INLINE` from `zstd_internal.h`.
2017-03-08 14:00:21 -08:00