Commit Graph

265 Commits (2485f88bf8104e52a2ef9b50d00db35960daa270)

Author SHA1 Message Date
cyan4973 21fdf97e00 Merge branch 'dev' into fullbench 2017-07-01 07:01:08 -07:00
Yann Collet d5c046c609 implemented shortcut for zstd_compress_generic() in MT mode
added ZSTDMT_compress_advanced() API
2017-06-30 14:51:01 -07:00
Yann Collet 7f40bb1c39 Merge pull request #742 from stellamplau/stack-space
Reduce stack usage of HUF_readDTableX4 and HUF_readDTableX2
2017-06-30 14:50:23 -07:00
Stella Lau 32df49e9f8 Fix typo 2017-06-30 12:56:24 -07:00
Stella Lau b0513b519c Add comment to HUF_DECOMPRESS_WORKSPACE_SIZE 2017-06-30 12:53:56 -07:00
Yann Collet a3d9926c40 compression optimization opportunity
switch to single-pass mode directly into output buffer
when outputSize >= ZSTD_compressBound(inputSize).
Speed gains observed with fullbench (~+15% on level 1)
2017-06-29 14:44:49 -07:00
Stella Lau 99e315999c Reduce stack usage of HUF_readDTableX4 and HUF_readDTableX2 2017-06-29 11:49:59 -07:00
Yann Collet 33a6639039 fixed ZSTD_refPrefix with Multithread-enabled CCtx 2017-06-28 11:09:43 -07:00
Yann Collet 7d3816183f exposed ZSTD_MAGIC_DICTIONARY in zstd.h
makes it easier to explain ZSTD_dictMode
2017-06-27 13:50:34 -07:00
Yann Collet 7bd1a2900e added ZSTD_dictMode_e to control dictionary loading mode 2017-06-21 11:50:33 -07:00
Yann Collet c3bce24ef4 fixed potential dangling pointer, detected by @terrelln 2017-06-20 16:09:11 -07:00
Yann Collet 559ee82e90 fixed : calling ZSTD_compress_generic() to end-flush a stream in multiple steps 2017-06-16 11:58:21 -07:00
cyan4973 c59162e053 minor fix for -Wdocumentation 2017-06-05 00:12:13 -07:00
cyan4973 8bcbf42617 fixed g++ prototype mismatch 2017-06-04 23:52:00 -07:00
Yann Collet 8c910d2097 updated ZSTDMT streaming API
ZSTDMT streaming API is now similar
and has same capabilites as single-thread streaming API.
It makes it easier to blend them together.
2017-06-03 01:15:02 -07:00
Yann Collet 58e8d793e1 made debug definitions common within zstd_internal.h 2017-06-02 18:20:48 -07:00
Yann Collet 4effccbf56 zlib_wrapper's uncompress() uses ZSTD_isFrame() for routing
more generic and safer than using own routing for magic number comparison
2017-06-02 14:27:11 -07:00
Yann Collet dcb7535352 ensure zlibwrapper uses ZSTD_malloc() and ZSTD_free()
which is compatible with { NULL, NULL, NULL }
2017-06-02 14:01:21 -07:00
Yann Collet 6056e4c3eb added POOL_sizeof() for single-thread 2017-06-02 11:36:47 -07:00
Yann Collet c4a5a21c5c created ZSTDMT_sizeof_CCtx() and POOL_sizeof()
required by ZSTD_sizeofCCtx() while adding a ZSTDMT_CCtx*
2017-06-01 17:56:14 -07:00
Yann Collet 9a691e0f55 fixed visual warnings 2017-05-31 01:17:44 -07:00
Yann Collet ae728a43b8 removed defaultCustomMem
now ZSTD_customCMem is promoted as new default.

Advantages : ZSTD_customCMem = { NULL, NULL, NULL},
so it's natural default after a memset.

ZSTD_customCMem is public constant
(defaultCustomMem was private only).

Also : makes it possible to introduce ZSTD_calloc(),
which can now default to stdlib's calloc()
when it detects system default.

Fixed zlibwrapper which depended on defaultCustomMem.
2017-05-30 17:11:39 -07:00
Yann Collet 44e45e8423 added ZSTDMT_createCCtx_advanced()
make it possible to use custom allocators
2017-05-30 16:12:06 -07:00
Yann Collet b81f19ffce move MEM_readMINMATCH() into zstd_opt.h
which is its only user.
Use case too narrow to belong to mem.h.
renamed to ZSTD_readMINMATCH()
2017-05-23 15:41:55 -07:00
Nick Terrell a1280406b0 [libzstd] Allow users to define custom visibility 2017-05-19 18:01:59 -07:00
Yann Collet bfff8999c5 added prototype ZSTD_versionString() 2017-05-16 16:12:23 -07:00
Jos Collin 05286fdd5a lib/common: warning: this statement may fall through
The following warning appears during the build. Fixed the review comments too.

zstd/lib/common/bitstream.h: In function ‘BIT_initDStream’:
zstd/lib/common/bitstream.h:277:33: warning: this statement may fall through [-Wimplicit-fallthrough=]
      case 7: bitD->bitContainer += (size_t)(((const BYTE*)(srcBuffer))[6]) <<
      (sizeof(bitD->bitContainer)*8 - 16);

Signed-off-by: Jos Collin <jcollin@redhat.com>
2017-05-11 09:10:02 +05:30
Nick Terrell 5f2c7213c7 Merge remote-tracking branch 'upstream/dev' into btopt
* upstream/dev: (305 commits)
  added test for ZSTD_estimateCStreamSize()
  changed variable name, for clarity
  fixed ZSTD_estimateCStreamSize()
  shortened ZSTD_createCStream_Advanced()
  fixed symbols test
  added ZSTD_estimateDStreamSize()
  changed name frameParams into frameHeader
  regroup memory usage function declarations
  separated ZSTD_estimateCStreamSize() from ZSTD_estimateCCtxSize()
  bumped version number
  added ZSTD_estimateCDictSize() and ZSTD_estimateDDictSize()
  Updated ZSTD_freeCCtx()
  updated ZSTD_estimateCCtxSize()
  Updated ZSTD_sizeof_CCtx()
  merged CCtx and CStream as a single same object
  cli : -d and -t do not stop after a failed decompression
  added dev branch CircleCI badge
  added dev branch Appveyor badge
  keep dev branch status only
  creates a binary archive without the `programs` directory
  ...
2017-05-10 16:49:58 -07:00
Yann Collet d47709b6ea Merge pull request #654 from iburinoc/splittable
[RFC] Splittable Format and API
2017-05-08 13:41:56 -07:00
Yann Collet 33c38b0925 fixed const in prototype, that Visual doesn't accept 2017-05-01 11:12:30 -07:00
Yann Collet f39a6731ec sync bitstream.h from fse library 2017-05-01 09:56:03 -07:00
Yann Collet 202082f285 sync bitstream from FSE project
add assert into unsafe *_fast() variants
2017-04-28 17:00:31 -07:00
Yann Collet 68a7d3d49a added HUF_PUBLIC_API macro to huf.h
to make it possible to control symbol visibility.
Also : better separation and comments between "public" and "static" sections
2017-04-28 12:46:48 -07:00
Yann Collet d3694e6c70 removed C4204 2017-04-27 14:29:35 -07:00
Yann Collet 69a54d138a fixed compilation warning : declaration-after-statement 2017-04-27 01:11:26 -07:00
Yann Collet e42afbc6fa Comply with suggested comments by @terrelln
created FSE_CTABLE_SIZE() and FSE_DTABLE_SIZE()
2017-04-26 11:39:35 -07:00
Sean Purcell 7d37ca1d5b Merge remote-tracking branch 'origin/dev' into splittable 2017-04-21 14:18:39 -07:00
Yann Collet e6fa70a0a1 reorganized ZSTD_resetCCtx_internal()
clearer separation between variables and buffers
clearer buffers category
kept static buffers at the beginning, favoring cache locality
(it will be easier to add FSE tables there later)

This break a few assumptions that hashTable was always at the beginning.
This is fixed.
And remaining assumptions (namely that tables stand next to each other in memory)
are now tested with assert.
2017-04-20 17:28:31 -07:00
Sean Purcell 0f7bd772e6 Update seekable API to simplify IO 2017-04-18 16:48:30 -07:00
Yann Collet 4f818182b8 clarified frame parameters for ZSTD_compress*_usingCDict()
created ZSTD_compressBegin_usingCDict_internal(),
which gives direct control to frame Parameters.
ZSTD_resetCStream_internal() now points into it.
2017-04-17 18:29:06 -07:00
Yann Collet c47c68f6ca proper evaluation of Huffman CTable size 2017-04-17 16:14:21 -07:00
Sean Purcell 5ee1135f30 s/chunk/frame/ 2017-04-12 11:15:50 -07:00
Yann Collet 20d5e03893 content size is controlled at bufferless level
so it's active for all entry points

Also : added relevant test (wrong content size) in fuzzer
2017-04-11 18:34:02 -07:00
Sean Purcell d048fefef7 Move seekable format content to /contrib 2017-04-11 14:38:56 -07:00
Sean Purcell c3ba15e48f Seekable compression demo 2017-04-11 13:53:09 -07:00
Nick Terrell 16a739cab0 Switch call of FSE_count() to FSE_count_wksp() 2017-04-04 16:17:21 -07:00
Yann Collet 274f59919d Changed memory strategy to __packed for gcc
Method 1 __packed is always as good or better than memcpy().
But it's not portable, as it depends on compiler extension.

For gcc, __pakced directive works fine.
Furthermore, gcc has serious performance issues with memcpy() on ARM 32 bits.
See #620
2017-03-30 12:52:14 -07:00
Nick Terrell 5152fb2cb2 Convert all tabs to spaces 2017-03-29 18:51:58 -07:00
Yann Collet f332ece468 dictBuilder fails to create dictionary on certain input
Properly expressed with an error code (see zstd_errors.h)
and a cli return code != 0
2017-03-23 16:24:02 -07:00
Sean Purcell 8fe5c6862c Fix undefined behaviour in decompressor 2017-03-10 10:17:42 -08:00