Commit Graph

7156 Commits (d9c475f3b3e7e22d8f78d2b9c501ccf875d8d81f)

Author SHA1 Message Date
Eric van Gyzen 46ee10dfb5 Include errno in all fwrite error messages in fileio.c 2019-10-10 09:21:29 -05:00
Sen Huang c5ebb37051 Only check for filename collisions when using --output-dir-flat 2019-10-09 09:39:52 -04:00
Nick Terrell f5f929fc61
Merge pull request #1820 from mhlakhani/dev
Add OSS-Fuzz fuzzing badge
2019-10-08 17:58:16 -04:00
Hasnain Lakhani 8826f3b433 Add OSS-Fuzz fuzzing badge
This is basically redoing https://github.com/facebook/zstd/pull/1818 by @inferno-chromium from the OSS-Fuzz team
2019-10-08 14:13:45 -07:00
Sen Huang 6e406b55ee Casting to void* to avoid C4090 warnings 2019-10-08 09:54:59 -04:00
Sen Huang 332b5988d8 Suppress C4090 warning 2019-10-08 09:44:24 -04:00
Sen Huang f9b1da6f62 Hopefully fix VS compiler warning 2019-10-08 00:15:28 -04:00
Yann Collet e0d413d648 fixed init warning 2019-10-04 15:09:52 -07:00
Bimba Shrestha 24bc2e062d
Merge pull request #1 from facebook/dev
Merging main dev
2019-10-04 10:29:29 -07:00
Sen Huang 6b81bfb257 Changed function name, fix test 2019-10-03 15:23:49 -04:00
Sen Huang 64bc441d7d Now constructs final destination path without allocating new table 2019-10-03 13:53:04 -04:00
Bimba Shrestha 36528b96c4 Manually moving instead of memcpy on decoder and using genBuffer() 2019-10-03 09:26:51 -07:00
Bimba Shrestha 61ec4c2e7f Cleaning sequence parsing logic 2019-10-03 06:42:40 -07:00
Yann Collet cadff8cdc4 zlibwrapper : fixed flag-variable name
release doesn't use specific std/debug flags
2019-10-02 17:41:19 -07:00
Yann Collet c69ed0f8d7 updated tests for zlibwrapper C90 strict compatibility 2019-10-02 17:32:19 -07:00
Sen Huang 4dc604cab8 Addressed comments on malloc 2019-10-02 18:34:42 -04:00
Sen Huang c763457e0a Static analyze fix 2019-10-02 15:30:24 -04:00
Sen Huang b93f1b2a30 CI Tests fix 2019-10-02 11:29:34 -04:00
Sen Huang 147b761454 Removed strdup() dependency 2019-10-02 11:18:14 -04:00
Sen Huang 1e4d602412 Modified message 2019-10-02 11:11:35 -04:00
Sen Huang f80437c586 Add support for --output-dir-flat
New flag to specify output directory destination for multiple files.
2019-10-02 11:08:20 -04:00
Bimba Shrestha b63a1e7ae5 Typo fix 2019-09-27 07:20:20 -07:00
Bimba Shrestha 91daee5c06 Fixing appveyor test 2019-09-26 16:21:57 -07:00
Bimba Shrestha 75b1286354 Fixing shortest failure 2019-09-26 16:07:34 -07:00
Bimba Shrestha bb27472afc Adding more realistic test for get sequences 2019-09-26 15:38:31 -07:00
Yann Collet 69c9401932 simplified Makefile
and fixed a few c++-compat issues
2019-09-26 15:01:29 -07:00
Yann Collet 0582b27cae added c++-compat build flag
ensure code can be compiled "as is" in C++ mode

also : restructured flags so that they can be individually changed / disabled on command line
2019-09-26 13:08:25 -07:00
Yann Collet cb18fffe65 enforce C90 compatibility for zlibWrapper 2019-09-24 17:50:58 -07:00
Yann Collet ad2a2785f7 bump version number to v1.4.4
so that future reports on `dev` branch use this number instead
2019-09-24 15:15:33 -07:00
Bimba Shrestha c04245b257 Replacing assert with memory_allocation error code throw 2019-09-23 15:42:16 -07:00
Bimba Shrestha be0bebd24e Adding test and null check for malloc 2019-09-23 15:08:18 -07:00
Yann Collet a07da7b0db
Merge pull request #1808 from davidbolvansky/highset-codegen
Improve ZSTD_highbit32's codegen
2019-09-23 13:44:27 -07:00
Dávid Bolvanský 1ab1a40c9c Fixed one more place 2019-09-23 21:32:56 +02:00
Dávid Bolvanský 1f7228c040 Use clz ^ 31 instead of 31 - clz; better codegen for GCC 2019-09-23 21:23:09 +02:00
Nick Terrell 7451c6578c
Merge pull request #1804 from terrelln/wild-and-fast
Optimize (de)compression and fix wildcopy overread
2019-09-21 17:04:36 -07:00
Nick Terrell 5cb7615f1f Add UNUSED_ATTR to ZSTD_storeSeq() 2019-09-20 21:37:13 -07:00
Nick Terrell 5dc0a1d659 HINT_INLINE ZSTD_storeSeq()
Clang on Mac wasn't inlining `ZSTD_storeSeq()` in level 1, which was
causing a 5% performance regression. This fixes it.
2019-09-20 16:39:27 -07:00
Bimba Shrestha f3c4fd17e3 Passing in dummy dst buffer of compressbound(srcSize) 2019-09-20 15:50:58 -07:00
Felix Handte c047fcf7bf
Merge pull request #1806 from felixhandte/estimate-cctx-doc
Update Comment on `ZSTD_estimateCCtxSize()`
2019-09-20 15:36:00 -04:00
Nick Terrell 44c65da97e Remove literals overread in ZSTD_storeSeq() for ~neutral perf 2019-09-20 12:23:25 -07:00
W. Felix Handte f7d9b36835 Update Comment on `ZSTD_estimateCCtxSize()` 2019-09-20 14:11:29 -04:00
Nick Terrell fde217df04 Fix bounds check in ZSTD_storeSeq() 2019-09-20 08:25:12 -07:00
Nick Terrell 67b1f5fc72 Fix too strict assert 2019-09-20 01:23:35 -07:00
Nick Terrell e068bd01df [tests] Fix decodecorpus 2019-09-20 01:09:47 -07:00
Nick Terrell ddab2a94e8 Pass iend into ZSTD_storeSeq() to allow ZSTD_wildcopy() 2019-09-20 00:56:20 -07:00
Nick Terrell cdad7fa512 Widen ZSTD_wildcopy to 32 bytes 2019-09-20 00:52:15 -07:00
Nick Terrell efd37a64ea Optimize decompression and fix wildcopy overread
* Bump `WILDCOPY_OVERLENGTH` to 16 to fix the wildcopy overread.
* Optimize `ZSTD_wildcopy()` by removing unnecessary branches and
  unrolling the loop.
* Extract `ZSTD_overlapCopy8()` into its own function.
* Add `ZSTD_safecopy()` for `ZSTD_execSequenceEnd()`. It is
  optimized for single long sequences, since that is the important
  case that can end up in `ZSTD_execSequenceEnd()`. Without this
  optimization, decompressing a block with 1 long match goes
  from 5.7 GB/s to 800 MB/s.
* Refactor `ZSTD_execSequenceEnd()`.
* Increase the literal copy shortcut to 16.
* Add a shortcut for offset >= 16.
* Simplify `ZSTD_execSequence()` by pushing more cases into
  `ZSTD_execSequenceEnd()`.
* Delete `ZSTD_execSequenceLong()` since it is exactly the
  same as `ZSTD_execSequence()`.

clang-8 seeds +17.5% on silesia and +21.8% on enwik8.
gcc-9 sees +12% on silesia and +15.5% on enwik8.

TODO: More detailed measurements, and on more datasets.

Crdit to OSS-Fuzz for finding the wildcopy overread.
2019-09-19 21:07:14 -07:00
Bimba Shrestha ae6d0e64ae Addressing comments 2019-09-19 15:25:20 -07:00
Nick Terrell 0e76000dee
Merge pull request #1801 from terrelln/int-max
[test] Test the bounds of ZSTD_c_srcSizeHint
2019-09-19 11:10:13 -07:00
Yann Collet 3cac061db5
Merge pull request #1802 from bimbashrestha/rle_block_bound_fix_pt2
Adding 4 blocks to FSE_BLOCKBOUND() in lib/common (different from las…
2019-09-18 16:32:37 -07:00