Commit Graph

163 Commits (145c2d12f96d14c41896c91ef517404fbeaf350d)

Author SHA1 Message Date
Yann Collet 04a1557e28 fuzzer tests : added -v command
so that unit tests get listed in CI console
and we can see which one failed.
2017-12-13 10:08:52 -08:00
Yann Collet e9dc0ab884 zstd binary, when compiled from /tests, enables debug mode
adds ZSTD_DEBUG=1 and -g
2017-12-12 18:20:04 -08:00
Yann Collet 453fb5e830 removed -ftrapv from tests/ debug flags
-ftrapv is apparently buggy for gcc.
versions >= 5 are supposed to work better,
but even then, some complaints say it's still flaky when optimizations are enabled.
I even saw a post saying it only works if one creates its own signal handler,
which would make this flag no longer transparent.

on clang, it seems to work correctly.
But we would need to add a method to selectively add flags depending on compiler.
That's too much troubles for the intended benefit
(just catch integer overflows, which we can also do using ubsan).
2017-12-01 10:30:53 -08:00
Yann Collet b3d76e0a94 Merge branch 'dev' of github.com:facebook/zstd into dev 2017-11-28 15:03:38 -08:00
Yann Collet f5747afe0f add integer overflow trapping to debug builds
doc: https://gcc.gnu.org/onlinedocs/gcc-4.0.2/gcc/Code-Gen-Options.html
2017-11-28 15:02:48 -08:00
W. Felix Handte 62c746dcf9 Add Test on LZ4 Format Input Buffer Overrun 2017-11-28 12:06:48 -05:00
Yann Collet 23767e950a fix one UB pointer arithmetic in encoder
Instead of calculating distance between 2 memory objects, which is UB,
we extract the offset from object 1, and transfer it into object 2.
2017-11-17 13:24:51 -08:00
Nick Terrell e600b5d0da [test] Exercise all codes in dictionary tables 2017-10-16 18:05:36 -07:00
Yann Collet 7f6a783862 fixed a small error in decodeCorpus
a compressed block must be strictly smaller than its decompressed size.
2017-10-07 15:19:52 -07:00
Yann Collet e580dc6a4a Merge pull request #860 from felixhandte/zstd-lz4-support-tests
Add Default LZ4 Support When Available
2017-09-29 22:32:54 -07:00
Yann Collet e0065cf660 make test : removed zstreamtest unit tests for variants
slightly reduced time to create dictionary
at beginning of unit tests
2017-09-28 18:34:38 -07:00
W. Felix Handte d0519d4b0c Add CLI Program Name Detection for LZ4 2017-09-28 19:18:15 -04:00
W. Felix Handte 5705d9f25a Add basic tests for the lz4 integration 2017-09-28 19:16:43 -04:00
Yann Collet 60059df051 shorter make test to avoid time out on travis CI
Timed tests (fuzzer) are reduced
Long tests are shortened (less data generated)
2017-09-27 15:16:27 -07:00
Yann Collet d7ad99b2ab Merge branch 'longRangeMatcher' into dev 2017-08-31 18:08:37 -07:00
Yann Collet e0cecd8736 fixed poolTests
needs more dependencies from zstd for custom allocators and error codes
2017-08-31 15:13:31 -07:00
Yann Collet b0cb081dc8 last batch of header files changed to reflect new license (#825)
only remains to update contrib/linux-kernel (@terrelln)
2017-08-31 12:20:50 -07:00
Stella Lau 9911153723 Move jobSize and overlapLog in zstdmt to cctxParams 2017-08-25 13:14:51 -07:00
Stella Lau 15fdeb9e41 Enforce nbThreads<=1 for estimateCCtxSize 2017-08-24 16:28:49 -07:00
Stella Lau 1c81f725ff Remove duplicated testing code 2017-08-23 15:47:15 -07:00
Stella Lau 91b30dbe84 Remove test parameter 2017-08-21 10:09:06 -07:00
Stella Lau f181f33bdf Disable tests and refactor 2017-08-21 01:59:08 -07:00
Stella Lau 023b24e6d4 Add cctx param tests 2017-08-20 22:55:07 -07:00
Yann Collet d6394cc4c3 fixed test-zstd-nolegacy 2017-08-20 10:15:44 -07:00
Yann Collet fa3aa04ccd Merge pull request #761 from paulcruz74/file-rename
renamed pool.c to poolTests.c
2017-07-14 09:09:45 -07:00
Yann Collet 88da8f1816 fix : propagate custom allocator to ZSTDMT though ZSTD_CCtx_setParameter()
also : compile fuzzer with MT enabled
2017-07-10 14:02:33 -07:00
Paul Cruz 89190ef07d renamed pool.c to poolTests.c 2017-07-10 11:32:30 -07:00
Yann Collet ed0243a63c removed zbufftest from list of `all` tests 2017-07-07 16:16:14 -07:00
Yann Collet 27e883371d fixed wrong assert() condition
A single job created by ZSTDMT_compress() can be < 256KB
if data to compress is < 256 KB
(in which case it is delegated to single thread mode)
2017-07-04 19:33:16 -07:00
Yann Collet 2cb9774f5e more precise estimation of amount to flush at end of stream (single thread mode)
also : can use DEBUGLEVEL variable in /tests
2017-07-04 12:39:26 -07:00
cyan4973 c07e43c2b5 added --show-leak-kind=all to valgrind tests 2017-07-01 07:05:11 -07:00
Yann Collet e7e5a8cef7 made fullbench compatible with multi-threading
fullbench 61/62 measure speed of ZSTD_compress_generic with 2 threads
2017-06-29 18:56:24 -07:00
Yann Collet 20eeb243d1 Merge pull request #729 from paulcruz74/corpus
Corpus
2017-06-26 17:47:28 -07:00
Paul Cruz 298fe9f23b added cli tests to test-all 2017-06-23 17:10:41 -07:00
Paul Cruz 6d7f442120 changed makefile test 2017-06-22 15:44:01 -07:00
Paul Cruz 2935239544 added description for dictionary cli tests 2017-06-22 15:23:59 -07:00
Paul Cruz 97a8f89bec made changes to cli test so it runs properly 2017-06-22 15:13:41 -07:00
Paul Cruz 829eb29033 added cli test for decodecorpus inside tests/Makefile. Also changed calculation of offset 2017-06-22 14:43:44 -07:00
Paul Cruz 028fb9d01b added cli test case in Makefile 2017-06-22 13:53:39 -07:00
Yann Collet 01743a36e7 fuzzer tests for new API 2017-06-16 17:56:41 -07:00
Paul Cruz 8dd621f788 changed makefile to include zdict functions, wrote out dictionary to path 2017-06-13 14:19:35 -07:00
Yann Collet 009d604e00 ZSTD_compress_generic() supports multiple successive frames
also : clarified streaming API implementation
2017-05-19 10:17:59 -07:00
Yann Collet 2d4d31c18a removed gcc compilation flag -Wbad-function-cast
It makes it more difficult to directly cast the result of a function,
requiring to store the result in an intermediate variable.
It does not necessarily help readability,
and this restriction can be difficult to overcome in some constructions,
like some macros.

also : fixed minor Visual conversion warnings in datagencli.c
2017-05-16 11:34:38 -07:00
Yann Collet 83d0c764dc added several compilation flags 2017-05-15 17:15:46 -07:00
Yann Collet 36e79a1a1f tests : -g is part of DEBUGFLAGS 2017-04-27 17:29:40 -07:00
Yann Collet c17e020c9a disable assert when compiling paramgrill
paramgrill is a benchmark calibration function.
Speed accuracy is critical, it cannot be altered by assert.
2017-04-20 12:50:02 -07:00
Yann Collet 4b987ad8ce Introduce ZSTD_initCStream_internal()
This is now the regroup point for ZSTD_initCStream*() functions

ZSTD_initCStream_advanced() now properly checks for parameters validity.

Also : added <assert.h> usage inside zstd_compress.c
Needs ZSTD_DEBUG=1 macro to be triggered.
Will be triggered by default from `tests` directory
2017-04-10 17:50:44 -07:00
Sean Purcell 5a2b857029 Update appveyor.yml to have short and long tests 2017-03-20 17:04:44 -07:00
Sean Purcell 0b0b10ec83 Add --no-big-tests flag for qemu fuzz-tests 2017-03-20 11:32:24 -07:00
Sean Purcell 120df494e9 Update builds to not support legacy v01-v03 2017-03-13 14:44:08 -07:00
Sean Purcell 7c8f5d5bc7 Make test times overwritable 2017-03-09 16:05:23 -08:00
Sean Purcell daec40db24 Update .travis.yml and Makefile for medium tests 2017-03-09 16:05:22 -08:00
Yann Collet 8b1d004031 added -Wformat-security flag, as recommended by @pixelb 2017-03-05 21:17:32 -08:00
Yann Collet 27526c7201 make : added target shortest
shortest only run fast part of playTests.sh .
cc @iburinoc
2017-03-01 17:02:49 -08:00
Yann Collet df9f9296e3 attempt to fix pthreat linking error
replacing -lpthread by -pthread
2017-02-24 00:16:05 -08:00
Yann Collet 831b4890ce minor tests/Makefile refactoring
and update of zstd_manual,html
2017-02-23 23:09:10 -08:00
Sean Purcell f119b62055 Create a tool that generates random, valid, Zstd frames for decoder testing
Note: Does not handle dictionaries currently
2017-02-22 16:08:34 -08:00
Przemyslaw Skibinski 337ec875b6 minor tweaks 2017-02-22 10:31:30 +01:00
Przemyslaw Skibinski 3d836bfd18 travis.yml: fix versionsTest target 2017-02-22 09:36:42 +01:00
Yann Collet ff176d3208 Merge pull request #548 from inikep/gz_compress
Circle CI tests
2017-02-16 14:57:59 -08:00
Przemyslaw Skibinski 9b5a1e9d97 added circle.yml 2017-02-14 20:06:41 +01:00
Nick Terrell 74b81ada25 Don't run test-pool with QEMU
> make test -n
    ...
    ./pool
    > make test -n QEMU_SYS=valgrind
    ...
    ./legacy
    # ./pool not run
2017-02-14 10:08:18 -08:00
Sean Purcell 5069b6c2c3 Merge branch 'dev' into multiframe 2017-02-10 10:08:55 -08:00
Sean Purcell 478d7174e5 Merge branch 'dev' into multiframe 2017-02-09 11:54:22 -08:00
Sean Purcell 2db7249265 Make pledgedSrcSize meaning clear for other functions
- Added tests
- Moved new size functions to static link only
2017-02-09 11:49:58 -08:00
Przemyslaw Skibinski 80c1e04c33 Merge branch 'dev' into dev11 2017-02-09 09:42:16 +01:00
Sean Purcell f07ddf88e8 Test multiframe legacy decoding with simple and streaming APIs 2017-02-08 15:13:20 -08:00
Nick Terrell b5e46b1255 Remove test-longmatch from test target and only run it once 2017-02-08 12:00:21 -08:00
Przemyslaw Skibinski 4b4f8c2d71 turn off test-pool for qemu-ppc64-static 2017-02-08 13:58:04 +01:00
Yann Collet 717c65d690 Merge pull request #519 from inikep/dev11
Dev11
2017-01-26 14:23:44 -08:00
Przemyslaw Skibinski 3bb010a667 .travis.yml: optimized order of short tests 2017-01-25 11:19:35 +01:00
Yann Collet d7e3cb58c5 Resolved merge conflict dev+zstdmt 2017-01-20 16:44:50 -08:00
Yann Collet 19d670ba9d Added ZSTDMT_initCStream_advanced() variant
Correctly compress with custom params and dictionary
Added relevant fuzzer test in zstreamtest

Also :
new macro ZSTDMT_SECTION_LOGSIZE_MIN, which sets a minimum size for a full job
(note : a flush() command can still generate a partial job anytime)
2017-01-19 15:32:07 -08:00
Yann Collet 0f984d94c4 changed MT enabling macro to ZSTD_MULTITHREAD 2017-01-19 14:05:07 -08:00
Yann Collet 736788f8e8 added streaming fuzzer tests for MT API
Also : fixed corner case, where nb of jobs completed becomes > jobQueueSize
which is possible when many flushes are issued
while there is not enough dst buffer to flush completed ones.
2017-01-19 12:15:29 -08:00
Przemyslaw Skibinski 8a0bc30a2d Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2017-01-17 13:02:29 +01:00
Przemyslaw Skibinski d72f4b6b7a added "Makefile is validated" 2017-01-17 12:40:06 +01:00
Yann Collet 33fce03045 added test checking dictID when using ZSTD_initCStream_usingCDict()
It shows that dictID is not properly added into frame header
2017-01-16 19:46:22 -08:00
Nick Terrell d132433534 Switch thread pool test to threading.h 2016-12-31 19:10:13 -05:00
Nick Terrell 9c499648e3 Add thread pool tests 2016-12-29 23:41:03 -08:00
Przemyslaw Skibinski 5f5a902453 "make test" is now compatible with Solaris 2016-12-22 18:05:07 +01:00
Nick Terrell 9d08597364 Add test for invalid offset rep codes 2016-12-20 11:13:45 -08:00
Yann Collet 31ff2a23be fix Travis long test list;
added zbufftest-dll
2016-12-17 19:10:10 +01:00
Yann Collet d46ecb58a5 added dll compilation tests 2016-12-17 16:28:12 +01:00
Nick Terrell 61e62c014f Test that all API symbols are exported 2016-12-16 13:29:23 -08:00
Yann Collet e795c8a5f6 Added ZSTD_initCStream_srcSize().
Added relevant test cases in zstreamtest
2016-12-13 17:00:14 +01:00
Nick Terrell 0acae734f1 Add exposing test case 2016-12-10 19:12:13 -08:00
Yann Collet 825dffbc43 moved zbuff source files into lib/deprecated 2016-12-05 19:28:19 -08:00
Przemyslaw Skibinski 7c6bbc3298 updated FIO_passThrough 2016-12-05 18:31:14 +01:00
Przemyslaw Skibinski 6b508b1770 updated test-gzstd 2016-12-05 18:02:40 +01:00
Przemyslaw Skibinski 0efaf7e7b1 added test-gzstd 2016-11-30 15:20:24 +01:00
Przemyslaw Skibinski cc3887085f updated build\README.md 2016-11-21 13:58:58 +01:00
Przemyslaw Skibinski 8bb86e330b create DLL with Windows 2016-11-21 12:51:01 +01:00
Yann Collet 52afb3993e zbuff API now generates deprecation warnings 2016-11-16 08:50:54 -08:00
Przemyslaw Skibinski eb977a42ef "file" moved to tests/Makefile 2016-11-14 17:26:58 +01:00
Przemyslaw Skibinski e579ab5faa introduced QEMU_SYS 2016-11-14 12:57:05 +01:00
Przemyslaw Skibinski 9adf7bfd8a fixed MinGW compilation (2) 2016-11-03 15:38:13 +01:00
Yann Collet 861cd06ded fix test-zstd-nolegacy 2016-11-03 01:11:56 -07:00
Yann Collet 589f011761 changed ZBUFF_* by ZSTD_*Stream() within fullbench,
in anticipation of future deprecation of ZBUFF_*.
2016-10-28 15:17:38 -07:00
Nick Terrell 3e82f2b61a Remove references to GPLv2 license
License headers added to Makefiles were taken from `zstd/Makefile`.
2016-10-11 15:28:40 -07:00