Commit Graph

122 Commits (13fddac46d6997109ee56537445b294a78717b6d)

Author SHA1 Message Date
George Lu d4730a4f66 Update fulltable to use same interface
Add seperateFiles flag
2018-08-13 16:15:52 -07:00
George Lu 43b4971ca8 Renames, Documentation Updates 2018-08-13 16:15:52 -07:00
George Lu a884b76bc2 Style Changes
Add single run dictionaries
Change MB to be consistent 1 << 20 rather than 1,000,000
2018-08-13 16:15:52 -07:00
George Lu b3544217b7 Cleanup 2018-08-13 16:15:52 -07:00
George Lu 8ff0de15e4 Generalize, macro magic numbers 2018-08-13 16:15:52 -07:00
George Lu 3a2e95eba4 Perf improvements
try decay
strategy selection skipping
2018-08-13 16:15:52 -07:00
George Lu 2bdfe6ca71 Better Display 2018-08-13 16:15:52 -07:00
George Lu f67d040c39 Bugfixes, style changes
Complete euclidean distance climb
2018-08-13 16:15:52 -07:00
George Lu 5f4502fc07 New climb
feas part 2 uses euclidean metric
2018-08-13 16:15:52 -07:00
George Lu 13611249a5 Table
Compiling
+Euclidean Metric
2018-08-13 16:15:52 -07:00
George Lu 0cea754024 Revert "Reorder declaration"
This reverts commit 3ac2c22485.
2018-08-13 16:15:34 -07:00
George Lu 486e586eed Revert "Default lvl 1"
This reverts commit 0cc75d6ee0.
2018-08-13 16:13:46 -07:00
George Lu 0cc75d6ee0 Default lvl 1
MB to 2^20
2018-08-13 14:55:56 -07:00
George Lu 3ac2c22485 Reorder declaration 2018-08-09 16:38:32 -07:00
George Lu bfe8392e23 Remove ctx from benchMem 2018-08-09 12:07:57 -07:00
George Lu 0ece2e5cdc Add consts
+ fix gcc-8 warnings
2018-08-09 11:38:09 -07:00
George Lu 6f480927af argument parsing cleanup
+ clarifying comment
2018-08-09 10:42:58 -07:00
George Lu ad16a69408 Readability improvements, renaming 2018-08-09 10:42:58 -07:00
George Lu 8278a49cb6 const srcPtrs 2018-08-09 10:42:58 -07:00
George Lu 3d230db853 Change speed representation from floating point to integral 2018-08-09 10:42:58 -07:00
George Lu 8faeb41679 Update Documentation
Change comment // to  /* */
Add more description of what functions do
Remove outdated comments
2018-08-09 10:42:58 -07:00
George Lu dd270b2f75 Renaming / Style fixes 2018-08-09 10:42:58 -07:00
George Lu e148db366e Separate capacity vs size
Also:
Make suggested fixes
-varInds_t
-reorder some arguments
-remove code duplication
-update README / -h
-Fix memory leaks
2018-08-09 10:42:58 -07:00
George Lu df026e159f Fix windows implicit casting bugs 2018-08-09 10:42:58 -07:00
George Lu 0f91b039ff Add Levels 2018-08-09 10:42:58 -07:00
George Lu 7b5b3d7ae3 BenchMem with block compressed sizes passed back up 2018-08-09 10:42:58 -07:00
George Lu 3adc217ea4 Total Changes:
Add different constraint types (decompression speed, compression memory, parameter constraints)
Separate search space by strategy + strategy selection
Memoize results
Real random restarts
Support multiple files
Support Dictionary inputs
Debug Macro for extra printing
2018-08-09 10:42:58 -07:00
George Lu fab4438801 Dictionary + Multiple file Loading 2018-08-09 10:42:58 -07:00
George Lu eb21b7f482 Not crashing 2018-08-09 10:42:58 -07:00
George Lu 5f49034520 Working V1 2018-08-09 10:42:58 -07:00
George Lu cffb6da339 Parses additional parameters
Additional constraint checking

Minor fixes

more param parsing

Add Memory

Change paramVariation

work on feasibility

reformat bench

Changed Paramgrill to use bench.c benchmarking

customlevel macro

Printing Flag

Minor changes

Explicit casting

Makefile fix

casting, type fix

Printing Flag

Minor Changes

comments, helper fn's
2018-08-09 10:42:58 -07:00
George Lu 20f4f32379 Add to bench
-Remove global variables
-Remove gv setting functions
-Add advancedParams struct
-Add defaultAdvancedParams();
-Change return type of bench Files
-Change cli to use new interface
-Changed error returns to own struct value
-Change default compression benchmark to use decompress_generic
-Add CustomBench function
-Add Documentation for new functions
2018-06-14 14:23:24 -04:00
George Lu cc6539f4b9 Requested changes
Remove g_displaylevel/setNotificationLevel function
Add extern "C"
Remove averaging
Reorder arguments
2018-06-14 14:18:46 -04:00
Yann Collet c986dbf241
Merge pull request #1168 from GeorgeLu97/paramgrillfeatures
Have paramgrill share bench.c benchmarking function
2018-06-13 11:38:29 -04:00
George Lu 01d940b670 Requested changes
-Remove g_displaylevel/setNotificationLevel function
-Add extern "C"
-Remove averaging
-Reorder arguments

More fixes

-Added BMK_return_t (result + possible error)
-Correct comment'
-Nullcheck ctx, dctx when allocated
-Remove extra assert
2018-06-12 17:02:44 -04:00
George Lu 0e808d608b Make paramgrill use bench.c benchmarking 2018-06-08 12:01:05 -07:00
Yann Collet c2c47e24e0 support targetlen==0 with strategy==ZSTD_fast
to mean "normal compression",
targetlen >= 1 now means "disable huffman compression of literals"
2018-06-07 15:49:01 -07:00
George Lu 3f054dceb4 forgot \n, ; 2018-06-04 13:38:37 -07:00
George Lu ddf143ba6a Update usage_advanced 2018-06-04 10:16:05 -07:00
George Lu 5e586aa025 -O# with no file fails 2018-06-01 18:02:56 -07:00
George Lu e355f0a580 Added Level Option 2018-06-01 14:27:53 -07:00
George Lu ae6d1fd3fa Add Error Print 2018-06-01 13:54:08 -07:00
George Lu c9b4d20f02 Added new --zstd= format 2018-06-01 12:39:39 -07:00
George Lu cec205c842 copy paste 2018-05-31 17:39:36 -07:00
Yann Collet 2c392952f9 paramgrill: use NB_LEVELS_TRACKED in loop
make it easier to generate/track more levels
than ZSTD_maxClevel()
2018-05-13 17:25:53 -07:00
Yann Collet 3f89cd1081 minor : factor out errorOut() 2018-05-12 14:09:32 -07:00
Yann Collet 50993901b2 paramgrill: subtle change in level spacing
distance between levels is slightly increased
to compensate for level 1 speed improvements
and the will to have stronger level 19
extending the range of speed to cover.
2018-05-12 09:40:04 -07:00
Yann Collet a3f2e84a37 added programmable constraints 2018-05-11 19:43:08 -07:00
Yann Collet 17c19fbbb5 generalized use of readU32FromChar()
and check input overflow
2018-05-11 17:32:26 -07:00
Yann Collet b7b80e1569 fixed paramgrill 2018-03-21 16:09:15 -07:00
Nick Terrell dab8cfa3c7 Combine definitions of SEC_TO_MICRO 2017-11-30 19:40:53 -08:00
Nick Terrell 9a2f6f477b Use util.h for timing 2017-11-30 14:57:25 -08:00
Yann Collet 42c1e64270 slightly improved ratio at -22
merging of repcode search into btsearch introduced a small compression ratio regressio at max level :
1.3.2 : 52728769
after repMerge patch : 52760789 (+32020)

A few minor changes have produced this difference.
They can be hard to spot.

This patch buys back about half of the difference,
by no longer inserting position at hc3 when a long match is found there.
It feels strangely counter-intuitive, but works :
after this patch : 52742555 (-18234)
2017-11-19 14:00:55 -08:00
Yann Collet 99435dbbab minor : search early-out on sufficient_len for hc3 and rep
very very small speed and ratio increases
2017-11-19 12:58:04 -08:00
Yann Collet 18b795374a UTIL_getFileSize() returns UTIL_FILESIZE_UNKNOWN on failure
UTIL_getFileSize() used to return zero on failure.
This made it impossible to distinguish a failure from a genuine empty file.
Both cases where coalesced.

Adding UTIL_FILESIZE_UNKNOWN constant has many consequences on user code,
since in many places, the `0` was assumed to mean "error".
This is no longer the case, and the error code must be actively checked.
2017-10-17 16:14:25 -07:00
Yann Collet 96f0cde31a minor function rename
ZSTD_estimateCStreamSize_advanced_usingCParams -> ZSTD_estimateCStreamSize_usingCParams
_usingX is clear.
_advanced feels redundant
2017-09-24 16:47:02 -07:00
Yann Collet 3128e03be6 updated license header
to clarify dual-license meaning as "or"
2017-09-08 00:09:23 -07:00
Yann Collet d7ad99b2ab Merge branch 'longRangeMatcher' into dev 2017-08-31 18:08:37 -07:00
Stella Lau b5b9275e67 Rename estimateCCtxSize_advanced() and estimateCStreamSize_advanced() 2017-08-29 10:49:29 -07:00
Yann Collet 32fb407c9d updated a bunch of headers
for the new license
2017-08-18 16:52:05 -07:00
Yann Collet 31af8290d1 ZSTD_estimateCCtx_advanced()
ZSTD_estimateCCtx() is now a "simple" function,
taking int compressionLevel as single argument.

ZSTD_estimateCCtx_advanced() takes a CParams argument,
which is both more complete and more complex to generate.
2017-06-26 15:52:39 -07:00
Yann Collet 83d0c764dc added several compilation flags 2017-05-15 17:15:46 -07:00
Nick Terrell 5f2c7213c7 Merge remote-tracking branch 'upstream/dev' into btopt
* upstream/dev: (305 commits)
  added test for ZSTD_estimateCStreamSize()
  changed variable name, for clarity
  fixed ZSTD_estimateCStreamSize()
  shortened ZSTD_createCStream_Advanced()
  fixed symbols test
  added ZSTD_estimateDStreamSize()
  changed name frameParams into frameHeader
  regroup memory usage function declarations
  separated ZSTD_estimateCStreamSize() from ZSTD_estimateCCtxSize()
  bumped version number
  added ZSTD_estimateCDictSize() and ZSTD_estimateDDictSize()
  Updated ZSTD_freeCCtx()
  updated ZSTD_estimateCCtxSize()
  Updated ZSTD_sizeof_CCtx()
  merged CCtx and CStream as a single same object
  cli : -d and -t do not stop after a failed decompression
  added dev branch CircleCI badge
  added dev branch Appveyor badge
  keep dev branch status only
  creates a binary archive without the `programs` directory
  ...
2017-05-10 16:49:58 -07:00
Yann Collet 4f818182b8 clarified frame parameters for ZSTD_compress*_usingCDict()
created ZSTD_compressBegin_usingCDict_internal(),
which gives direct control to frame Parameters.
ZSTD_resetCStream_internal() now points into it.
2017-04-17 18:29:06 -07:00
Yann Collet 9631e603ae paramgrill : results from optimizer use cli format
Best found configuration is displayed
using documented cli advanced parameter syntax.
2017-04-04 16:54:33 -07:00
Yann Collet 89b32f3db0 fix paramgrill -O#
ensure proposed config does not require more memory than necessary
2017-04-04 16:41:11 -07:00
Yann Collet 2724f25567 fixed paramgrill -O#
find optimal settings for a minimum speed
2017-04-04 16:31:17 -07:00
Yann Collet c2007388a5 fixed bench.c : optional advanced parameters applied
before creating cdict
2017-04-04 15:35:06 -07:00
Nick Terrell eeb31eed39 s/ZSTD_btopt2/ZSTD_btultra/g 2017-03-09 11:44:25 -08:00
Przemyslaw Skibinski 5c5f01f3da added ZSTD_btopt2 strategy 2016-10-25 12:25:07 +02:00
Yann Collet 4ded9e591c added boilerplate 2016-08-30 11:06:28 -07:00
inikep 2868d565e4 test-related files moved to tests/ 2016-08-18 13:18:11 +02:00