161 Commits

Author SHA1 Message Date
Yann Collet
59a7116cc2 benchfn dependencies reduced to only timefn
benchfn used to rely on mem.h, and util,
which in turn relied on platform.h.
Using benchfn outside of zstd required to bring all these dependencies.

Now, dependency is reduced to timefn only.
This required to create a separate timefn from util,
and rewrite benchfn and timefn to no longer need mem.h.

Separating timefn from util has a wide effect accross the code base,
as usage of time functions is widespread.
A lot of build scripts had to be updated to also include timefn.
2019-04-10 12:37:03 -07:00
Peter (Stig) Edwards
4a9e0502e6
-Wformat-security not needed with -Wformat=2 2019-02-01 09:29:08 +00:00
Yann Collet
ededcfca57 fix confusion between unsigned <-> U32
as suggested in #1441.

generally U32 and unsigned are the same thing,
except when they are not ...

case : 32-bit compilation for MIPS (uint32_t == unsigned long)

A vast majority of transformation consists in transforming U32 into unsigned.
In rare cases, it's the other way around (typically for internal code, such as seeds).

Among a few issues this patches solves :
- some parameters were declared with type `unsigned` in *.h,
  but with type `U32` in their implementation *.c .
- some parameters have type unsigned*,
  but the caller user a pointer to U32 instead.

These fixes are useful.

However, the bulk of changes is about %u formating,
which requires unsigned type,
but generally receives U32 values instead,
often just for brevity (U32 is shorter than unsigned).
These changes are generally minor, or even annoying.

As a consequence, the amount of code changed is larger than I would expect for such a patch.

Testing is also a pain :
it requires manually modifying `mem.h`,
in order to lie about `U32`
and force it to be an `unsigned long` typically.
On a 64-bit system, this will break the equivalence unsigned == U32.
Unfortunately, it will also break a few static_assert(), controlling structure sizes.
So it also requires modifying `debug.h` to make `static_assert()` a noop.
And then reverting these changes.

So it's inconvenient, and as a consequence,
this property is currently not checked during CI tests.
Therefore, these problems can emerge again in the future.

I wonder if it is worth ensuring proper distinction of U32 != unsigned in CI tests.
It's another restriction for coding, adding more frustration during merge tests,
since most platforms don't need this distinction (hence contributor will not see it),
and while this can matter in theory, the number of platforms impacted seems minimal.

Thoughts ?
2018-12-21 18:09:41 -08:00
Yann Collet
3583d19c4e changed parameter names from ZSTD_p_* to ZSTD_c_*
for naming consistency
2018-12-05 17:26:02 -08:00
Rohit Jain
705e0b18ab Making changes to make it compile on my laptop 2018-10-11 15:51:57 -07:00
Ryan Schmidt
b567ce9d68 Fix name of macOS 2018-06-09 14:31:17 -05:00
Yann Collet
5373e44ba7 fixed contrib/adaptive-compression 2018-03-15 17:10:15 -07:00
Yann Collet
b0cb081dc8 last batch of header files changed to reflect new license (#825)
only remains to update contrib/linux-kernel (@terrelln)
2017-08-31 12:20:50 -07:00
Yann Collet
e21384fffb fixed more file headers after license change (#825) 2017-08-31 12:11:57 -07:00
Paul Cruz
733ca51360 Updating README.md 2017-08-15 17:48:23 -07:00
Paul Cruz
fc90469587 updated program name print statement 2017-08-10 16:11:59 -07:00
Paul Cruz
7069bb9458 Merge branch 'adapt-approach-4' into bug-fixes 2017-08-07 13:18:53 -07:00
Paul Cruz
e100a311eb removed direct assignment of 22, used ZSTD_maxCLevel() instead 2017-08-07 13:11:07 -07:00
Paul Cruz
01237e3b35 changed multi to zstd-adaptive in the help menu 2017-08-03 15:13:49 -07:00
Paul Cruz
8be7bba08c added mutex for compression level to avoid data race 2017-08-02 10:27:33 -07:00
Paul Cruz
69ef22c0ac added detach statements to prevent resource leak 2017-08-01 17:36:13 -07:00
Paul Cruz
0295737ad7 change signal to broadcast for jobCompressed condition varaible since multiple threads waiting 2017-07-31 13:43:03 -07:00
Paul Cruz
9ea7df03de add install target in makefile 2017-07-31 11:04:17 -07:00
Paul Cruz
f60cd3f99b print defaults and range, remove EXT 2017-07-31 09:47:09 -07:00
Paul Cruz
e22b60cb76 removed ternary operation, added assert statement, check to make sure initial compression level is within bounds 2017-07-28 17:46:51 -07:00
Paul Cruz
cb9af53e77 delete empty line 2017-07-28 17:28:25 -07:00
Paul Cruz
51788225db remove exe extension from makefile, reinclude pthread flag 2017-07-28 17:27:36 -07:00
Paul Cruz
4d904ac800 add flags for multithreading 2017-07-28 16:12:58 -07:00
Paul Cruz
0f4cb67b00 add tests for compression bounds, fix another warning 2017-07-28 15:55:02 -07:00
Paul Cruz
ff54fced64 patched style errors, add ability to bound compression level variation 2017-07-28 15:30:46 -07:00
Paul Cruz
2320e7378a remove unused variable, add documentation for context fields 2017-07-26 17:02:47 -07:00
Paul Cruz
ab5a78547e fix leaky abstraction regarding measuring completion 2017-07-26 16:40:05 -07:00
Paul Cruz
715f36ca81 added definitions for conversion constants, moved forced compression check to top of adaptCompressionLevel, used ZSTD_BLOCKSIZE_MAX 2017-07-26 15:52:15 -07:00
Paul Cruz
6c1c1242fc set the window log value before performing compression 2017-07-26 14:29:59 -07:00
Paul Cruz
a959cc881a moved reset of completion to right after wait 2017-07-26 10:34:48 -07:00
Paul Cruz
305d5ee70f change to >= convergence counter 2017-07-26 10:20:29 -07:00
Paul Cruz
be92a38d6a decrease completion requirements for change, move create thread wait, merge cases where compression thread should wait 2017-07-26 10:05:10 -07:00
Paul Cruz
0b18d21e03 building on readme, added another help tip in the menu 2017-07-25 17:47:02 -07:00
Paul Cruz
7cc74e0b27 adding more to readme 2017-07-25 16:55:16 -07:00
Paul Cruz
8dbb07d822 updated progress bar with better representation of time, added const 2017-07-25 16:03:43 -07:00
Paul Cruz
9a132707af changing time units to seconds 2017-07-25 15:26:26 -07:00
Paul Cruz
31a9ed9883 updated const values, added more comments 2017-07-25 14:53:40 -07:00
Paul Cruz
5cfbf609a4 removed old debug statements no longer being used 2017-07-25 14:31:48 -07:00
Paul Cruz
0882cd1981 progress bar -- don't print num jobs, time elapsed shown in seconds 2017-07-25 14:26:55 -07:00
Paul Cruz
310c12d07e moved debug statements to a compiler flag 2017-07-25 14:08:39 -07:00
Paul Cruz
e02c79f833 started using decrease cooldown so that compression level would not decrease several times in a row 2017-07-25 11:16:27 -07:00
Paul Cruz
85d7c919f6 created independent function for controlling how completion relates to compression level change 2017-07-25 10:32:14 -07:00
Paul Cruz
6f1e260edd added mechanism for getting rid of spikes 2017-07-25 10:01:10 -07:00
Paul Cruz
700758d676 added help statement for -p, switched it to hide progress bar now that progress bar is default 2017-07-24 16:26:20 -07:00
Paul Cruz
df3754b6ed add quiet option, make progress bar default 2017-07-24 16:19:07 -07:00
Paul Cruz
4dc83ca64c compression thread should take measurements independently based on whether or not the create/write thread will actually bottleneck performance 2017-07-24 15:14:58 -07:00
Paul Cruz
0ee3f8c2f8 adding more debug 2017-07-24 15:06:11 -07:00
Paul Cruz
8328f8192a updating debug statements again 2017-07-24 14:40:23 -07:00
Paul Cruz
d3d759301f changing position of endline for debug 2017-07-24 13:47:39 -07:00
Paul Cruz
e508f632d6 updated comments and debug statements 2017-07-24 11:01:36 -07:00