facebook/zstd
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,192 Commits 2 Branches 0 Tags
Commit Graph

1882 Commits

Author SHA1 Message Date
Yann Collet
ce6cd07c33 updated build documentation 2020-10-22 12:31:23 -07:00
Yann Collet
e3867fb735 fixed libzstd.dll compilation on mingw 
and zstd linking
 2020-10-22 11:52:19 -07:00
Yann Collet
dd24496951 programs/zstd also automatically generate object dir per conf 
same rules as lib/libzstd
can also be controlled via HASH and BUILD_DIR
 2020-10-21 23:38:33 -07:00
Yann Collet
01ecad2326 zstd in programs/ can also receive a DEBUGLEVEL value 
simplify tests : only set DEBUGLEVEL, not the flags directly
 2020-10-21 23:13:46 -07:00
Yann Collet
2224ec33ed
Merge pull request #2367 from facebook/progressive_build 
faster rebuild of zstd
 2020-10-21 15:43:14 -07:00
Yann Collet
684f8ea970 decouple zstd object files from lib/ 
That was a subtle one :
VPATH is affecting search for both %.c source and %.o object files.
This meant that, when an object file already exists in lib/,
it's used in programs/,
even though programs/ is supposed to generate its own %.o object files.

With the new vpath directive, this is no longer the case :
the search is only activated for %.c source files.
Now, local programs/%.o are always generated
even if equivalent ones are already created in lib/.

It more clearly guarantees that lib/ and programs/ can use different compilation directives
without mixing resulting %.o object files.
 2020-10-20 18:16:11 -07:00
Yann Collet
eb1959d926 fix .deps list 
and minor simplications
 2020-10-16 11:08:52 -07:00
Yann Collet
c355d149de support verbose build output with V=1 2020-10-16 00:21:49 -07:00
Yann Collet
b6b87c2b77 track header dependencies during build 
modifying a header file will now trigger recompilation of related *.c units
 2020-10-16 00:18:16 -07:00
Yann Collet
80cee8d3fe faster rebuild of zstd 
Building the zstd CLI costs time.
Some part of it is incompressible, leading to substantial iteration delay when testing code modifications.
That's mainly because all source files from the library must be rebuilt from source every time.

The main reason we don't build the CLI from library object files
is that we can't just build the object directly in the lib/ directory
(which they would by default)
since they use different compilation flags.
Specifically, the CLI enables multithreading, while the library doesn't (by default).

This is solved in this commit, by generating the object files locally.
Now, the CLI and the library can employ different sets of flags, without tripping over each other.
All library object files are generated directly into programs/ dir.
This works because no 2 source files have the same name.

Now, modifying a file doesn't require to recompile the entire lib, just the modified files.
The recipe is also compatible with `-j` parallel build, leading to large build time reductions on multi-core systems.
 2020-10-16 00:01:41 -07:00
senhuang42
467e4383b0 Merge branch 'dev' of github.com:senhuang42/zstd into change_ldm_mt_config 2020-10-14 10:17:50 -04:00
senhuang42
a39614de1f Fix incorrect usage of cycleLog() in --patch-from 2020-10-13 13:00:27 -04:00
Like Ma
cc907770bd Fix building on AIX 5.1 2020-10-09 18:34:00 +08:00
senhuang42
7259b258d1 Add callsites to zstdcli.c and tests to playTests.sh 2020-10-07 13:47:38 -04:00
senhuang42
93cd9d8a6e Add hasStdoutOutput setter to fileio.h 2020-10-07 13:44:25 -04:00
senhuang42
dd3dac9ce0 Fixed logic for stdout output 2020-10-07 13:43:27 -04:00
senhuang42
1ebe360d0f Add new stdoutOutput field 2020-10-07 13:42:34 -04:00
Yann Collet
f7d4943788
Merge pull request #2330 from senhuang42/fix_stdinout_error_messages 
Improve error messages on console input/output
 2020-10-06 15:58:37 -07:00
Yann Collet
83461ce963
Merge pull request #2322 from senhuang42/guard_against_stdin_for_warning_prompts 
Don't let warning messages consume input from stdin
 2020-09-30 08:26:50 -07:00
senhuang42
ce56810a32 Modify error messages on console input/output 2020-09-28 12:15:18 -04:00
senhuang42
02422db841 Fix Stdin typo 2020-09-25 11:51:35 -04:00
Yann Collet
236b98e7de
Merge pull request #2310 from senhuang42/fix_multifile_status_bar 
Fix multifile status bar and summary - clear out extraneous characters
 2020-09-25 07:48:33 -07:00
senhuang42
9f7212a48b Update unit tests 2020-09-24 16:44:33 -04:00
senhuang42
88f4410390 Add more useful failure message when stdin is an input 2020-09-24 16:29:12 -04:00
senhuang42
93d63eaeb8 Expand UTIL_requireUserConfirmation to include stdin input check 2020-09-24 15:58:06 -04:00
senhuang42
432186cbea Add FIO_determineHasStdinInput() function and member to fCtx 2020-09-24 15:55:30 -04:00
senhuang42
0e8ac6b995 Add fCtx to FIO_openDstFile() 2020-09-24 15:49:30 -04:00
senhuang42
7aa3da1cd7 Use IS_CONSOLE macro to detect that we're indeed using a console 2020-09-22 14:15:52 -04:00
senhuang42
6b6cc80196 Support .zstd suffix only for decompression 2020-09-18 12:49:51 -04:00
senhuang42
29b8f2eb08 Fix multifile decompression summary 2020-09-16 10:38:13 -04:00
senhuang42
1d5c6fda9f Add filename truncations to decompression 2020-09-16 10:28:45 -04:00
senhuang42
ab0d332563 Add display clears 2020-09-15 15:53:32 -04:00
senhuang42
cad6bf99af Add padding to remove previous line's leftovers, and keep the printed line to around the same size 2020-09-15 13:01:46 -04:00
Yann Collet
05622992d5
Merge pull request #2292 from senhuang42/multifile_status_update_fio_refactor 
Make multifile (de)compression print out a summary
 2020-09-14 14:06:26 -07:00
Yann Collet
e583e0be8c
Merge pull request #2299 from senhuang42/env_var_num_threads 
Allow environment variable to specify number of threads for compression
 2020-09-14 14:04:19 -07:00
W. Felix Handte
d46306087b Enable Dedicated Dict Search in the CLI 2020-09-10 18:51:52 -04:00
senhuang42
a71963c7b8 nbThreads instead of numThreads 2020-09-09 12:40:00 -04:00
senhuang42
243c8dfb41 Add include guards for init_numThreads() 2020-09-08 09:26:16 -04:00
senhuang42
cc29492c5f Update documentation for environment variable 2020-09-08 08:42:46 -04:00
senhuang42
136a0673f6 Add quick documentation for ZSTD_NUMTHREADS in the code 2020-09-07 18:19:31 -04:00
senhuang42
888c385a49 Change param name to ZSTDCLI_NUMTHREADS_DEFAULT 2020-09-07 18:11:42 -04:00
senhuang42
972e063219 Change default num_threads value 2020-09-07 18:09:48 -04:00
senhuang42
5123496104 Adjust function signatures 2020-09-07 13:13:05 -04:00
senhuang42
28a9dc78c3 Always update bytes status in FIO_ctx_t 2020-09-03 20:23:30 -04:00
senhuang42
60ee0519ca Remove extraneous FIO_ctx_t functions 2020-09-03 20:22:56 -04:00
senhuang42
48bca10792 Address memory leak in CLI from fCtx 2020-09-03 10:14:04 -04:00
senhuang42
23feaecbe5 Fix FIO_removeMultiFilesWarning conflict 2020-09-03 09:49:13 -04:00
senhuang42
49949f1d51 Merge branch 'dev' into multifile_status_update_fio_refactor 2020-09-03 09:44:47 -04:00
senhuang42
202b2954a2 Remove comma-separated decls 2020-09-03 09:28:40 -04:00
senhuang42
a480b02044 Adjust comments and struct to better reflect FIO_ctx_t 2020-09-03 09:26:30 -04:00