facebook/zstd
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,308 Commits 2 Branches 0 Tags
Commit Graph

159 Commits

Author SHA1 Message Date
binhdvo
d7e1736375
Fix build for cygwin/bsd (#2882) 2021-11-29 14:11:39 -05:00
Dimitris Apostolou
ebbd675998
Fix typos 2021-11-13 10:04:04 +02:00
Yann Collet
eab692211e removed pretty-print of sizes in benchmark 
This is less appropriate for this mode :
benchmark is about accuracy,
it's important to read the exact values.
 2021-09-03 12:51:02 -07:00
W. Felix Handte
a719edbbc3 Pull utime() Call into Helper 2021-08-04 14:49:00 -04:00
Binh Vo
6a46e38deb Add option to use logical cores for default threads 2021-06-16 15:46:17 -04:00
W. Felix Handte
2af3687c50 Switch to Binary Size Prefixes (e.g., "MB" -> "MiB") 
Suggested by @aqrit, a little more verbose, but hopefully addresses a real
ambiguity.
 2021-06-10 12:53:07 -04:00
W. Felix Handte
9c340ce014 Require -vv to Enable Full Precision 2021-06-10 12:53:07 -04:00
W. Felix Handte
93bb368d74 Change Suffix (e.g., "G" -> " GB") 2021-06-10 12:53:07 -04:00
W. Felix Handte
464bfb022e In Verbose Mode, Preserve Full Precision Where Possible 2021-06-10 12:53:07 -04:00
W. Felix Handte
9b67219b1e Fix Integer Constants; Fix Comparison 2021-06-10 12:53:07 -04:00
W. Felix Handte
bbb81c8801 Avoid snprintf() in Preparing Human-Readable Sizes; Improve Formatting 
This produces the following formatting:

   Size    | `zstd` | `ls -lh`
---------- | ------ | --------
1          | 1      | 1
12         | 12     | 12
123        | 123    | 123
1234       | 1.21K  | 1.3K
12345      | 12.1K  | 13K
123456     | 121K   | 121K
1234567    | 1.18M  | 1.2M
12345678   | 11.8M  | 12M
123456789  | 118M   | 118M
1234567890 | 1.15G  | 1.2G
999        | 999    | 999
1000       | 1000   | 1000
1001       | 1001   | 1001
1023       | 1023   | 1023
1024       | 1.000K | 1.0K
1025       | 1.00K  | 1.1K
999999     | 977K   | 977K
1000000    | 977K   | 977K
1000001    | 977K   | 977K
1023999    | 1000K  | 1000K
1024000    | 1000K  | 1000K
1024001    | 1000K  | 1001K
1048575    | 1024K  | 1.0M
1048576    | 1.000M | 1.0M
1048577    | 1.00M  | 1.1M

This was produced with the following invocation:

```
for N in 1 12 123 1234 12345 123456 1234567 12345678 123456789 1234567890 999 1000 1001 1023 1024 1025 999999 1000000 1000001 1023999 1024000 1024001 1048575 1048576 1048577; do
  head -c $N /dev/urandom > r$N
done
./zstd -i1 -b1 -S r1 r12 r123 r1234 r12345 r123456 r1234567 r12345678 r123456789 r1234567890 r999 r1000 r1001 r1023 r1024 r1025 r999999 r1000000 r1000001 r1023999 r1024000 r1024001 r1048575 r1048576 r1048577
```
 2021-06-10 12:53:07 -04:00
Scott Baker
8e0a9695d7 Attempt to fix a failing test with help from @aqrit 2021-06-10 12:53:07 -04:00
Scott Baker
1eb852854b Some fixes to address things @felixhandte found 2021-06-10 12:53:07 -04:00
Scott Baker
376a2730a8 Try enabling the BIG strings now the unsigned long long is in effect 2021-06-10 12:53:07 -04:00
Scott Baker
20b9b00b41 Try unsigned long long 2021-06-10 12:53:07 -04:00
Scott Baker
64385ef7cb Update humanSize() to skip the big numbers (it requires 64 bit) 2021-06-10 12:53:07 -04:00
Scott Baker
1ef6f3d079 Use unsigned long instead to help with some tests 2021-06-10 12:53:07 -04:00
Scott Baker
e5fc830795 human_size() should use size_t 2021-06-10 12:53:07 -04:00
Scott Baker
eefdbcd93a Make the variable types match 2021-06-10 12:53:07 -04:00
Scott Baker
b6b23dfe64 Convert names to CamelCase 2021-06-10 12:53:07 -04:00
Scott Baker
b70175e5ec Put the human_size() function in util.c 2021-06-10 12:53:07 -04:00
Yann Collet
df05b2ba7c fix --filelist compatibility with Windows cr+lf line ending 2021-05-05 18:01:55 -07:00
W. Felix Handte
b87f97b3ea Create Files with Desired Permissions; Avoid chmod(); Remove UTIL_chmod() 2021-05-05 13:10:34 -04:00
W. Felix Handte
33f3e293e8 Allow Reading from Block Devices with --force 2021-05-04 16:25:26 -04:00
Nick Terrell
a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files 
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
 2021-03-30 10:30:43 -07:00
Sen Huang
f27e326456 Restrict dictmode regression tests only to advanced API, fix some compiler warnings 2021-03-25 10:39:08 -07:00
Paul Bone
4d6c78fb89 Only set numPhysicalCores if ratio is valid 2021-03-03 10:59:00 +11:00
Paul Bone
eb1a09df61 If cpuinfo parsing fails fallback to sysconf 2021-03-03 10:58:51 +11:00
W. Felix Handte
61db590ad8 Detect .. in Paths Correctly 
This commit addresses #2509.
 2021-02-26 12:29:42 -05:00
W. Felix Handte
a774c57973 Use umask() to Constrain Created File Permissions 
This commit addresses #2491.

Note that a downside of this solution is that it is global: `umask()` affects
all file creation calls in the process. I believe this is safe since
`fileio.c` functions should only ever be used in the zstd binary, and these
are (almost) the only files ever created by zstd, and AIUI they're only
created in a single thread. So we can get away with messing with global state.

Note that this doesn't change the permissions of files created by `dibio.c`.
I'm not sure what those should be...
 2021-02-17 15:27:39 -05:00
Nick Terrell
66e811d782 [license] Update year to 2021 2021-01-04 17:53:52 -05:00
Yann Collet
fed1c62571 fix gcc10 warnings 
gcc10 doesn't like its own strncpy
 2020-11-30 04:44:37 -08:00
senhuang42
02422db841 Fix Stdin typo 2020-09-25 11:51:35 -04:00
senhuang42
88f4410390 Add more useful failure message when stdin is an input 2020-09-24 16:29:12 -04:00
senhuang42
93d63eaeb8 Expand UTIL_requireUserConfirmation to include stdin input check 2020-09-24 15:58:06 -04:00
senhuang42
0e8ac6b995 Add fCtx to FIO_openDstFile() 2020-09-24 15:49:30 -04:00
senhuang42
7aa3da1cd7 Use IS_CONSOLE macro to detect that we're indeed using a console 2020-09-22 14:15:52 -04:00
senhuang42
7991c55181 Move logic into new function FIO_removeMultiFilesWarning, add support for decompression 2020-08-26 16:50:20 -04:00
senhuang42
aab11ce3db Unified warning prompts into new function UTIL_requireUserConfirmationToProceed() 2020-08-25 11:25:49 -04:00
W. Felix Handte
51ac0207af Remove UTIL_statFile() and UTIL_statDir(); Decompose Former Call-Sites 2020-08-10 15:28:02 -04:00
W. Felix Handte
93dda988c8 Remove Unused Function UTIL_fileExist() 2020-08-10 15:22:53 -04:00
W. Felix Handte
c1449143c5 Share stat() Calls in Uses of UTIL_chmod() 2020-08-05 12:10:42 -04:00
W. Felix Handte
0a8aacb4db Use stat() to Check that File Should be chmod()-ed 
Rather than special-casing a check for `/dev/null`, this uses `stat()` to
avoid `chmod()`-ing any non-regular file. I believe this is the desirable
behavior. `UTIL_chmod()` is never called on directories at the moment, only
output files.
 2020-08-05 12:00:12 -04:00
W. Felix Handte
7238cca1a1 Deduplicate Some Low-Hanging Fruit of Redundant Stat Calls 2020-08-05 01:08:34 -04:00
W. Felix Handte
44fa052599 Introduce Variants of Various UTIL Functions that Take Pre-Populated stat_t Structs 
Instead of calling `stat()`, these functions accept the result of a previous
`stat()` call on the file in question, which will allow us to make multiple
decisions around a file without redundant `stat()` calls.
 2020-08-05 01:00:06 -04:00
W. Felix Handte
b6e24bc4dc Rename UTIL_getFileStat() -> UTIL_statFile() and UTIL_getDirectoryStat() -> UTIL_statDir() 
I want to introduce versions of many of these functions that take pre-
populated `stat_t` objects and use those rather than doing their own redundant
`stat()` internally. These functions will have `...Stat()` suffixes. So this
commit renames these existing functions into the active voice, to avoid
confusion.
 2020-08-05 00:40:16 -04:00
W. Felix Handte
1a1003f996 Mark stat_t Arg to UTIL_setFileStat() const 2020-08-05 00:35:21 -04:00
W. Felix Handte
5fbc6addb6 Additionally Convert UTIL_getFileStat() Calls to UTIL_stat() Where Appropriate 2020-08-05 00:31:48 -04:00
W. Felix Handte
69cb9e7798 Use New Stat Helper 2020-08-05 00:24:32 -04:00
W. Felix Handte
b11bea56a5 Introduce Dedicated Helper to Call stat() 2020-08-05 00:21:21 -04:00