facebook
/
zstd
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5,887 Commits
2 Branches
0 Tags
28 MiB
Commit Graph

5887 Commits (b7421f8e6994fc8317b50993a0b5ccf5a843395a)

Author SHA1 Message Date
Yann Collet 0e5b447aaa
Merge pull request #1316 from facebook/coldDict 
Cold dictionary mitigation
 2018-09-14 10:37:46 -07:00
Yann Collet 5512400677 updated code comments, based on @terrelln review 2018-09-13 16:44:04 -07:00
Yann Collet d195eec97e fixed msan error 
cold dictionary is detected through a comparison with dictEnd,
which was not initialized at the beginning of first DCtx usage.
 2018-09-13 12:29:52 -07:00
Yann Collet 674dd21bd0 final parameter tuning 2018-09-12 17:25:34 -07:00
Yann Collet 419dfd4ea3 clean traces 2018-09-12 16:40:28 -07:00
Yann Collet 2618253da2 fixed PREFETCH() macro 
for corner cases and platforms without this instruction
 2018-09-12 16:15:37 -07:00
Yann Collet b287b8f7c6
Merge pull request #1315 from cmcginty/dev 
Add build define to manually enable -rdynamic LD flag
 2018-09-12 16:12:00 -07:00
Yann Collet 44d3b83bb1 conditional dict content prefetching 
based on nbSeq.
 2018-09-12 15:35:21 -07:00
Casey McGinty d019d15f2b
Add build define to manually enable -rdynamic LD flag 2018-09-12 14:00:08 -07:00
Yann Collet 7800d0235c
Merge pull request #1314 from terrelln/cpuid 
[libzstd] Fix cpu for MSFT ARM
 2018-09-12 13:30:16 -07:00
Yann Collet 5fb5ed3b31 adjust heuristic decisions 2018-09-12 12:32:09 -07:00
Nick Terrell f6daddf2db Also allow x86 2018-09-12 12:05:32 -07:00
Yann Collet c49ccbc8e7 largeNbDicts : can select a nb of blocks 
will automatically truncate or repeat input as needed,
to create the requested nb of blocks.
default: nb of files, eventually increased appropriately if blockSize is set
 2018-09-12 11:31:28 -07:00
Nick Terrell 1e0bac6a9c [libzstd] Fix cpu for MSFT ARM 
The `__cpuid()` and `__cpuidex()` intrinsics are only available
on x86 and x86_64.
 2018-09-12 10:35:16 -07:00
Yann Collet 4de344d505 added conditional prefetch 
depending on amount of work to do.
 2018-09-12 10:29:47 -07:00
Yann Collet cf5db388f3
Merge pull request #1311 from cmcginty/dev 
Print a stack trace on unexpected term signal (e.g. SIGABRT)
 2018-09-12 10:13:23 -07:00
Yann Collet 63a519dbf6 implemented first prefetch 
based on dictID.
dictContent is prefetched up to 32 KB
(no contentSize adaptation)
 2018-09-11 17:23:44 -07:00
Casey McGinty b703181f93
Disable -rdynamic LD option on Windows 2018-09-11 16:19:34 -07:00
Casey McGinty b9118ecdab
Update comments, and LD flag usage in Make 2018-09-11 14:49:47 -07:00
Casey McGinty 2224ecd718
Remove dead code and method name typo 2018-09-11 11:56:50 -07:00
Casey McGinty d4337b6f1d
Move ABRThandler func out of internal lib 2018-09-11 11:39:49 -07:00
Casey McGinty 2a3967b7c4
Refactor abort signal handling 
- Print signal name to term
- Add -rdynamic option to generate Linux symbol names in backtrace
- Raise default signal after handler to ensure program termination
 2018-09-11 10:40:45 -07:00
Yann Collet 51a246da82
Merge pull request #1310 from facebook/Dworkspace 
reduce DDict size by 2KB
 2018-09-10 13:11:11 -07:00
Yann Collet 3675ef4762 added comment about minimum size of FSE tables 
required for DDict creation,
which use this space as workspace during Hufman table building stage.
 2018-09-10 11:24:17 -07:00
Yann Collet 7ff74c89fa
Merge pull request #1307 from facebook/spec 
spec clarification
 2018-09-07 15:42:04 -07:00
Casey McGinty a06574fc97
Print a stack trace on unexpected term signal (e.g. SIGABRT) 
For OSX and Linux, add a signal handler to SIGABRT, SGIFPE, SIGILL,
SIGSEGV, and SIGBUS. When the program terminates unexpectedly the
handler will print the current stack to the terminal to help determine
the location of the failure.

On OSX the output will look like:

```
Stack trace:
4   zstd                                0x000000010927ed96 main + 16886
5   libdyld.dylib                       0x00007fff767d1015 start + 1
6   ???                                 0x0000000000000001 0x0 + 1
```

On Linux the output will look like:

```
Stack trace:
./zstd() [0x4b8e1b]
./zstd() [0x4b928a]
./zstd() [0x403dc2]
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7f5e0fbb0445]
./zstd() [0x405754]
```

As is, the code does not function on WIN32.

See also: https://oroboro.com/stack-trace-on-crash/
 2018-09-06 18:46:52 -07:00
Yann Collet 112c18d6d5
Merge pull request #1309 from facebook/islink 
fixed UTIL_isLink() for Mac OS-X
 2018-09-06 18:01:10 -07:00
Yann Collet f97ca36eab strengthened conditions for using workplace into fse table space 
ensure that the structure layout is as expected.
will trigger an error if it changes in the future.

Another solution would be to use a union,
this would be cleaner and get rid of these static asserts.

However, in order to keep the current code unmodified,
it would be necessary to use an un-named unions.
And apparently, un-named unions are only possible on "recent" compilers (C99+).
 2018-09-06 17:54:13 -07:00
Yann Collet b036b9a177 Merge branch 'islink' into Dworkspace 2018-09-06 17:33:27 -07:00
Yann Collet deb03d8a26 fixed UTIL_isLink() for Mac OS-X 2018-09-06 17:32:16 -07:00
Yann Collet 87406548f0 reduced DDict size, by -2KB 
corresponding to the removal of workspace
which is needed while building huffman table
and is now either present in DCtx,
or temporarily borrowed from available FSE table space.
 2018-09-06 17:07:53 -07:00
Yann Collet 79106d8de9
Merge pull request #1308 from facebook/codeofconduct 
added code of conduct
 2018-09-06 14:38:32 -07:00
Yann Collet 161ecddf3c added code of conduct 
as requested by Facebook OSS program.
 2018-09-06 11:20:39 -07:00
Yann Collet 50b216146f
Merge pull request #1304 from facebook/largeNbDicts 
contrib/largeNbDicts
 2018-09-06 09:50:56 -07:00
Yann Collet c57a856d64 fixed minor static analyzer warning 2018-09-05 14:33:51 -07:00
Yann Collet 55a8f84a2c spec clarification 
following #1305 comments from @ulikunitz
 2018-09-05 12:31:33 -07:00
Nick Terrell d492ef4e07
Merge pull request #1306 from jennifermliu/dev 
Change default f of FASTCOVER to 20
 2018-09-04 18:03:36 -07:00
Jennifer Liu 21721b75a3 Change default f to 20 2018-09-04 17:15:14 -07:00
Nick Terrell 7a02df8dbe
Merge pull request #1302 from jennifermliu/splitpoint 
Always use splitPoint=1.0 for non-optimize cover and fastcover and some minor changes on comments
 2018-09-04 16:59:41 -07:00
Yann Collet 1d487d587f updated documentation 2018-09-04 14:57:45 -07:00
Yann Collet 11b8b8c100 silenced false-positive scan-build warning 2018-08-31 10:01:06 -07:00
Yann Collet cc1f14a16b
Merge pull request #1303 from cmcginty/dev 
Zstandard - better `make list`
 2018-08-30 18:09:33 -07:00
Casey McGinty 986ead646b
Add optional description field support to "list" target 2018-08-30 17:00:17 -07:00
Yann Collet 0ff67511e6 fixed link order for old compilers 2018-08-30 16:43:28 -07:00
Yann Collet f76253bb70 minor : createDictionaryBuffer() can create dictionaries of different sizes 2018-08-30 16:24:44 -07:00
Yann Collet 39c55a118f fixed minor compatibility issues with older compilers 2018-08-30 16:00:57 -07:00
Jennifer Liu f87383507d Update comment about default dictionary builder 2018-08-30 15:46:39 -07:00
Jennifer Liu 944c9986e0 Update comment on default steps of cover and fastcover 2018-08-30 15:37:29 -07:00
Yann Collet 39ef91a599 -std=c99 for largeNbDicts 2018-08-30 14:59:23 -07:00
Jennifer Liu 16db0337b1 Always use splitPoint=1.0 for non-optimize cover and fastcover 2018-08-30 14:59:22 -07:00