Commit Graph

691 Commits (bd547232bc0aa43591b9eabf51a78ae3cb10fdcf)

Author SHA1 Message Date
sen 698f261b35
[1.5.0] Deprecate some functions (#2582)
* Add deprecated macro to zstd.h, mark certain functions as deprecated

* Remove ZSTD_compress.c dependencies on deprecated functions
2021-05-06 17:59:32 -04:00
Felix Handte 909925785a
Merge pull request #2618 from felixhandte/single-file-build-mv
Move Single-File Build Script from `contrib/` to `build/`
2021-05-06 14:09:42 -04:00
sen d6be7659b0
Add seekable roundtrip fuzzer (#2617) 2021-05-06 10:08:21 -04:00
W. Felix Handte 1d65917323 Move Single-File Build Script from `contrib/` to `build/` 2021-05-05 16:07:51 -04:00
Azat Khuzhin 53a60e98de
seekable decompression fixes (#2594)
* seekable_format: fix from-file reading (not in-memory)

It tries to check the buffer boundary, but there is no buffer for
from-file reading.

* seekable_decompression: break when ZSTD_seekable_decompress() returns zero

* seekable_decompression_mem: break when ZSTD_seekable_decompress() returns zero

* seekable_format: cap the offset+len up to the last dOffset

This will allow to read the whole file w/o gotting corruption error if
the offset is more then the data left in file, i.e.:

    $ ./seekable_compression seekable_compression.c 8192 | head
    $ zstd -cdq seekable_compression.c.zst | wc -c
    4737

Before this patch:

    $ ./seekable_decompression seekable_compression.c.zst 0 10000000 | wc -c
    ZSTD_seekable_decompress() error : Corrupted block detected
    0

After:

    $ ./seekable_decompression seekable_compression.c.zst 0 10000000 | wc -c
    4737
2021-05-05 10:05:41 -04:00
Nick Terrell 09149beaf8 [1.5.0] Move `zstd_errors.h` and `zdict.h` to `lib/` root
`zstd_errors.h` and `zdict.h` are public headers, so they deserve to be
in the root `lib/` directory with `zstd.h`, not mixed in with our private
headers.
2021-04-30 15:13:54 -07:00
Nick Terrell fbb9006e18 [linux-kernel] Replace kernel-style comments
Replace kernel-style comments with regular comments.

E.g.

```
/** Before */

/* After */

/**
 * Before
 */

/*
 * After
 */

/***********************************
 * Before
 ***********************************/

/* *********************************
 * After
 ***********************************/
```
2021-04-29 15:50:23 -07:00
senhuang42 a423305e7b Remove ZBUFF tests 2021-04-19 17:27:05 -04:00
Nick Terrell d334ad2ff4 [contrib][linux-kernel] Add zstd_min_clevel() and zstd_max_clevel() 2021-03-30 10:30:59 -07:00
Nick Terrell a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:30:43 -07:00
Nick Terrell f8ac0ea7ef
Merge pull request #2539 from terrelln/linux-kernel-fixes
Fixes for the next linux kernel patch version
2021-03-24 10:34:29 -07:00
Nick Terrell cd1551d261 [lib][tracing] Add ZSTD_NO_TRACE macro
When defined, it disables tracing, and avoids including the header.
2021-03-16 11:47:27 -07:00
Nick Terrell 7222614a19 [contrib][freestanding] Remove tracing support
Remove tracing support from `freestanding.py` to keep things simple.
2021-03-16 11:47:27 -07:00
Nick Terrell e4b914e663 [contrib][linux] Expose zstd headers to avoid duplication
Expose the zstd headers in `include/linux` to avoid struct duplication.
This makes the member names not follow Kernel style guidelines, and
exposes the zstd symbols. But, the LMKL reviewers are okay with that.
2021-03-16 11:47:22 -07:00
Nick Terrell 49a9e070f5 [contrib][linux-kernel] Update test include stubs
Update the test include stubs so they are able to run the current zstd
version in the kernel, so I can compare stack usage.
2021-03-16 11:40:24 -07:00
Nick Terrell d2dd35ae65 [contrib][linux-kernel] Fix unaligned.h
Fix the `unaligned.h` shim in the tests that was causing corruption in
the tests. Note that this is a problem with the test shim, not the
kernel code.
2021-03-16 11:40:24 -07:00
Fotis Xenakis 3c6f5d5eca Fix seekable test to provide valid descriptor 2021-03-13 00:00:08 +02:00
Fotis Xenakis 21697b9c9e Fix seek table descriptor check when loading 2021-03-12 23:07:15 +02:00
Yann Collet 2fa4c8c405 added code comments for new API ZSTD_seekTable 2021-03-03 22:54:04 -08:00
Yann Collet 6e390ced1f Merge branch 'seekTable' of github.com:facebook/zstd into seekTable 2021-03-03 22:44:38 -08:00
Yann Collet 16ec1cf355 added test case for seekTable API
and simple roundtrip test
2021-03-03 18:56:23 -08:00
Yann Collet 713d4953f7 fixed gcc-7 conversion warning 2021-03-03 18:00:41 -08:00
Yann Collet 6c0bfc468c fixed wrong assert condition 2021-03-03 15:30:55 -08:00
Yann Collet a1d7b9d654 fixed gcc conversion warnings 2021-03-03 15:17:12 -08:00
Yann Collet 24d59a655d Merge branch 'dev' into seekTable 2021-03-03 15:08:40 -08:00
Yann Collet ac95a30455 various minor style fixes 2021-03-02 16:03:18 -08:00
Yann Collet 029f974ddc strengthen compilation flags 2021-03-02 15:43:52 -08:00
Yann Collet c7e42e147b fixed const guarantees
read-only objects are properly const-ified in parameters
2021-03-02 15:24:30 -08:00
Yann Collet a80b10f5e6 fix potential leak on exit 2021-03-02 15:03:37 -08:00
Sen Huang 527a20c3cd Fix seekable decompress hanging 2021-03-02 14:30:03 -08:00
Martin Lindsay 3cbdbb888b ZSTD_seekable_decompress() example that hangs. 2021-03-02 14:25:17 -08:00
Yann Collet ce6d1b9376
Merge pull request #2113 from mdittmer/expose-seek-table
[contrib] Support seek table-only API
2021-03-02 10:50:47 -08:00
Stephen Kitt adb54293ab
Stop using deprecated reset?Stream functions
These are replaced by the corresponding context resets. When
converting resetCStream, CCtx_setPledgedSrcSize isn't called if the
source size is "unknown".

This helps reduce the reliance on "static only" symbols, as well as
reducing the use of deprecated functions.

Signed-off-by: Stephen Kitt <steve@sk2.org>
2021-02-23 21:56:01 +01:00
Nick Terrell 54a4998a80 Add basic tracing functionality 2021-02-05 16:28:52 -08:00
Nick Terrell b45d22c851 [contrib][recovery] Add recovery_directory program
This program takes a file with concatenated zstd frames and splits the
file up by frame. E.g. if `dir.zst` has 4 frames:

```
> ./recover_directory dir.zst recovery/file
Recovering 4 files...
Recovered recovery/file0
Recovered recovery/file1
Recovered recovery/file2
Recovered recovery/file3
Complete
```
2021-01-15 08:45:22 -08:00
senhuang42 4f7584e7a3 Allow freestanding lib script regex to detect XXH64( 2021-01-07 12:29:12 -05:00
Nick Terrell 66e811d782 [license] Update year to 2021 2021-01-04 17:53:52 -05:00
Yann Collet 0b39531d75 moving all references to `release` branch
was previously `master`
2020-12-16 23:00:35 -08:00
sen f34d2f4192
Merge pull request #2408 from senhuang42/seekable_hang_fix
Remove possibility of hanging when using seekable decompression
2020-12-07 08:46:27 -05:00
senhuang42 26f89d47aa Clean up makefile for seekable tests 2020-12-03 09:25:04 -05:00
senhuang42 152b55879c Add unit tests to seekable 2020-12-02 15:33:12 -05:00
Nick Terrell 10f87d2f43 [contrib][linux] Add SPDX-License-Identifier 2020-12-02 12:00:44 -08:00
Nick Terrell d0cc2e3868 [contrib][linux] Fix make import-upstream 2020-12-02 11:43:28 -08:00
Nick Terrell 3256c00c6e [contrib][linux] Follow style guide more closely 2020-12-02 11:07:38 -08:00
Nick Terrell 2c570e7f87 [contrib][linux] Add a make target to import upstream 2020-12-02 10:42:33 -08:00
senhuang42 9db49a3989 Add a forward progress requirement bound to seekable streaming decompression 2020-12-02 12:24:16 -05:00
Nick Terrell 87b38156be [contrib][linux-kernel] Fix test warnings 2020-12-02 01:12:42 -08:00
Nick Terrell f58e9a9830 [contrib][linux] Add wrapper API
Add the kernel wrapper API. This keeps the same API and semantics as the
existing kernel API with name changes to be more kernel style and avoid
symbol collisions with zstd.
2020-12-02 01:08:08 -08:00
Nick Terrell 6a1e526ea7 [lib] Add ZSTD_COMPRESS_HEAPMODE tuning parameter 2020-09-24 19:42:04 -07:00
Nick Terrell b841387218 [freestanding] Improve macro resolution to handle #if X 2020-09-24 19:42:04 -07:00