This commit adds some unit tests for `std.fs.File.readAllAlloc`
function. It also updates the docs of `Reader.readNoEof`
which were outdated, and swaps `inStream()` for `reader()` in
`File.readAllAlloc` with the former being deprecated.
* `std.fs.Dir.Entry.Kind` is moved to `std.fs.File.Kind`
* `std.fs.File.Stat` gains the `kind` field, so performing a stat() on
a File now tells what kind of file it is. On Windows this only will
distinguish between directories and files.
* rework zig fmt logic so that in the case of opening a file and
discovering it to be a directory, it closes the file descriptor
before re-opening it with O_DIRECTORY, using fewer simultaneous open
file descriptors when walking a directory tree.
* rework zig fmt logic so that it pays attention to the kind of
directory entries, and when it sees a sub-directory it attempts to
open it as a directory rather than a file, reducing the number of
open() syscalls when walking a directory tree.
Start implementing https://github.com/ziglang/zig/issues/4917 which is to rename instream/outstream to reader/writer. This first change allows code to use Writer/writer instead of OutStream/outStream, but still maintains the old outstream names with "Deprecated" comments.
* change miscellaneous things to more idiomatic zig style
* change the digest length to 24 bytes instead of 48. This is
still 70 more bits than UUIDs. For an analysis of probability of
collisions, see:
https://en.wikipedia.org/wiki/Universally_unique_identifier#Collisions
* fix the API having the possibility of mismatched allocators
* fix some error paths to behave properly
* modify the guarantees about when file contents are loaded for input files
* pwrite instead of seek + write
* implement isProblematicTimestamp
* fix tests with regards to a working isProblematicTimestamp function.
this requires sleeping until the current timestamp becomes
unproblematic.
* introduce std.fs.File.INode, a cross platform type abstraction
so that cache hash implementation does not need to reach into std.os.
Remove the constants that assume a base unit in favor of explicit
x_per_y constants.
nanosecond calendar timestamps now use i128 for the type. This affects
fs.File.Stat, std.time.nanoTimestamp, and fs.File.updateTimes.
calendar timestamps are now signed, because the value can be less than
the epoch (the user can set their computer time to whatever they wish).
implement std.os.clock_gettime for Windows when clock id is
CLOCK_CALENDAR.
This rather large commit adds/fixes missing WASI functionality
in `libstd` needed to pass the `libstd` tests. As such, now by
default tests targeting `wasm32-wasi` target are enabled in
`test/tests.zig` module. However, they can be disabled by passing
the `-Dskip-wasi=true` flag when invoking the `zig build test`
command. When the flag is set to `false`, i.e., when WASI tests are
included, `wasmtime` with `--dir=.` is used as the default testing
command.
Since the majority of `libstd` tests were relying on `fs.cwd()`
call to get current working directory handle wrapped in `Dir`
struct, in order to make the tests WASI-friendly, `fs.cwd()`
call was replaced with `testing.getTestDir()` function which
resolved to either `fs.cwd()` for non-WASI targets, or tries to
fetch the preopen list from the WASI runtime and extract a
preopen for '.' path.
The summary of changes introduced by this commit:
* implement `Dir.makeDir` and `Dir.openDir` targeting WASI
* implement `Dir.deleteFile` and `Dir.deleteDir` targeting WASI
* fix `os.close` and map errors in `unlinkat`
* move WASI-specific `mkdirat` and `unlinkat` from `std.fs.wasi`
to `std.os` module
* implement `lseek_{SET, CUR, END}` targeting WASI
* implement `futimens` targeting WASI
* implement `ftruncate` targeting WASI
* implement `readv`, `writev`, `pread{v}`, `pwrite{v}` targeting WASI
* make sure ANSI escape codes are _not_ used in stderr or stdout
in WASI, as WASI always sanitizes stderr, and sanitizes stdout if
fd is a TTY
* fix specifying WASI rights when opening/creating files/dirs
* tweak `AtomicFile` to be WASI-compatible
* implement `os.renameatWasi` for WASI-compliant `os.renameat` function
* implement sleep() targeting WASI
* fix `process.getEnvMap` targeting WASI
* add TypedValue.Managed which represents a Type, a Value, and some
kind of memory management strategy.
* introduce an analysis queue
* flesh out how incremental compilation works with respect to exports
* ir.text.Module is only capable of one error message during parsing
* link.zig no longer has a decl table map and instead has structs that
exist directly on ir.Module.Decl and ir.Module.Export
* implement primitive .text block allocation
* implement linker code for updating Decls and Exports
* implement null Type
Some supporting std lib changes:
* add std.ArrayList.appendSliceAssumeCapacity
* add std.fs.File.copyRange and copyRangeAll
* fix std.HashMap having modification safety on in ReleaseSmall builds
* add std.HashMap.putAssumeCapacityNoClobber
Before it was possible for .intended_io_mode = .blocking,
.capable_io_mode = .evented, and then the implementation would put a
request on the fs thread, which is the wrong behavior. Now it always
calls the appropriate WriteFile/ReadFile function, passing the intended
io mode directly as a parameter.
This makes the behavior tests pass on Windows with --test-evented-io.
`flock` locks based on the file handle, instead of the process id.
This brings the file locking on unix based systems closer to file
locking on Windows.
The main goal here is to make the function pointers comptime, so that we
don't have to do the crazy stuff with async function frames.
Since InStream, OutStream, and SeekableStream are already generic
across error sets, it's not really worse to make them generic across the
vtable as well.
See #764 for the open issue acknowledging that using generics for these
abstractions is a design flaw.
See #130 for the efforts to make these abstractions non-generic.
This commit also changes the OutStream API so that `write` returns
number of bytes written, and `writeAll` is the one that loops until the
whole buffer is written.
* improve `std.fs.AtomicFile` to use sendfile()
- also fix AtomicFile cleanup not destroying tmp files under some
error conditions
* improve `std.fs.updateFile` to take advantage of the new `makePath`
which no longer needs an Allocator.
* rename std.fs.makeDir to std.fs.makeDirAbsolute
* rename std.fs.Dir.makeDirC to std.fs.Dir.makeDirZ
* add std.fs.Dir.makeDirW and provide Windows implementation of
std.os.mkdirat. std.os.windows.CreateDirectory is now implemented
by calling ntdll, supports an optional root directory handle,
and returns an open directory handle. Its error set has a few more
errors in it.
* rename std.fs.Dir.changeTo to std.fs.Dir.setAsCwd
* fix std.fs.File.writevAll and related functions when len 0 iovecs
supplied.
* introduce `std.fs.File.writeFileAll`, exposing a convenient
cross-platform API on top of sendfile().
* `NoDevice` added to std.os.MakeDirError error set.
* std.os.fchdir gets a smaller error set.
* std.os.windows.CloseHandle is implemented with ntdll call rather than
kernel32.
* rework os.sendfile and add macosx support, and a fallback
implementation for any OS.
* fix sendto compile error
* std.os write functions support partial writes. closes#3443.
* std.os pread / pwrite functions can now return `error.Unseekable`.
* std.fs.File read/write functions now have readAll/writeAll variants
which loop to complete operations even when partial reads/writes
happen.
* Audit std.os read/write functions with respect to Linux returning
EINVAL for lengths greater than 0x7fff0000.
* std.os read/write shim functions do not unnecessarily loop. Since
partial reads/writes are part of the API, the caller will be forced
to loop anyway, and so that would just be code bloat.
* Improve doc comments
* Add a non-trivial test for std.os.sendfile
* Fix std.os.pread on 32 bit Linux
* Add missing SYS_sendfile bit on aarch64
* re-introduce `std.build.Target` which is distinct from `std.Target`.
`std.build.Target` wraps `std.Target` so that it can be annotated as
"the native target" or an explicitly specified target.
* `std.Target.Os` is moved to `std.Target.Os.Tag`. The former is now a
struct which has the tag as well as version range information.
* `std.elf` gains some more ELF header constants.
* `std.Target.parse` gains the ability to parse operating system
version ranges as well as glibc version.
* Added `std.Target.isGnuLibC()`.
* self-hosted dynamic linker detection and glibc version detection.
This also adds the improved logic using `/usr/bin/env` rather than
invoking the system C compiler to find the dynamic linker when zig
is statically linked. Related: #2084
Note: this `/usr/bin/env` code is work-in-progress.
* `-target-glibc` CLI option is removed in favor of the new `-target`
syntax. Example: `-target x86_64-linux-gnu.2.27`
closes#1907
* libc_installation.cpp is deleted.
src-self-hosted/libc_installation.zig is now used for both stage1 and
stage2 compilers.
* (breaking) move `std.fs.File.access` to `std.fs.Dir.access`. The API
now encourages use with an open directory handle.
* Add `std.os.faccessat` and related functions.
* Deprecate the "C" suffix naming convention for null-terminated
parameters. "C" should be used when it is related to libc. However
null-terminated parameters often have to do with the native system
ABI rather than libc. "Z" suffix is the new convention. For example,
`std.os.openC` is deprecated in favor of `std.os.openZ`.
* Add `std.mem.dupeZ` for using an allocator to copy memory and add a
null terminator.
* Remove dead struct field `std.ChildProcess.llnode`.
* Introduce `std.event.Batch`. This API allows expressing concurrency
without forcing code to be async. It requires no Allocator and does
not introduce any failure conditions. However it is not thread-safe.
* There is now an ongoing experiment to transition away from
`std.event.Group` in favor of `std.event.Batch`.
* `std.os.execvpeC` calls `getenvZ` rather than `getenv`. This is
slightly more efficient on most systems, and works around a
limitation of `getenv` lack of integration with libc.
* (breaking) `std.os.AccessError` gains `FileBusy`, `SymLinkLoop`, and
`ReadOnlyFileSystem`. Previously these error codes were all reported
as `PermissionDenied`.
* Add `std.Target.isDragonFlyBSD`.
* stage2: access to the windows_sdk functions is done with a manually
maintained .zig binding file instead of `@cImport`.
* Update src-self-hosted/libc_installation.zig with all the
improvements that stage1 has seen to src/libc_installation.cpp until
now. In addition, it now takes advantage of Batch so that evented I/O
mode takes advantage of concurrency, but it still works in blocking
I/O mode, which is how it is used in stage1.
* `zig test` gainst `--test-evented-io` parameter and gains the ability
to seamlessly run async tests.
* `std.ChildProcess` opens its child process pipe with O_NONBLOCK when
using evented I/O
* `std.io.getStdErr()` gives a File that is blocking even in evented
I/O mode.
* Delete `std.event.fs`. The functionality is now merged into `std.fs`
and async file system access (using a dedicated thread) is
automatically handled.
* `std.fs.File` can be configured to specify whether its handle is
expected to block, and whether that is OK to block even when in
async I/O mode. This makes async I/O work correctly for e.g. the
file system as well as network.
* `std.fs.File` has some deprecated functions removed.
* Missing readv,writev,pread,pwrite,preadv,pwritev functions are added
to `std.os` and `std.fs.File`. They are all integrated with async
I/O.
* `std.fs.Watch` is still bit rotted and needs to be audited in light
of the new async/await syntax.
* `std.io.OutStream` integrates with async I/O
* linked list nodes in the std lib have default `null` values for
`prev` and `next`.
* Windows async I/O integration is enabled for reading/writing file
handles.
* Added `std.os.mode_t`. Integer sizes need to be audited.
* Fixed#4403 which was causing compiler to crash.
This is working towards:
./zig test ../test/stage1/behavior.zig --test-evented-io
Which does not successfully build yet. I'd like to enable behavioral
tests and std lib tests with --test-evented-io in the test matrix in the
future, to prevent regressions.
