158 Commits

Author SHA1 Message Date
Timon Kruiper
67e51311c3 fix behavior test with --test-evented-io on windows
also make simple file operations work asynchronously on windows
2020-03-27 17:03:06 +01:00
LeRoyce Pearson
113b217593 Merge branch 'master' into feature-file-locks 2020-03-23 21:39:16 -06:00
Andrew Kelley
46ffc798b6
fix swapped logic for Windows
Remove `std.fs.deleteTree`. Callers instead should use
`std.fs.cwd().deleteTree`.

Add `std.fs.deleteTreeAbsolute` for when the caller has an absolute
path.
2020-03-18 16:42:47 -04:00
LeRoyce Pearson
b773a8b175 Make fcntlFlock follow conventions of os.zig 2020-03-17 20:53:43 -06:00
LeRoyce Pearson
43ccc2d81e Add note about mandatory locks on linux 2020-03-14 10:12:46 -06:00
LeRoyce Pearson
a636b59cb5 Add lock option to CreateFlags 2020-03-14 10:12:46 -06:00
LeRoyce Pearson
3110060351 Add lock to fs.File.OpenFlags 2020-03-14 10:12:46 -06:00
LemonBoy
de53537f10 Add NtDll-based ftruncate implementation 2020-03-13 08:45:37 +01:00
LemonBoy
bd0b51477a Address review comments 2020-03-12 19:40:42 +01:00
LemonBoy
11df0d0cf8 std: Add setEndPos to fs.file
Allow the user to shrink/grow the file size as needed.
2020-03-12 09:43:45 +01:00
Andrew Kelley
d96b6c0d9f
fix footguns in File readAll functions 2020-03-11 13:06:30 -04:00
Andrew Kelley
ba0e3be5cf
(breaking) rework stream abstractions
The main goal here is to make the function pointers comptime, so that we
don't have to do the crazy stuff with async function frames.

Since InStream, OutStream, and SeekableStream are already generic
across error sets, it's not really worse to make them generic across the
vtable as well.

See #764 for the open issue acknowledging that using generics for these
abstractions is a design flaw.

See #130 for the efforts to make these abstractions non-generic.

This commit also changes the OutStream API so that `write` returns
number of bytes written, and `writeAll` is the one that loops until the
whole buffer is written.
2020-03-10 15:32:32 -04:00
LeRoyce Pearson
25d9ab95dd Use os.ino_t for everything
Also, define ino_t for windows
2020-03-08 21:52:36 -06:00
LeRoyce Pearson
cb84875eed Define ino_t for systems not yet defining it
Also, use ino_t instead of u64 in fs.File.INode
2020-03-08 18:31:18 -06:00
LeRoyce Pearson
e1c1ca9903 Add documentation about Stat.inode 2020-03-08 15:47:50 -06:00
LeRoyce Pearson
55077435bf Expose file inode (linux) and file index (windows) 2020-03-08 15:19:02 -06:00
Andrew Kelley
4a67dd04c9
breaking changes to std.fs, std.os
* improve `std.fs.AtomicFile` to use sendfile()
   - also fix AtomicFile cleanup not destroying tmp files under some
     error conditions
 * improve `std.fs.updateFile` to take advantage of the new `makePath`
   which no longer needs an Allocator.
 * rename std.fs.makeDir to std.fs.makeDirAbsolute
 * rename std.fs.Dir.makeDirC to std.fs.Dir.makeDirZ
 * add std.fs.Dir.makeDirW and provide Windows implementation of
   std.os.mkdirat. std.os.windows.CreateDirectory is now implemented
   by calling ntdll, supports an optional root directory handle,
   and returns an open directory handle. Its error set has a few more
   errors in it.
 * rename std.fs.Dir.changeTo to std.fs.Dir.setAsCwd
 * fix std.fs.File.writevAll and related functions when len 0 iovecs
   supplied.
 * introduce `std.fs.File.writeFileAll`, exposing a convenient
   cross-platform API on top of sendfile().
 * `NoDevice` added to std.os.MakeDirError error set.
 * std.os.fchdir gets a smaller error set.
 * std.os.windows.CloseHandle is implemented with ntdll call rather than
   kernel32.
2020-03-03 15:23:27 -05:00
daurnimator
1ca5f06762
Update callers of fs.makePath 2020-03-03 13:25:43 -05:00
Andrew Kelley
c81345c8ae
breaking: std.os read/write functions + sendfile
* rework os.sendfile and add macosx support, and a fallback
   implementation for any OS.
 * fix sendto compile error
 * std.os write functions support partial writes. closes #3443.
 * std.os pread / pwrite functions can now return `error.Unseekable`.
 * std.fs.File read/write functions now have readAll/writeAll variants
   which loop to complete operations even when partial reads/writes
   happen.
 * Audit std.os read/write functions with respect to Linux returning
   EINVAL for lengths greater than 0x7fff0000.
 * std.os read/write shim functions do not unnecessarily loop. Since
   partial reads/writes are part of the API, the caller will be forced
   to loop anyway, and so that would just be code bloat.
 * Improve doc comments
 * Add a non-trivial test for std.os.sendfile
 * Fix std.os.pread on 32 bit Linux
 * Add missing SYS_sendfile bit on aarch64
2020-03-03 02:25:26 -05:00
Andrew Kelley
4616af0ca4
introduce operating system version ranges as part of the target
* re-introduce `std.build.Target` which is distinct from `std.Target`.
   `std.build.Target` wraps `std.Target` so that it can be annotated as
   "the native target" or an explicitly specified target.
 * `std.Target.Os` is moved to `std.Target.Os.Tag`. The former is now a
   struct which has the tag as well as version range information.
 * `std.elf` gains some more ELF header constants.
 * `std.Target.parse` gains the ability to parse operating system
   version ranges as well as glibc version.
 * Added `std.Target.isGnuLibC()`.
 * self-hosted dynamic linker detection and glibc version detection.
   This also adds the improved logic using `/usr/bin/env` rather than
   invoking the system C compiler to find the dynamic linker when zig
   is statically linked. Related: #2084
   Note: this `/usr/bin/env` code is work-in-progress.
 * `-target-glibc` CLI option is removed in favor of the new `-target`
   syntax. Example: `-target x86_64-linux-gnu.2.27`

closes #1907
2020-02-28 14:51:53 -05:00
Andrew Kelley
0a88352689
fix behavior tests with --test-evented-io 2020-02-26 13:17:38 -05:00
xackus
783e8ad031 remove @bytesToSlice, @sliceToBytes from std lib 2020-02-21 19:46:53 +01:00
Andrew Kelley
4b02a39aa9
self-hosted libc detection
* libc_installation.cpp is deleted.
   src-self-hosted/libc_installation.zig is now used for both stage1 and
   stage2 compilers.
 * (breaking) move `std.fs.File.access` to `std.fs.Dir.access`. The API
   now encourages use with an open directory handle.
 * Add `std.os.faccessat` and related functions.
 * Deprecate the "C" suffix naming convention for null-terminated
   parameters. "C" should be used when it is related to libc. However
   null-terminated parameters often have to do with the native system
   ABI rather than libc. "Z" suffix is the new convention. For example,
   `std.os.openC` is deprecated in favor of `std.os.openZ`.
 * Add `std.mem.dupeZ` for using an allocator to copy memory and add a
   null terminator.
 * Remove dead struct field `std.ChildProcess.llnode`.
 * Introduce `std.event.Batch`. This API allows expressing concurrency
   without forcing code to be async. It requires no Allocator and does
   not introduce any failure conditions. However it is not thread-safe.
 * There is now an ongoing experiment to transition away from
   `std.event.Group` in favor of `std.event.Batch`.
 * `std.os.execvpeC` calls `getenvZ` rather than `getenv`. This is
   slightly more efficient on most systems, and works around a
   limitation of `getenv` lack of integration with libc.
 * (breaking) `std.os.AccessError` gains `FileBusy`, `SymLinkLoop`, and
   `ReadOnlyFileSystem`. Previously these error codes were all reported
   as `PermissionDenied`.
 * Add `std.Target.isDragonFlyBSD`.
 * stage2: access to the windows_sdk functions is done with a manually
   maintained .zig binding file instead of `@cImport`.
 * Update src-self-hosted/libc_installation.zig with all the
   improvements that stage1 has seen to src/libc_installation.cpp until
   now. In addition, it now takes advantage of Batch so that evented I/O
   mode takes advantage of concurrency, but it still works in blocking
   I/O mode, which is how it is used in stage1.
2020-02-16 13:25:30 -05:00
Benjamin Feng
699c50a375 Switch a bunch of FBA to use testing.allocator 2020-02-12 17:17:56 -06:00
Andrew Kelley
014f66e6de Merge pull request #4404 from ziglang/async-std
a big step towards std lib integration with async I/O
2020-02-10 00:22:59 -05:00
Andrew Kelley
5b10d9f917
std: fix bitrotted evented code 2020-02-08 16:24:26 -05:00
Andrew Kelley
4a60689309
more carefully calculate llvm field indexes
more correctly solves #4403
2020-02-07 14:54:58 -05:00
Andrew Kelley
3fce8008cc
skip self-hosted for now as we work towards async I/O
1. behavior tests with --test-evented-io
 2. std lib tests with --test-evented-io
 3. fuzz test evented I/O a bit, make it robust
 4. make sure it works on all platforms (kqueue, Windows IOCP,
    epoll/other)
 5. restart efforts on self-hosted
2020-02-07 12:30:16 -05:00
Andrew Kelley
0b5bcd2f56
more std lib async I/O integration
* `zig test` gainst `--test-evented-io` parameter and gains the ability
   to seamlessly run async tests.
 * `std.ChildProcess` opens its child process pipe with O_NONBLOCK when
   using evented I/O
 * `std.io.getStdErr()` gives a File that is blocking even in evented
   I/O mode.
 * Delete `std.event.fs`. The functionality is now merged into `std.fs`
   and async file system access (using a dedicated thread) is
   automatically handled.
 * `std.fs.File` can be configured to specify whether its handle is
   expected to block, and whether that is OK to block even when in
   async I/O mode. This makes async I/O work correctly for e.g. the
   file system as well as network.
 * `std.fs.File` has some deprecated functions removed.
 * Missing readv,writev,pread,pwrite,preadv,pwritev functions are added
   to `std.os` and `std.fs.File`. They are all integrated with async
   I/O.
 * `std.fs.Watch` is still bit rotted and needs to be audited in light
   of the new async/await syntax.
 * `std.io.OutStream` integrates with async I/O
 * linked list nodes in the std lib have default `null` values for
   `prev` and `next`.
 * Windows async I/O integration is enabled for reading/writing file
   handles.
 * Added `std.os.mode_t`. Integer sizes need to be audited.
 * Fixed #4403 which was causing compiler to crash.

This is working towards:

./zig test ../test/stage1/behavior.zig --test-evented-io

Which does not successfully build yet. I'd like to enable behavioral
tests and std lib tests with --test-evented-io in the test matrix in the
future, to prevent regressions.
2020-02-06 18:05:50 -05:00
LemonBoy
8c55c4550a std: Rename isAbsoluteW to isAbsoluteWindowsW 2020-02-05 20:39:14 +01:00
LemonBoy
3e7c02edc1 std: Fix edge case in isAbsolute fn family
* Empty strings are not considered absolute paths.
* Refactor some common code.

Closes #4382
2020-02-04 15:52:38 -05:00
daurnimator
7cf0b02ab4
NTSTATUS is a non-exhaustive enum 2020-01-31 22:33:17 +11:00
Benjamin Feng
b077f3ab7d Promoted "leak_count_allocator" to the main testing.allocator 2020-01-29 22:22:00 -06:00
Benjamin Feng
aa9caf5064 Create leak_count_allocator 2020-01-29 14:37:01 -06:00
Benjamin Feng
4d134a01f5 Move debug.global_allocator to testing.allocator 2020-01-29 12:21:29 -06:00
daurnimator
2662e50d27
std: sentinel terminated pointers for utf16 apis 2019-12-29 10:35:05 +11:00
Andrew Kelley
b36c07a95a
Merge remote-tracking branch 'origin/master' into remove-array-type-coercion 2019-12-01 09:56:01 -05:00
Andrew Kelley
034ccb4e4e add missing error code handling on Windows 2019-11-30 16:58:32 -05:00
Andrew Kelley
413f9a5cfc
move std.fs.Dir.cwd to std.fs.cwd
update to non-deprecated std.fs APIs throughout the codebase

Related: #3811
2019-11-30 15:35:27 -05:00
Andrew Kelley
d039fed831
introduce std.fs.Dir.openFile and std.fs.Dir.createFile
These functions have flags parameters which cover all the use cases. The
other functions are now deprecated.
2019-11-30 13:32:11 -05:00
Andrew Kelley
bf3ac66150
remove type coercion from array values to references
* Implements #3768. This is a sweeping breaking change that requires
   many (trivial) edits to Zig source code. Array values no longer
   coerced to slices; however one may use `&` to obtain a reference to
   an array value, which may then be coerced to a slice.

 * Adds `IrInstruction::dump`, for debugging purposes. It's useful to
   call to inspect the instruction when debugging Zig IR.

 * Fixes bugs with result location semantics. See the new behavior test
   cases, and compile error test cases.

 * Fixes bugs with `@typeInfo` not properly resolving const values.

 * Behavior tests are passing but std lib tests are not yet. There
   is more work to do before merging this branch.
2019-11-27 03:37:50 -05:00
Andrew Kelley
29e438fd1f
more sentinel-terminated pointers std lib integration
See #3767
2019-11-25 00:43:36 -05:00
Andrew Kelley
47f06be369
string literals are now null terminated
this also deletes C string literals from the language, and then makes
the std lib changes and compiler changes necessary to get the behavior
tests and std lib tests passing again.
2019-11-21 20:43:41 -05:00
Andrew Kelley
cd5f4de2a6
std: remove O_LARGEFILE from OS bits when the OS does not define it 2019-11-21 20:34:55 -05:00
Andrew Kelley
cd37c1a377
add std.fs.Dir.openRead
This is progress towards file system APIs that encourage avoiding
Time Of Check, Time Of Use bugs.
2019-11-21 17:07:29 -05:00
Andrew Kelley
e0db54e89d
update the codebase to use @as 2019-11-08 15:57:24 -05:00
Andrew Kelley
6ee3cabe5c
allow type coercion from *[0]T to E![]const T
This is an unambiguous, safe cast.
2019-11-06 20:44:15 -05:00
Andrew Kelley
61d5a0bf48
Merge branch 'std.net' 2019-10-30 21:30:16 -04:00
Tse
33cc204481 DragonFlyBSD support 2019-10-30 21:21:58 -04:00
Andrew Kelley
d7f15aa2cb
Merge remote-tracking branch 'origin/master' into std.net 2019-10-30 20:43:56 -04:00