43 Commits

Author SHA1 Message Date
Andrew Kelley
a3dfe36ca1 zir-to-elf skeleton 2020-04-22 23:42:58 -04:00
LeRoyce Pearson
317f06dc77 Add lock_nonblocking flag for creating or opening files
Also, make windows share delete access. Rationale: this is how it works
on Unix systems, mostly because locks are (usually) advisory on Unix.
2020-04-07 18:00:12 -06:00
LeRoyce Pearson
71c5aab3e7 Make lock option an enum
For some reason, this breaks file locking on windows. Not sure if this is
a problem with wine.
2020-04-07 16:49:30 -06:00
LeRoyce Pearson
ea6525797d Use flock instead of fcntl to lock files
`flock` locks based on the file handle, instead of the process id.
This brings the file locking on unix based systems closer to file
locking on Windows.
2020-04-02 22:57:02 -06:00
LeRoyce Pearson
35c462caf0 Merge branch 'master' into feature-file-locks 2020-04-02 21:46:48 -06:00
Andrew Kelley
9e7ae06249
std lib API deprecations for the upcoming 0.6.0 release
See #3811
2020-03-30 14:23:22 -04:00
LeRoyce Pearson
b773a8b175 Make fcntlFlock follow conventions of os.zig 2020-03-17 20:53:43 -06:00
LeRoyce Pearson
43ccc2d81e Add note about mandatory locks on linux 2020-03-14 10:12:46 -06:00
LeRoyce Pearson
a636b59cb5 Add lock option to CreateFlags 2020-03-14 10:12:46 -06:00
LeRoyce Pearson
3110060351 Add lock to fs.File.OpenFlags 2020-03-14 10:12:46 -06:00
LemonBoy
de53537f10 Add NtDll-based ftruncate implementation 2020-03-13 08:45:37 +01:00
LemonBoy
bd0b51477a Address review comments 2020-03-12 19:40:42 +01:00
LemonBoy
11df0d0cf8 std: Add setEndPos to fs.file
Allow the user to shrink/grow the file size as needed.
2020-03-12 09:43:45 +01:00
Andrew Kelley
d96b6c0d9f
fix footguns in File readAll functions 2020-03-11 13:06:30 -04:00
Andrew Kelley
ba0e3be5cf
(breaking) rework stream abstractions
The main goal here is to make the function pointers comptime, so that we
don't have to do the crazy stuff with async function frames.

Since InStream, OutStream, and SeekableStream are already generic
across error sets, it's not really worse to make them generic across the
vtable as well.

See #764 for the open issue acknowledging that using generics for these
abstractions is a design flaw.

See #130 for the efforts to make these abstractions non-generic.

This commit also changes the OutStream API so that `write` returns
number of bytes written, and `writeAll` is the one that loops until the
whole buffer is written.
2020-03-10 15:32:32 -04:00
LeRoyce Pearson
25d9ab95dd Use os.ino_t for everything
Also, define ino_t for windows
2020-03-08 21:52:36 -06:00
LeRoyce Pearson
cb84875eed Define ino_t for systems not yet defining it
Also, use ino_t instead of u64 in fs.File.INode
2020-03-08 18:31:18 -06:00
LeRoyce Pearson
e1c1ca9903 Add documentation about Stat.inode 2020-03-08 15:47:50 -06:00
LeRoyce Pearson
55077435bf Expose file inode (linux) and file index (windows) 2020-03-08 15:19:02 -06:00
Andrew Kelley
4a67dd04c9
breaking changes to std.fs, std.os
* improve `std.fs.AtomicFile` to use sendfile()
   - also fix AtomicFile cleanup not destroying tmp files under some
     error conditions
 * improve `std.fs.updateFile` to take advantage of the new `makePath`
   which no longer needs an Allocator.
 * rename std.fs.makeDir to std.fs.makeDirAbsolute
 * rename std.fs.Dir.makeDirC to std.fs.Dir.makeDirZ
 * add std.fs.Dir.makeDirW and provide Windows implementation of
   std.os.mkdirat. std.os.windows.CreateDirectory is now implemented
   by calling ntdll, supports an optional root directory handle,
   and returns an open directory handle. Its error set has a few more
   errors in it.
 * rename std.fs.Dir.changeTo to std.fs.Dir.setAsCwd
 * fix std.fs.File.writevAll and related functions when len 0 iovecs
   supplied.
 * introduce `std.fs.File.writeFileAll`, exposing a convenient
   cross-platform API on top of sendfile().
 * `NoDevice` added to std.os.MakeDirError error set.
 * std.os.fchdir gets a smaller error set.
 * std.os.windows.CloseHandle is implemented with ntdll call rather than
   kernel32.
2020-03-03 15:23:27 -05:00
Andrew Kelley
c81345c8ae
breaking: std.os read/write functions + sendfile
* rework os.sendfile and add macosx support, and a fallback
   implementation for any OS.
 * fix sendto compile error
 * std.os write functions support partial writes. closes #3443.
 * std.os pread / pwrite functions can now return `error.Unseekable`.
 * std.fs.File read/write functions now have readAll/writeAll variants
   which loop to complete operations even when partial reads/writes
   happen.
 * Audit std.os read/write functions with respect to Linux returning
   EINVAL for lengths greater than 0x7fff0000.
 * std.os read/write shim functions do not unnecessarily loop. Since
   partial reads/writes are part of the API, the caller will be forced
   to loop anyway, and so that would just be code bloat.
 * Improve doc comments
 * Add a non-trivial test for std.os.sendfile
 * Fix std.os.pread on 32 bit Linux
 * Add missing SYS_sendfile bit on aarch64
2020-03-03 02:25:26 -05:00
Andrew Kelley
4616af0ca4
introduce operating system version ranges as part of the target
* re-introduce `std.build.Target` which is distinct from `std.Target`.
   `std.build.Target` wraps `std.Target` so that it can be annotated as
   "the native target" or an explicitly specified target.
 * `std.Target.Os` is moved to `std.Target.Os.Tag`. The former is now a
   struct which has the tag as well as version range information.
 * `std.elf` gains some more ELF header constants.
 * `std.Target.parse` gains the ability to parse operating system
   version ranges as well as glibc version.
 * Added `std.Target.isGnuLibC()`.
 * self-hosted dynamic linker detection and glibc version detection.
   This also adds the improved logic using `/usr/bin/env` rather than
   invoking the system C compiler to find the dynamic linker when zig
   is statically linked. Related: #2084
   Note: this `/usr/bin/env` code is work-in-progress.
 * `-target-glibc` CLI option is removed in favor of the new `-target`
   syntax. Example: `-target x86_64-linux-gnu.2.27`

closes #1907
2020-02-28 14:51:53 -05:00
Andrew Kelley
0a88352689
fix behavior tests with --test-evented-io 2020-02-26 13:17:38 -05:00
Andrew Kelley
4b02a39aa9
self-hosted libc detection
* libc_installation.cpp is deleted.
   src-self-hosted/libc_installation.zig is now used for both stage1 and
   stage2 compilers.
 * (breaking) move `std.fs.File.access` to `std.fs.Dir.access`. The API
   now encourages use with an open directory handle.
 * Add `std.os.faccessat` and related functions.
 * Deprecate the "C" suffix naming convention for null-terminated
   parameters. "C" should be used when it is related to libc. However
   null-terminated parameters often have to do with the native system
   ABI rather than libc. "Z" suffix is the new convention. For example,
   `std.os.openC` is deprecated in favor of `std.os.openZ`.
 * Add `std.mem.dupeZ` for using an allocator to copy memory and add a
   null terminator.
 * Remove dead struct field `std.ChildProcess.llnode`.
 * Introduce `std.event.Batch`. This API allows expressing concurrency
   without forcing code to be async. It requires no Allocator and does
   not introduce any failure conditions. However it is not thread-safe.
 * There is now an ongoing experiment to transition away from
   `std.event.Group` in favor of `std.event.Batch`.
 * `std.os.execvpeC` calls `getenvZ` rather than `getenv`. This is
   slightly more efficient on most systems, and works around a
   limitation of `getenv` lack of integration with libc.
 * (breaking) `std.os.AccessError` gains `FileBusy`, `SymLinkLoop`, and
   `ReadOnlyFileSystem`. Previously these error codes were all reported
   as `PermissionDenied`.
 * Add `std.Target.isDragonFlyBSD`.
 * stage2: access to the windows_sdk functions is done with a manually
   maintained .zig binding file instead of `@cImport`.
 * Update src-self-hosted/libc_installation.zig with all the
   improvements that stage1 has seen to src/libc_installation.cpp until
   now. In addition, it now takes advantage of Batch so that evented I/O
   mode takes advantage of concurrency, but it still works in blocking
   I/O mode, which is how it is used in stage1.
2020-02-16 13:25:30 -05:00
Andrew Kelley
4a60689309
more carefully calculate llvm field indexes
more correctly solves #4403
2020-02-07 14:54:58 -05:00
Andrew Kelley
0b5bcd2f56
more std lib async I/O integration
* `zig test` gainst `--test-evented-io` parameter and gains the ability
   to seamlessly run async tests.
 * `std.ChildProcess` opens its child process pipe with O_NONBLOCK when
   using evented I/O
 * `std.io.getStdErr()` gives a File that is blocking even in evented
   I/O mode.
 * Delete `std.event.fs`. The functionality is now merged into `std.fs`
   and async file system access (using a dedicated thread) is
   automatically handled.
 * `std.fs.File` can be configured to specify whether its handle is
   expected to block, and whether that is OK to block even when in
   async I/O mode. This makes async I/O work correctly for e.g. the
   file system as well as network.
 * `std.fs.File` has some deprecated functions removed.
 * Missing readv,writev,pread,pwrite,preadv,pwritev functions are added
   to `std.os` and `std.fs.File`. They are all integrated with async
   I/O.
 * `std.fs.Watch` is still bit rotted and needs to be audited in light
   of the new async/await syntax.
 * `std.io.OutStream` integrates with async I/O
 * linked list nodes in the std lib have default `null` values for
   `prev` and `next`.
 * Windows async I/O integration is enabled for reading/writing file
   handles.
 * Added `std.os.mode_t`. Integer sizes need to be audited.
 * Fixed #4403 which was causing compiler to crash.

This is working towards:

./zig test ../test/stage1/behavior.zig --test-evented-io

Which does not successfully build yet. I'd like to enable behavioral
tests and std lib tests with --test-evented-io in the test matrix in the
future, to prevent regressions.
2020-02-06 18:05:50 -05:00
daurnimator
7cf0b02ab4
NTSTATUS is a non-exhaustive enum 2020-01-31 22:33:17 +11:00
daurnimator
2662e50d27
std: sentinel terminated pointers for utf16 apis 2019-12-29 10:35:05 +11:00
Andrew Kelley
034ccb4e4e add missing error code handling on Windows 2019-11-30 16:58:32 -05:00
Andrew Kelley
413f9a5cfc
move std.fs.Dir.cwd to std.fs.cwd
update to non-deprecated std.fs APIs throughout the codebase

Related: #3811
2019-11-30 15:35:27 -05:00
Andrew Kelley
d039fed831
introduce std.fs.Dir.openFile and std.fs.Dir.createFile
These functions have flags parameters which cover all the use cases. The
other functions are now deprecated.
2019-11-30 13:32:11 -05:00
Andrew Kelley
29e438fd1f
more sentinel-terminated pointers std lib integration
See #3767
2019-11-25 00:43:36 -05:00
Andrew Kelley
47f06be369
string literals are now null terminated
this also deletes C string literals from the language, and then makes
the std lib changes and compiler changes necessary to get the behavior
tests and std lib tests passing again.
2019-11-21 20:43:41 -05:00
Andrew Kelley
cd5f4de2a6
std: remove O_LARGEFILE from OS bits when the OS does not define it 2019-11-21 20:34:55 -05:00
Andrew Kelley
cd37c1a377
add std.fs.Dir.openRead
This is progress towards file system APIs that encourage avoiding
Time Of Check, Time Of Use bugs.
2019-11-21 17:07:29 -05:00
Andrew Kelley
e0db54e89d
update the codebase to use @as 2019-11-08 15:57:24 -05:00
Andrew Kelley
67058b9b70
basic DNS address resolution for linux without libc 2019-10-29 02:19:22 -04:00
Andrew Kelley
60cd11bd4b
get rid of std.os.foo.is_the_target
It had the downside of running all the comptime blocks and resolving
all the usingnamespaces of each system, when just trying to discover if
the current system is a particular one.

For Darwin, where it's nice to use `std.Target.current.isDarwin()`, this
demonstrates the utility that #425 would provide.
2019-10-24 01:14:52 -04:00
Jonathan Marler
ef62452363 Handle TERM=dumb for stdout/stderr 2019-10-23 11:44:39 -04:00
Andrew Kelley
ef67c49785
[wip] use NtDll APIs on Windows to implement std.fs.Dir 2019-10-21 14:18:01 -04:00
Andrew Kelley
09abd09ab8
add docs for std.fs.File.updateTimes 2019-10-16 18:13:40 -04:00
LemonBoy
6a549a7f0c Fix compilation of updateTimes on 32bit platforms
Add a test to avoid regressions.

Fixes #3412
2019-10-16 18:10:32 -04:00
Andrew Kelley
ed36dbbd9c
mv std/ lib/
that's all this commit does. further commits will fix cli flags and
such.

see #2221
2019-09-25 23:35:41 -04:00