139 Commits

Author SHA1 Message Date
Timon Kruiper
67e51311c3 fix behavior test with --test-evented-io on windows
also make simple file operations work asynchronously on windows
2020-03-27 17:03:06 +01:00
Jonathan S
cf4cbea88e Factor out arbitrary constant 12 to AtomicFile.RANDOM_BYTES 2020-03-26 15:07:40 -05:00
Jonathan S
a779a96d38 In AtomicFile, work relative to the destination's parent directory. This is more robust against concurrent filesystem reorganization and avoids path length issues. 2020-03-25 23:22:36 -05:00
Michael Dusan
d554070de1
self-hosted: use fs.selfExePathAlloc
- add fs.selfExePathAlloc
- use fs.selfExePathAlloc instead of fs.selfExeDirPathAlloc
- remove redundant code from fs.selfExeDirPath

closes #4634
2020-03-25 18:40:28 -04:00
LeRoyce Pearson
113b217593 Merge branch 'master' into feature-file-locks 2020-03-23 21:39:16 -06:00
LeRoyce Pearson
5b278fb606 Use locking open flags if they are defined 2020-03-23 20:59:09 -06:00
LemonBoy
761602e3e8 std: Use getdents on all the BSDs
* Use the correct versioned libc calls to avoid nasty surprises
2020-03-23 18:54:14 +01:00
Andrew Kelley
dc04e97098
Merge pull request #4752 from ziglang/slice-array
slicing with comptime start and end indexes results in pointer-to-array
2020-03-19 18:06:16 -04:00
Andrew Kelley
555a2c0328
(breaking) std.fs.copyFile now integrates with Dir
Removed:
 * `std.fs.updateFile`
 * `std.fs.updateFileMode`
 * `std.fs.copyFile`
 * `std.fs.copyFileMode`

Added:
 * `std.fs.Dir.copyFile`
 * `std.fs.copyFileAbsolute`
 * `std.fs.updateFileAbsolute`

Moved:
 * `std.fs.Dir.UpdateFileOptions` => `std.fs.CopyFileOptions`

Deprecated:
 * `std.fs.deleteDir`
 * `std.fs.deleteDirC`
 * `std.fs.deleteDirW`
 * `std.fs.readLink`
 * `std.fs.readLinkC`
2020-03-19 14:43:41 -04:00
Andrew Kelley
7fa88cc0a6
std lib fixups for new semantics
std lib tests are passing now
2020-03-19 09:53:55 -04:00
Andrew Kelley
46ffc798b6
fix swapped logic for Windows
Remove `std.fs.deleteTree`. Callers instead should use
`std.fs.cwd().deleteTree`.

Add `std.fs.deleteTreeAbsolute` for when the caller has an absolute
path.
2020-03-18 16:42:47 -04:00
Andrew Kelley
27affde592
(breaking) clarify openDir API
* remove deprecated `std.fs.Dir` APIs
 * `std.fs.Dir.openDir` now takes a options struct with bool fields for
   `access_sub_paths` and `iterate`. It's now much more clear how
   opening directories works.
 * fixed the std lib and various zig code calling the wrong openDir
   function.
 * the runtime safety check for dir flags is removed in favor of the
   cheaper option of putting a comment on the same line as handling
   EBADF / ACCESS_DENIED, since that will show up in stack traces.
2020-03-18 16:10:57 -04:00
LemonBoy
e15605e1c1
std: Safety check for iterate()
Calling iterate() on a Dir object returned by openDirTraverse is always
an error.
2020-03-18 13:45:52 -04:00
LeRoyce Pearson
613956cc47 Remove fcntlFlock and replace with plain fcntl 2020-03-17 21:02:19 -06:00
LeRoyce Pearson
b773a8b175 Make fcntlFlock follow conventions of os.zig 2020-03-17 20:53:43 -06:00
LeRoyce Pearson
819537d70a Skip file lock test in single threaded mode 2020-03-14 20:36:26 -06:00
LeRoyce Pearson
947abb7626 Fix compile error on windows 2020-03-14 17:13:46 -06:00
LeRoyce Pearson
49128c86f8 Extract run_lock_file_test 2020-03-14 15:31:54 -06:00
LeRoyce Pearson
7ab77685a8 Make lock tests more flexible 2020-03-14 14:57:56 -06:00
LeRoyce Pearson
873c7a59e9 Add multiple read lock test 2020-03-14 13:29:49 -06:00
LeRoyce Pearson
72eb9933fd Call std.os.waitpid instead of std.os.linux.waitpid 2020-03-14 11:34:38 -06:00
LeRoyce Pearson
43c4faba55 Add test to check that locking works 2020-03-14 10:12:46 -06:00
LeRoyce Pearson
a636b59cb5 Add lock option to CreateFlags 2020-03-14 10:12:46 -06:00
LeRoyce Pearson
4eff48b12e Add flock command paramter to os.fcntlFlock
Also, replace `os.fcntlFlockBlocking` with `os.fcntlFlock`
2020-03-14 10:12:46 -06:00
LeRoyce Pearson
f66a607607 Define Flock for all posix systems 2020-03-14 10:12:46 -06:00
LeRoyce Pearson
e1868029e9 Implement blocking file locking API for windows 2020-03-14 10:12:46 -06:00
LeRoyce Pearson
9af0590a28 Add fnctlFlock system call, use it to lock files 2020-03-14 10:12:46 -06:00
Andrew Kelley
66e76a0209
zig build system: correctly handle multiple output artifacts
Previously the zig build system incorrectly assumed that the only build
artifact was a binary. Now, when you enable the cache, only the output
dir is printed to stdout, and the zig build system iterates over the
files in that directory, copying them to the output directory.

To support this change:

 * Add `std.os.renameat`, `std.os.renameatZ`, and `std.os.renameatW`.
 * Fix `std.os.linux.renameat` not compiling due to typos.
 * Deprecate `std.fs.updateFile` and `std.fs.updateFileMode`.
 * Add `std.fs.Dir.updateFile`, which supports using open directory
   handles for both the source and destination paths, as well as an
   options parameter which allows overriding the mode.
 * Update `std.fs.AtomicFile` to support operating based on an open
   directory handle. Instead of `std.fs.AtomicFile.init`, use
   `std.fs.Dir.atomicFile`.
 * `std.fs.AtomicFile` deinit() better handles the situation when the
    rename fails but the temporary file still exists, by still
    attempting to remove the temporary file.
 * `std.fs.Dir.openFileWindows` is moved to `std.os.windows.OpenFileW`.
 * `std.os.RenameError` gains the error codes `NoDevice`,
   `SharingViolation`, and `PipeBusy` which have been observed from
   Windows.

Closes #4733
2020-03-13 21:22:08 -04:00
Andrew Kelley
d96b6c0d9f
fix footguns in File readAll functions 2020-03-11 13:06:30 -04:00
Andrew Kelley
ba0e3be5cf
(breaking) rework stream abstractions
The main goal here is to make the function pointers comptime, so that we
don't have to do the crazy stuff with async function frames.

Since InStream, OutStream, and SeekableStream are already generic
across error sets, it's not really worse to make them generic across the
vtable as well.

See #764 for the open issue acknowledging that using generics for these
abstractions is a design flaw.

See #130 for the efforts to make these abstractions non-generic.

This commit also changes the OutStream API so that `write` returns
number of bytes written, and `writeAll` is the one that loops until the
whole buffer is written.
2020-03-10 15:32:32 -04:00
Andrew Kelley
0720f338d4
add std.event.Loop pread and faccessat
progress towards std lib tests passing with evented I/O mode
2020-03-07 19:13:21 -05:00
Andrew Kelley
c25d9417d3
fix std.fs.makeDirAbsolute
closes #4671
2020-03-07 15:14:47 -05:00
Andrew Kelley
447a89cd4d
update self-hosted zig targets to print correct glibc availability 2020-03-04 01:05:42 -05:00
Andrew Kelley
4a67dd04c9
breaking changes to std.fs, std.os
* improve `std.fs.AtomicFile` to use sendfile()
   - also fix AtomicFile cleanup not destroying tmp files under some
     error conditions
 * improve `std.fs.updateFile` to take advantage of the new `makePath`
   which no longer needs an Allocator.
 * rename std.fs.makeDir to std.fs.makeDirAbsolute
 * rename std.fs.Dir.makeDirC to std.fs.Dir.makeDirZ
 * add std.fs.Dir.makeDirW and provide Windows implementation of
   std.os.mkdirat. std.os.windows.CreateDirectory is now implemented
   by calling ntdll, supports an optional root directory handle,
   and returns an open directory handle. Its error set has a few more
   errors in it.
 * rename std.fs.Dir.changeTo to std.fs.Dir.setAsCwd
 * fix std.fs.File.writevAll and related functions when len 0 iovecs
   supplied.
 * introduce `std.fs.File.writeFileAll`, exposing a convenient
   cross-platform API on top of sendfile().
 * `NoDevice` added to std.os.MakeDirError error set.
 * std.os.fchdir gets a smaller error set.
 * std.os.windows.CloseHandle is implemented with ntdll call rather than
   kernel32.
2020-03-03 15:23:27 -05:00
daurnimator
695b0976c3
std: move makePath to be a Dir method 2020-03-03 13:25:43 -05:00
daurnimator
a19a30bb17
std: move null byte check into toPosixPath
Note that windows NT paths *can* contain nulls
2020-03-03 13:25:43 -05:00
daurnimator
d8f966a04b
std: fix fs.makePath
The previous behaviour of using path.resolve has unexpected behaviour around symlinks.
This more simple implementation is more correct and doesn't require an allocator
2020-03-03 13:25:43 -05:00
daurnimator
627618a38d
std: add Dir.changeDir as wrapper around fchdir 2020-03-03 13:25:43 -05:00
daurnimator
dfb420e6d7
std: add fs.Dir.makeDir 2020-03-03 13:25:43 -05:00
Andrew Kelley
c81345c8ae
breaking: std.os read/write functions + sendfile
* rework os.sendfile and add macosx support, and a fallback
   implementation for any OS.
 * fix sendto compile error
 * std.os write functions support partial writes. closes #3443.
 * std.os pread / pwrite functions can now return `error.Unseekable`.
 * std.fs.File read/write functions now have readAll/writeAll variants
   which loop to complete operations even when partial reads/writes
   happen.
 * Audit std.os read/write functions with respect to Linux returning
   EINVAL for lengths greater than 0x7fff0000.
 * std.os read/write shim functions do not unnecessarily loop. Since
   partial reads/writes are part of the API, the caller will be forced
   to loop anyway, and so that would just be code bloat.
 * Improve doc comments
 * Add a non-trivial test for std.os.sendfile
 * Fix std.os.pread on 32 bit Linux
 * Add missing SYS_sendfile bit on aarch64
2020-03-03 02:25:26 -05:00
Andrew Kelley
4616af0ca4
introduce operating system version ranges as part of the target
* re-introduce `std.build.Target` which is distinct from `std.Target`.
   `std.build.Target` wraps `std.Target` so that it can be annotated as
   "the native target" or an explicitly specified target.
 * `std.Target.Os` is moved to `std.Target.Os.Tag`. The former is now a
   struct which has the tag as well as version range information.
 * `std.elf` gains some more ELF header constants.
 * `std.Target.parse` gains the ability to parse operating system
   version ranges as well as glibc version.
 * Added `std.Target.isGnuLibC()`.
 * self-hosted dynamic linker detection and glibc version detection.
   This also adds the improved logic using `/usr/bin/env` rather than
   invoking the system C compiler to find the dynamic linker when zig
   is statically linked. Related: #2084
   Note: this `/usr/bin/env` code is work-in-progress.
 * `-target-glibc` CLI option is removed in favor of the new `-target`
   syntax. Example: `-target x86_64-linux-gnu.2.27`

closes #1907
2020-02-28 14:51:53 -05:00
Andrew Kelley
0a88352689
fix behavior tests with --test-evented-io 2020-02-26 13:17:38 -05:00
Andrew Kelley
ec889d5888
NO_MEDIA_IN_DEVICE => return error.NoDevice 2020-02-20 12:39:46 -05:00
LemonBoy
ae16a6773f std: Handle NO_MEDIA_IN_DEVICE error in openFileWindows
Closes #4507
2020-02-20 13:25:45 +01:00
Andrew Kelley
5c54d7bee7
add missing implementations of libc installation to detect msvc paths 2020-02-16 19:58:27 -05:00
Andrew Kelley
8173fbfb66
implement os.faccessat for Windows 2020-02-16 17:10:43 -05:00
Andrew Kelley
4b02a39aa9
self-hosted libc detection
* libc_installation.cpp is deleted.
   src-self-hosted/libc_installation.zig is now used for both stage1 and
   stage2 compilers.
 * (breaking) move `std.fs.File.access` to `std.fs.Dir.access`. The API
   now encourages use with an open directory handle.
 * Add `std.os.faccessat` and related functions.
 * Deprecate the "C" suffix naming convention for null-terminated
   parameters. "C" should be used when it is related to libc. However
   null-terminated parameters often have to do with the native system
   ABI rather than libc. "Z" suffix is the new convention. For example,
   `std.os.openC` is deprecated in favor of `std.os.openZ`.
 * Add `std.mem.dupeZ` for using an allocator to copy memory and add a
   null terminator.
 * Remove dead struct field `std.ChildProcess.llnode`.
 * Introduce `std.event.Batch`. This API allows expressing concurrency
   without forcing code to be async. It requires no Allocator and does
   not introduce any failure conditions. However it is not thread-safe.
 * There is now an ongoing experiment to transition away from
   `std.event.Group` in favor of `std.event.Batch`.
 * `std.os.execvpeC` calls `getenvZ` rather than `getenv`. This is
   slightly more efficient on most systems, and works around a
   limitation of `getenv` lack of integration with libc.
 * (breaking) `std.os.AccessError` gains `FileBusy`, `SymLinkLoop`, and
   `ReadOnlyFileSystem`. Previously these error codes were all reported
   as `PermissionDenied`.
 * Add `std.Target.isDragonFlyBSD`.
 * stage2: access to the windows_sdk functions is done with a manually
   maintained .zig binding file instead of `@cImport`.
 * Update src-self-hosted/libc_installation.zig with all the
   improvements that stage1 has seen to src/libc_installation.cpp until
   now. In addition, it now takes advantage of Batch so that evented I/O
   mode takes advantage of concurrency, but it still works in blocking
   I/O mode, which is how it is used in stage1.
2020-02-16 13:25:30 -05:00
Andrew Kelley
014f66e6de Merge pull request #4404 from ziglang/async-std
a big step towards std lib integration with async I/O
2020-02-10 00:22:59 -05:00
Andrew Kelley
c48831512b
std lib typo fixups 2020-02-08 01:38:01 -05:00
Andrew Kelley
71873e7133
implement os.pipe2 for darwin 2020-02-07 11:28:42 -05:00