130 Commits

Author SHA1 Message Date
LeRoyce Pearson
af730c64bd Put base64 alphabet into a named constant 2020-05-25 13:48:43 -04:00
LeRoyce Pearson
8c8813a5cf Add filesystem base64 decoder 2020-05-25 13:48:43 -04:00
Jakub Konka
d43c08a3e5 Add/fix missing WASI functionality to pass libstd tests
This rather large commit adds/fixes missing WASI functionality
in `libstd` needed to pass the `libstd` tests. As such, now by
default tests targeting `wasm32-wasi` target are enabled in
`test/tests.zig` module. However, they can be disabled by passing
the `-Dskip-wasi=true` flag when invoking the `zig build test`
command. When the flag is set to `false`, i.e., when WASI tests are
included, `wasmtime` with `--dir=.` is used as the default testing
command.

Since the majority of `libstd` tests were relying on `fs.cwd()`
call to get current working directory handle wrapped in `Dir`
struct, in order to make the tests WASI-friendly, `fs.cwd()`
call was replaced with `testing.getTestDir()` function which
resolved to either `fs.cwd()` for non-WASI targets, or tries to
fetch the preopen list from the WASI runtime and extract a
preopen for '.' path.

The summary of changes introduced by this commit:
* implement `Dir.makeDir` and `Dir.openDir` targeting WASI
* implement `Dir.deleteFile` and `Dir.deleteDir` targeting WASI
* fix `os.close` and map errors in `unlinkat`
* move WASI-specific `mkdirat` and `unlinkat` from `std.fs.wasi`
  to `std.os` module
* implement `lseek_{SET, CUR, END}` targeting WASI
* implement `futimens` targeting WASI
* implement `ftruncate` targeting WASI
* implement `readv`, `writev`, `pread{v}`, `pwrite{v}` targeting WASI
* make sure ANSI escape codes are _not_ used in stderr or stdout
  in WASI, as WASI always sanitizes stderr, and sanitizes stdout if
  fd is a TTY
* fix specifying WASI rights when opening/creating files/dirs
* tweak `AtomicFile` to be WASI-compatible
* implement `os.renameatWasi` for WASI-compliant `os.renameat` function
* implement sleep() targeting WASI
* fix `process.getEnvMap` targeting WASI
2020-05-18 16:09:49 +02:00
Jakub Konka
07a968b344 Add docs 2020-05-05 17:05:30 +02:00
Jakub Konka
558bb24601 Move preopen and path wasi helpers to std.fs.wasi module
Previously, the path and preopens helpers were prototyped in `std.os.wasi`
module, but since they are higher-level abstraction over wasi, they belong in
`std.fs.wasi` module.
2020-05-05 15:08:52 +02:00
Jakub Konka
d4c33129cf Shuffle things around; add PreopenList.findByPath method
This commit removes `std.os.openatWasi` function, and renames it to
`std.os.wasi.openat`. Additionally, the added `PreopenList.findByPath`
method allows querying the list for a matching preopen by path.
2020-05-05 15:08:52 +02:00
Jakub Konka
b8112b3d17 Simplify File constructors 2020-05-05 15:08:52 +02:00
Jakub Konka
8bce1b6981 Refactor resolve_preopen into getPreopens function
This commit refactors `std.os.wasi.resolve_preopen` into a (higher-level)
`std.os.wasi.getPreopens` funtion which returns a slice with _all_
preopens at any given time. This fn allows the WASI module to
inquire at any given time for all preopens provided by the runtime.

This commit also makes `cwd()` a compile error on WASI.
2020-05-05 15:08:52 +02:00
Jakub Konka
05fb3e79fe Make std.fs.cwd() return preopen for "." if exists
This commit adds WASI specific impl of `std.fs.cwd()` in which we
emulate the `cwd` behaviour by inquiring the runtime for a "."
preopen if available. This is OK for simple relative ops, but will
not work for any ops which require absolute paths.
2020-05-05 15:08:52 +02:00
Jakub Konka
8e1cd69717 Implement std.fs.Dir.openFileWasi
It seems that `std.os.openZ` is too POSIX-specific, so I think it
should not be a point of entry for WASI `open` call. I figure
WASI should be treated as a separate "os" that's _not_ POSIX
especially given the incoming changes in the ephemeral snapshot.
2020-05-05 15:08:52 +02:00
Andrew Kelley
7998e2b0f4 Merge remote-tracking branch 'origin/master' into FireFox317-windows-evented-io 2020-05-02 14:16:59 -04:00
nycex
77376a54bf
correct usages of std.fs.dir.DeleteFileError (#5058)
* correct usages of std.fs.dir.DeleteFileError

* test std.fs.createFileAbsolute() and std.fs.deleteFileAbsolute()
2020-05-02 04:19:07 -04:00
Andrew Kelley
beebcbb677 Merge remote-tracking branch 'origin/master' into FireFox317-windows-evented-io 2020-05-02 01:53:24 -04:00
Andrew Kelley
43f7856bac fix regressions in windows std lib tests 2020-05-02 01:25:22 -04:00
Andrew Kelley
2272a07ca0 std.event.Loop: promote the fs thread to be available for all OS's 2020-05-02 00:41:19 -04:00
Andrew Kelley
45bce27b8f cleanup and fixes. behavior tests passing with evented I/O 2020-05-01 23:17:28 -04:00
Andrew Kelley
988031c07c Merge branch 'windows-evented-io' of https://github.com/FireFox317/zig into FireFox317-windows-evented-io 2020-05-01 19:02:16 -04:00
Andrew Kelley
2bae942800 add ZIR compare output test case to test suite 2020-05-01 06:47:20 -04:00
Andrew Kelley
a3dfe36ca1 zir-to-elf skeleton 2020-04-22 23:42:58 -04:00
Andrew Kelley
cc1c2bd568 simplify ZIR spec; separate parsing/rendering from analysis 2020-04-20 19:21:03 -04:00
Andrew Kelley
4ceaa0595a move fs tests to separate file; disable flaky test
See #5006
2020-04-11 17:50:38 -04:00
Andrew Kelley
a6e288d5fe
Merge pull request #4711 from leroycep/feature-file-locks
Add lock option to File.OpenFlags and File.CreateFlags
2020-04-10 15:00:45 -04:00
LeRoyce Pearson
5951211d3f Reduce file lock test sleep time 2020-04-08 18:03:52 -06:00
LeRoyce Pearson
d4161e1667 Close file1 in nonblocking lock test 2020-04-08 16:42:11 -06:00
LeRoyce Pearson
772bb1ade3 Disable open flock flags on darwin
The tests were put into a deadlock, and it seems that darwin doesn't
support `O_SYNC`, though it supports `O_NONBLOCK`. It shouldn't block
even with that, but I'm not sure why else it would fail.
2020-04-08 16:38:58 -06:00
LeRoyce Pearson
317f06dc77 Add lock_nonblocking flag for creating or opening files
Also, make windows share delete access. Rationale: this is how it works
on Unix systems, mostly because locks are (usually) advisory on Unix.
2020-04-07 18:00:12 -06:00
LeRoyce Pearson
71c5aab3e7 Make lock option an enum
For some reason, this breaks file locking on windows. Not sure if this is
a problem with wine.
2020-04-07 16:49:30 -06:00
Andrew Kelley
22dbeab29d
Merge pull request #4857 from LemonBoy/fix-4777
Rewrite the bound checks in slice operator
2020-04-07 14:51:25 -04:00
LeRoyce Pearson
28d71c97d1 Fix compile error on darwin 2020-04-06 23:19:39 -06:00
LeRoyce Pearson
20597c8596 Only call os.flock on systems that lack openat locks 2020-04-06 22:28:43 -06:00
LeRoyce Pearson
49886d2e45 Remove return value from os.flock() 2020-04-06 22:07:27 -06:00
LeRoyce Pearson
798207ec80 Merge branch 'master' into feature-file-locks 2020-04-06 21:51:57 -06:00
LemonBoy
aa5865b9be std: Fix oob slicing operator 2020-04-03 10:58:39 +02:00
LeRoyce Pearson
ea32a7d2bc Fix compile errors about adding error.FileLocksNotSupported 2020-04-03 00:27:34 -06:00
LeRoyce Pearson
733f1c25bd Fix compile errors in stage2 2020-04-02 23:39:25 -06:00
LeRoyce Pearson
ea6525797d Use flock instead of fcntl to lock files
`flock` locks based on the file handle, instead of the process id.
This brings the file locking on unix based systems closer to file
locking on Windows.
2020-04-02 22:57:02 -06:00
LeRoyce Pearson
35c462caf0 Merge branch 'master' into feature-file-locks 2020-04-02 21:46:48 -06:00
xackus
7a28c644aa new ArrayList API: fix everything else 2020-04-02 16:12:08 +02:00
Andrew Kelley
2e806682f4
(breaking) std.Buffer => std.ArrayListSentineled(u8, 0)
This new name (and the fact that it is a function returning a type) will
make it more clear which use cases are better suited for ArrayList and
which are better suited for ArrayListSentineled.

Also for consistency with ArrayList,
 * `append` => `appendSlice`
 * `appendByte` => `append`

Thanks daurnimator for pointing out the confusion of std.Buffer.
2020-04-01 13:30:07 -04:00
Andrew Kelley
9e7ae06249
std lib API deprecations for the upcoming 0.6.0 release
See #3811
2020-03-30 14:23:22 -04:00
LeRoyce Pearson
c7f4e68464
Merge branch 'master' into feature-file-locks 2020-03-27 11:32:33 -06:00
Timon Kruiper
67e51311c3 fix behavior test with --test-evented-io on windows
also make simple file operations work asynchronously on windows
2020-03-27 17:03:06 +01:00
Jonathan S
cf4cbea88e Factor out arbitrary constant 12 to AtomicFile.RANDOM_BYTES 2020-03-26 15:07:40 -05:00
Jonathan S
a779a96d38 In AtomicFile, work relative to the destination's parent directory. This is more robust against concurrent filesystem reorganization and avoids path length issues. 2020-03-25 23:22:36 -05:00
Michael Dusan
d554070de1
self-hosted: use fs.selfExePathAlloc
- add fs.selfExePathAlloc
- use fs.selfExePathAlloc instead of fs.selfExeDirPathAlloc
- remove redundant code from fs.selfExeDirPath

closes #4634
2020-03-25 18:40:28 -04:00
LeRoyce Pearson
113b217593 Merge branch 'master' into feature-file-locks 2020-03-23 21:39:16 -06:00
LeRoyce Pearson
5b278fb606 Use locking open flags if they are defined 2020-03-23 20:59:09 -06:00
LemonBoy
761602e3e8 std: Use getdents on all the BSDs
* Use the correct versioned libc calls to avoid nasty surprises
2020-03-23 18:54:14 +01:00
Andrew Kelley
dc04e97098
Merge pull request #4752 from ziglang/slice-array
slicing with comptime start and end indexes results in pointer-to-array
2020-03-19 18:06:16 -04:00
Andrew Kelley
555a2c0328
(breaking) std.fs.copyFile now integrates with Dir
Removed:
 * `std.fs.updateFile`
 * `std.fs.updateFileMode`
 * `std.fs.copyFile`
 * `std.fs.copyFileMode`

Added:
 * `std.fs.Dir.copyFile`
 * `std.fs.copyFileAbsolute`
 * `std.fs.updateFileAbsolute`

Moved:
 * `std.fs.Dir.UpdateFileOptions` => `std.fs.CopyFileOptions`

Deprecated:
 * `std.fs.deleteDir`
 * `std.fs.deleteDirC`
 * `std.fs.deleteDirW`
 * `std.fs.readLink`
 * `std.fs.readLinkC`
2020-03-19 14:43:41 -04:00