Remove the constants that assume a base unit in favor of explicit
x_per_y constants.
nanosecond calendar timestamps now use i128 for the type. This affects
fs.File.Stat, std.time.nanoTimestamp, and fs.File.updateTimes.
calendar timestamps are now signed, because the value can be less than
the epoch (the user can set their computer time to whatever they wish).
implement std.os.clock_gettime for Windows when clock id is
CLOCK_CALENDAR.
To prevent cache misses, token ids go in their own array, and the
start/end offsets go in a different one.
perf measurement before:
2,667,914 cache-misses:u
2,139,139,935 instructions:u
894,167,331 cycles:u
perf measurement after:
1,757,723 cache-misses:u
2,069,932,298 instructions:u
858,105,570 cycles:u
The DocComment AST node now only points to the first doc comment token.
API users are expected to iterate over the following tokens directly.
After this commit there are no more linked lists in use in the
self-hosted AST API.
Performance impact is negligible. Memory usage slightly reduced.
* Extract Call ast node tag out of SuffixOp; parameters go in memory
after Call.
* Demote AsmInput and AsmOutput from AST nodes to structs inside the
Asm node.
* The following ast nodes get their sub-node lists directly following
them in memory:
- ErrorSetDecl
- Switch
- BuiltinCall
* ast.Node.Asm gets slices for inputs, outputs, clobbers instead of
singly linked lists
Performance changes:
throughput: 72.7 MiB/s => 74.0 MiB/s
maxrss: 72 KB => 69 KB (nice)
block statements are now directly following the Block AST node rather
than a singly linked list. This had negligible impact on performance:
throughput: 72.3 MiB/s => 72.7 MiB/s
however it greatly improves the API since the statements are laid out in
a flat array in memory.
These SuffixOp nodes have their own ast.Node tags now:
* ArrayInitializer
* ArrayInitializerDot
* StructInitializer
* StructInitializerDot
Their sub-expression lists are general-purpose-allocator allocated
and then copied into the arena after completion of parsing.
throughput: 72.9 MiB/s => 74.4 MiB/s
maxrss: 68 KB => 72 KB
The API is also nicer since the sub expression lists are now flat arrays
instead of singly linked lists.
Instead of being its own node, it's a struct inside FnProto.
Instead of FnProto having a SinglyLinkedList of ParamDecl nodes,
ParamDecls are appended directly in memory after the FnProto.
throughput: 72.2 MiB/s => 72.9 MiB/s
maxrss: 70 KB => 68 KB
Importantly, the API is improved as well since the data is arranged
linearly in memory.
This makes fields and decl ast nodes part of the Root and ContainerDecl
AST nodes.
Surprisingly, it's a performance regression from using a singly-linked
list for these nodes:
throughput: 76.5 MiB/s => 69.4 MiB/s
However it has much better memory usage:
maxrss: 392 KB => 77 KB
It's also better API for consumers of the parser, since it is a flat
list in memory.
std.ast uses a singly linked list for lists of things. This is a
breaking change to the self-hosted parser API.
std.ast.Tree has been separated into a private "Parser" type which
represents in-progress parsing, and std.ast.Tree which has only
"output" data. This means cleaner, but breaking, API for parse results.
Specifically, `tokens` and `errors` are no longer SegmentedList but a
slice.
The way to iterate over AST nodes has necessarily changed since lists of
nodes are now singly linked lists rather than SegmentedList.
From these changes, I observe the following on the
self-hosted-parser benchmark from ziglang/gotta-go-fast:
throughput: 45.6 MiB/s => 55.6 MiB/s
maxrss: 359 KB => 342 KB
This commit breaks the build; more updates are necessary to fix API
usage of the self-hosted parser.
This rather large commit adds/fixes missing WASI functionality
in `libstd` needed to pass the `libstd` tests. As such, now by
default tests targeting `wasm32-wasi` target are enabled in
`test/tests.zig` module. However, they can be disabled by passing
the `-Dskip-wasi=true` flag when invoking the `zig build test`
command. When the flag is set to `false`, i.e., when WASI tests are
included, `wasmtime` with `--dir=.` is used as the default testing
command.
Since the majority of `libstd` tests were relying on `fs.cwd()`
call to get current working directory handle wrapped in `Dir`
struct, in order to make the tests WASI-friendly, `fs.cwd()`
call was replaced with `testing.getTestDir()` function which
resolved to either `fs.cwd()` for non-WASI targets, or tries to
fetch the preopen list from the WASI runtime and extract a
preopen for '.' path.
The summary of changes introduced by this commit:
* implement `Dir.makeDir` and `Dir.openDir` targeting WASI
* implement `Dir.deleteFile` and `Dir.deleteDir` targeting WASI
* fix `os.close` and map errors in `unlinkat`
* move WASI-specific `mkdirat` and `unlinkat` from `std.fs.wasi`
to `std.os` module
* implement `lseek_{SET, CUR, END}` targeting WASI
* implement `futimens` targeting WASI
* implement `ftruncate` targeting WASI
* implement `readv`, `writev`, `pread{v}`, `pwrite{v}` targeting WASI
* make sure ANSI escape codes are _not_ used in stderr or stdout
in WASI, as WASI always sanitizes stderr, and sanitizes stdout if
fd is a TTY
* fix specifying WASI rights when opening/creating files/dirs
* tweak `AtomicFile` to be WASI-compatible
* implement `os.renameatWasi` for WASI-compliant `os.renameat` function
* implement sleep() targeting WASI
* fix `process.getEnvMap` targeting WASI
* add TypedValue.Managed which represents a Type, a Value, and some
kind of memory management strategy.
* introduce an analysis queue
* flesh out how incremental compilation works with respect to exports
* ir.text.Module is only capable of one error message during parsing
* link.zig no longer has a decl table map and instead has structs that
exist directly on ir.Module.Decl and ir.Module.Export
* implement primitive .text block allocation
* implement linker code for updating Decls and Exports
* implement null Type
Some supporting std lib changes:
* add std.ArrayList.appendSliceAssumeCapacity
* add std.fs.File.copyRange and copyRangeAll
* fix std.HashMap having modification safety on in ReleaseSmall builds
* add std.HashMap.putAssumeCapacityNoClobber
* Add an upper case variant of `allocLowerString`
* Add case-sensitive variants of `eqlIgnoreCase`, `indexOfIgnoreCase`,
and `indexOfIgnoreCasePos`
* Add and update docstrings on functions
* introduce std.ArrayListUnmanaged for when you have the allocator
stored elsewhere
* move std.heap.ArenaAllocator implementation to its own file. extract
the main state into std.heap.ArenaAllocator.State, which can be
stored as an alternative to storing the entire ArenaAllocator, saving
24 bytes per ArenaAllocator on 64 bit targets.
* std.LinkedList.Node pointer field now defaults to being null
initialized.
* Rework self-hosted compiler Package API
* Delete almost all the bitrotted self-hosted compiler code. The only bit
rotted code left is in main.zig and compilation.zig
* Add call instruction to ZIR
* self-hosted compiler ir API and link API are reworked to support
a long-running compiler that incrementally updates declarations
* Introduce the concept of scopes to ZIR semantic analysis
* ZIR text format supports referencing named decls that are declared
later in the file
* Figure out how memory management works for the long-running compiler
and incremental compilation. The main roots are top level
declarations. There is a table of decls. The key is a cryptographic
hash of the fully qualified decl name. Each decl has an arena
allocator where all of the memory related to that decl is stored.
Each code block has its own arena allocator for the lifetime of
the block. Values that want to survive when going out of scope in
a block must get copied into the outer block. Finally, values must
get copied into the Decl arena to be long-lived.
* Delete the unused MemoryCell struct. Instead, comptime pointers are
based on references to Decl structs.
* Figure out how caching works. Each Decl will store a set of other
Decls which must be recompiled when it changes.
This branch is still work-in-progress; this commit breaks the build.
Previously, the path and preopens helpers were prototyped in `std.os.wasi`
module, but since they are higher-level abstraction over wasi, they belong in
`std.fs.wasi` module.
This commit removes `std.os.openatWasi` function, and renames it to
`std.os.wasi.openat`. Additionally, the added `PreopenList.findByPath`
method allows querying the list for a matching preopen by path.
This commit refactors `std.os.wasi.resolve_preopen` into a (higher-level)
`std.os.wasi.getPreopens` funtion which returns a slice with _all_
preopens at any given time. This fn allows the WASI module to
inquire at any given time for all preopens provided by the runtime.
This commit also makes `cwd()` a compile error on WASI.
This commit adds WASI specific impl of `std.fs.cwd()` in which we
emulate the `cwd` behaviour by inquiring the runtime for a "."
preopen if available. This is OK for simple relative ops, but will
not work for any ops which require absolute paths.
It seems that `std.os.openZ` is too POSIX-specific, so I think it
should not be a point of entry for WASI `open` call. I figure
WASI should be treated as a separate "os" that's _not_ POSIX
especially given the incoming changes in the ephemeral snapshot.
According to documentation ETIMEDOUT (110) is a valid error code for the read function. I just had my long-running (been running for about 7 weeks) network program crash because it did not handle the ETIMEDOUT error code from "read".
Before it was possible for .intended_io_mode = .blocking,
.capable_io_mode = .evented, and then the implementation would put a
request on the fs thread, which is the wrong behavior. Now it always
calls the appropriate WriteFile/ReadFile function, passing the intended
io mode directly as a parameter.
This makes the behavior tests pass on Windows with --test-evented-io.
Now there are 3 types:
* std.math.big.int.Const
- the memory is immutable, only stores limbs and is_positive
- all methods operating on constant data go here
* std.math.big.int.Mutable
- the memory is mutable, stores capacity in addition to limbs and
is_positive
- methods here have some Mutable parameters and some Const
parameters. These methods expect callers to pre-calculate the
amount of resources required, and asserts that the resources are
available.
* std.math.big.int.Managed
- the memory is mutable and additionally stores an allocator.
- methods here perform the resource calculations for the programmer.
- this is the high level abstraction from before
Each of these 3 types can be converted to the other ones.
You can see the use case for this in the self-hosted compiler, where we
only store limbs, and construct the big ints as needed.
This gets rid of the hack where the allocator was optional and the
notion of "fixed" versions of the struct. Such things are now modeled
with the `big.int.Const` type.
`wasi_snapshot_preview1` introduced a couple of ABI changes. This
commit fast-forwards the types and consts defined in `std.os.bits.wasi`
to match those changes.
This commit updates the WASI imports to use `wasi_snapshot_preview1`
instead of the old `wasi_unstable`. There are some minor ABI
differences between the two, however, the main motivator for using
the latest "stable" snapshot (aka preview1) is that, at least in
Wasmtime, there has been a lot of improvement work put into preview1
and unfortunately I might add, the improvements were not (in full)
backported to `wasi_unstable` snapshot.
Also, this commit removes the optional bound on the pointer to
`environ_get` syscall.
LeRoyce Pearson
data-man
Alexandros Naskos
Andrew Kelley
Jethro Nederhof
Greg Anders
Jakub Konka
Veikka Tuominen
Vincent Rischmann
Felix (xq) Queißner
Michael Dusan
DrDeano
Jason Merrill
Greg Anders
Jonathan Marler
Haze Booth
daurnimator
Josh Junon
Loris Cro
Jens Goldberg
Timon Kruiper
Tadeo Kondrak
StrangeBug
Cato
Chris Heyes
nycex
Cato Auestad
nia
wozeparrot
Jakub Konka