1954 Commits

Author SHA1 Message Date
Andrew Kelley
d57d9448aa stage2 parsing: rework block statements AST memory layout
block statements are now directly following the Block AST node rather
than a singly linked list. This had negligible impact on performance:

throughput: 72.3 MiB/s => 72.7 MiB/s

however it greatly improves the API since the statements are laid out in
a flat array in memory.
2020-05-20 23:47:04 -04:00
Andrew Kelley
688aa114e4 Revert "stage2 parser: elide memcpy of large initialization lists"
This reverts commit 84df1d4f3d0312553f5a3857ed67042319c20846.

Not worth the complexity! Always memcpy initialization lists into the
arena.
2020-05-20 22:42:43 -04:00
Andrew Kelley
84df1d4f3d stage2 parser: elide memcpy of large initialization lists
throughput: 71.4 MiB/s => 72.9 MiB/s
2020-05-20 22:41:18 -04:00
Andrew Kelley
44aeb38328 make rand function public, fixes crypto benchmark 2020-05-20 21:27:08 -04:00
Andrew Kelley
897f23f20f stage2 parser: split off some SuffixOp AST nodes into separate tags
These SuffixOp nodes have their own ast.Node tags now:
 * ArrayInitializer
 * ArrayInitializerDot
 * StructInitializer
 * StructInitializerDot

Their sub-expression lists are general-purpose-allocator allocated
and then copied into the arena after completion of parsing.

throughput: 72.9 MiB/s => 74.4 MiB/s
maxrss: 68 KB => 72 KB

The API is also nicer since the sub expression lists are now flat arrays
instead of singly linked lists.
2020-05-20 19:18:14 -04:00
Andrew Kelley
7c2c0e36f8 stage2 parser: different memory layout of ParamDecl
Instead of being its own node, it's a struct inside FnProto.
Instead of FnProto having a SinglyLinkedList of ParamDecl nodes,
ParamDecls are appended directly in memory after the FnProto.

throughput: 72.2 MiB/s => 72.9 MiB/s
maxrss: 70 KB => 68 KB

Importantly, the API is improved as well since the data is arranged
linearly in memory.
2020-05-20 17:39:54 -04:00
Andrew Kelley
ad4b591db1 small bump to ArenaAllocator minimum alloc size
self-hosted parser benchmark:
throughput: 69.4 MiB/s => 72.2 MiB/s
2020-05-20 16:30:28 -04:00
Andrew Kelley
64c149ca19 fields and decls: ArrayList appending, memcpy to ast arena
This makes fields and decl ast nodes part of the Root and ContainerDecl
AST nodes.

Surprisingly, it's a performance regression from using a singly-linked
list for these nodes:

throughput: 76.5 MiB/s => 69.4 MiB/s

However it has much better memory usage:

maxrss: 392 KB => 77 KB

It's also better API for consumers of the parser, since it is a flat
list in memory.
2020-05-20 16:13:02 -04:00
Andrew Kelley
b1bcdc96ca arena allocator: smaller minimum allocation size 2020-05-20 16:11:55 -04:00
Andrew Kelley
8c10178a1e golly jeepers it's taking a long time to update translate-c 2020-05-20 13:53:53 -04:00
Andrew Kelley
4617c5907a fix zig fmt test regressions 2020-05-20 00:19:00 -04:00
Andrew Kelley
82225f6568 zig fmt parser tests are compiling again 2020-05-19 23:08:48 -04:00
Andrew Kelley
93384f7428 use singly linked lists for std.zig.parse
std.ast uses a singly linked list for lists of things. This is a
breaking change to the self-hosted parser API.

std.ast.Tree has been separated into a private "Parser" type which
represents in-progress parsing, and std.ast.Tree which has only
"output" data. This means cleaner, but breaking, API for parse results.
Specifically, `tokens` and `errors` are no longer SegmentedList but a
slice.

The way to iterate over AST nodes has necessarily changed since lists of
nodes are now singly linked lists rather than SegmentedList.

From these changes, I observe the following on the
self-hosted-parser benchmark from ziglang/gotta-go-fast:

throughput: 45.6 MiB/s => 55.6 MiB/s
maxrss: 359 KB => 342 KB

This commit breaks the build; more updates are necessary to fix API
usage of the self-hosted parser.
2020-05-19 21:22:52 -04:00
Andrew Kelley
1cde0edff4 json: properly stringify error sets
I did this wrong originally in
feade9ef0010b1b47d7216e786ed964d09612c2b
2020-05-18 19:23:40 -04:00
Jakub Konka
cd8daa533a Undo accidentally checked-in changes to fs/test.zig 2020-05-18 22:22:27 +02:00
Jakub Konka
57719006bb Always return false for ANSI escape codes compat in WASI 2020-05-18 21:17:49 +02:00
Jakub Konka
3d267bab71 Re-enable refAllDecls gen and check in std.zig 2020-05-18 21:05:29 +02:00
Jakub Konka
2a59ecd7ec Integrate getTestDir with tmpDir logic 2020-05-18 17:10:49 +02:00
Jakub Konka
f26ab568aa Remove obsolete runtime panics from sleep impl 2020-05-18 17:10:17 +02:00
Jakub Konka
5186711a96 Change to Self from *const Self/*Self where possible 2020-05-18 17:10:06 +02:00
Jakub Konka
fae4af9e1c Make mode_t a 0-byte type in WASI 2020-05-18 17:09:52 +02:00
Jakub Konka
d43c08a3e5 Add/fix missing WASI functionality to pass libstd tests
This rather large commit adds/fixes missing WASI functionality
in `libstd` needed to pass the `libstd` tests. As such, now by
default tests targeting `wasm32-wasi` target are enabled in
`test/tests.zig` module. However, they can be disabled by passing
the `-Dskip-wasi=true` flag when invoking the `zig build test`
command. When the flag is set to `false`, i.e., when WASI tests are
included, `wasmtime` with `--dir=.` is used as the default testing
command.

Since the majority of `libstd` tests were relying on `fs.cwd()`
call to get current working directory handle wrapped in `Dir`
struct, in order to make the tests WASI-friendly, `fs.cwd()`
call was replaced with `testing.getTestDir()` function which
resolved to either `fs.cwd()` for non-WASI targets, or tries to
fetch the preopen list from the WASI runtime and extract a
preopen for '.' path.

The summary of changes introduced by this commit:
* implement `Dir.makeDir` and `Dir.openDir` targeting WASI
* implement `Dir.deleteFile` and `Dir.deleteDir` targeting WASI
* fix `os.close` and map errors in `unlinkat`
* move WASI-specific `mkdirat` and `unlinkat` from `std.fs.wasi`
  to `std.os` module
* implement `lseek_{SET, CUR, END}` targeting WASI
* implement `futimens` targeting WASI
* implement `ftruncate` targeting WASI
* implement `readv`, `writev`, `pread{v}`, `pwrite{v}` targeting WASI
* make sure ANSI escape codes are _not_ used in stderr or stdout
  in WASI, as WASI always sanitizes stderr, and sanitizes stdout if
  fd is a TTY
* fix specifying WASI rights when opening/creating files/dirs
* tweak `AtomicFile` to be WASI-compatible
* implement `os.renameatWasi` for WASI-compliant `os.renameat` function
* implement sleep() targeting WASI
* fix `process.getEnvMap` targeting WASI
2020-05-18 16:09:49 +02:00
Andrew Kelley
feade9ef00 std.json: support stringify for anyerror 2020-05-17 17:07:27 -04:00
Veikka Tuominen
2a5c0ef7f0
Merge pull request #5164 from daurnimator/in_stream-helpers
A couple of helpers for streams that I've found helpful
2020-05-17 22:02:10 +03:00
Andrew Kelley
16f100b82e
Merge pull request #5307 from ziglang/self-hosted-incremental-compilation
rework self-hosted compiler for incremental builds
2020-05-17 13:53:27 -04:00
Vexu
28d449b38d
fix zig fmt regression 2020-05-17 15:13:19 +03:00
Vincent Rischmann
e5ffb94911 linux: remove duplicated fields 2020-05-17 02:42:59 -04:00
Andrew Kelley
f2ceb023f4
Merge pull request #5217 from wozeparrot/master
build.zig recursive dep support
2020-05-16 14:24:51 -04:00
Vexu
eda03354dc
Merge pull request #5358 from Vexu/parser
Fix infinite loop with invalid comptime
2020-05-16 20:49:37 +03:00
Andrew Kelley
cf34480f2a
Merge pull request #5231 from kubkon/wasi-preopens
Add mechanism for extracting preopens from the runtime
2020-05-16 12:49:08 -04:00
Felix (xq) Queißner
7dc29fdee1 Creates std.ascii.control_code that contains the C0 control codes as named constants. 2020-05-16 12:46:44 -04:00
data-man
6647c3f054 Fixes fmt padding for some types 2020-05-16 12:45:43 -04:00
Michael Dusan
e05923f34b macos: use realpath$DARWIN_EXTSN
macOS 10.15 Catalina causes an ancient variant of `realpath` to fail,
likely due to 10.15's evolving security model.

closes #5325
2020-05-16 12:41:53 -04:00
Andrew Kelley
79462bb591
Merge pull request #5354 from DrDeano/master
Add enum to addBuildOption
2020-05-16 12:40:06 -04:00
Vexu
081ffe24cf
fix infinite loop with invalid comptime 2020-05-16 19:23:59 +03:00
Vexu
74f7d710bb
Merge pull request #5032 from LakeByTheWoods/redo_translate_c
Translate C: Redo Add comment containing c source location for failed decls.
2020-05-16 17:37:19 +03:00
DrDeano
72b72faa0b
Add Enum Option 2020-05-16 15:36:18 +01:00
DrDeano
b2cb8beed9
Add enum to addBuildOptions 2020-05-16 15:35:42 +01:00
Vexu
d061e5854a
Merge pull request #5020 from jinroh/docs/search-perf
docs: search perf
2020-05-16 16:43:41 +03:00
Vexu
758de71d97
Merge pull request #5353 from Vexu/parser
Self-hosted parser fixes
2020-05-16 15:56:03 +03:00
Vexu
092f726eec
Merge pull request #5175 from daurnimator/multi-out-stream
std: add io.MultiOutStream
2020-05-16 13:43:50 +03:00
Vexu
e2a0f5741d
Merge pull request #5189 from zigazeljko/patch-1
Split AES struct into AESEncrypt and AESDecrypt
2020-05-16 13:20:14 +03:00
Vexu
a05011d403
move ParamType to Node.ParamDecl 2020-05-16 12:54:51 +03:00
Vexu
ed62081d38
recover from missing semicolon after if stmt 2020-05-16 12:29:01 +03:00
Vexu
6ca0def499
recover from invalid global error set access 2020-05-16 12:09:34 +03:00
Andrew Kelley
69a5f0d797 Merge remote-tracking branch 'origin/master' into self-hosted-incremental-compilation 2020-05-16 01:26:18 -04:00
Vexu
b2f16d4484
fix infinite loop with mismatced bracket 2020-05-15 23:50:47 +03:00
Vexu
a00fd6e254
properly handle extra closing braces at top level 2020-05-15 14:30:49 +03:00
Vexu
f8b99331a2
Merge pull request #5336 from Vexu/parser
Make self-hosted parser more error tolerant
2020-05-15 14:15:30 +03:00
Vexu
440189a04a
cleanup* remove unecessary error* properly handle ReturnTypeInvalid in ast.zig functions* assert that the tree is clean in render.zig* simplify parser recovery with top level decls 2020-05-15 11:35:37 +03:00