2368 Commits

Author SHA1 Message Date
Andrew Kelley
f82b1831f7 std: handle sharing violation when deleting a file on windows 2020-09-15 15:24:13 -07:00
Andrew Kelley
5d8fec3d4c Merge remote-tracking branch 'origin/master' into stage2-zig-cc 2020-09-14 21:57:01 -07:00
Andrew Kelley
dffdb2844e track all TODO comments in BRANCH_TODO file
Before merging, do this for every item in the file:
 * solve the issue, or
 * convert the task to a github issue and update the comment
   to link to the issue (and remove "TODO" text from the comment).
Then delete the file.

Related: #363
2020-09-14 18:06:19 -07:00
Andrew Kelley
778bb4bc9c move std.cache_hash from std to stage2
The API is pretty specific to the implementationt details of the
self-hosted compiler. I don't want to have to independently support
and maintain this as part of the standard library, and be obligated
to not make breaking changes to it with changes to the implementation of
stage2.
2020-09-14 11:05:51 -07:00
Veikka Tuominen
d073836894
Merge pull request #6172 from tadeokondrak/@Type(.Union)
Implement @Type for Union
2020-09-14 16:43:49 +03:00
Andrew Kelley
c49435f76b
Merge pull request #6331 from mattnite/bpf-helper-defs
BPF: helper definitions
2020-09-14 04:00:00 -04:00
Andrew Kelley
2a8fc1a18e stage2: caching system integration & Module/Compilation splitting
* update to the new cache hash API
 * std.Target defaultVersionRange moves to std.Target.Os.Tag
 * std.Target.Os gains getVersionRange which returns a tagged union
 * start the process of splitting Module into Compilation and "zig
   module".
   - The parts of Module having to do with only compiling zig code are
     extracted into ZigModule.zig.
   - Next step is to rename Module to Compilation.
   - After that rename ZigModule back to Module.
 * implement proper cache hash usage when compiling C objects, and
   properly manage the file lock of the build artifacts.
 * make versions optional to match recent changes to master branch.
 * proper cache hash integration for compiling zig code
 * proper cache hash integration for linking even when not compiling zig
   code.
 * ELF LLD linking integrates with the caching system. A comment from
   the source code:

   Here we want to determine whether we can save time by not invoking LLD when the
   output is unchanged. None of the linker options or the object files that are being
   linked are in the hash that namespaces the directory we are outputting to. Therefore,
   we must hash those now, and the resulting digest will form the "id" of the linking
   job we are about to perform.
   After a successful link, we store the id in the metadata of a symlink named "id.txt" in
   the artifact directory. So, now, we check if this symlink exists, and if it matches
   our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD.

 * implement disable_c_depfile option
 * add tracy to a few more functions
2020-09-13 19:29:07 -07:00
Andrew Kelley
1baa56a25f std.cache_hash: break up the API and improve implementation
into smaller exposed components and expose all of them. This makes it
more flexible.

`*const Cache` is now passed in with an open manifest dir handle which
the caller is responsible for managing.

Expose some of the base64 stuff.

Extract the hash helper functions into `HashHelper` and add some more
methods such as addOptional and addListOfFiles.

Add `CacheHash.toOwnedLock` so that you can deinitialize everything
except the open file handle which represents the file system lock on the
build artifacts.

Use ArrayListUnmanaged, saving space per allocated CacheHash.

Avoid 1 memory allocation in hit() with a static buffer.

hit() returns a bool; caller code is responsible for calling final() in
either case. This is a simpler and easier to use API.

writeManifest() is no longer called from deinit() with errors ignored.
2020-09-13 18:04:52 -07:00
LemonBoy
61e9e82bdc std: Make the CRC32 calculation slightly faster
Speed up a little the slicing-by-8 code path by replacing the
(load+shift+xor)*4 sequence with a single u32 load plus a xor.

Before:

```
iterative:  1018 MiB/s [000000006c3b110d]
small keys:  1075 MiB/s [0035bf3dcac00000]
```

After:

```
iterative:  1114 MiB/s [000000006c3b110d]
small keys:  1324 MiB/s [0035bf3dcac00000]
```
2020-09-13 16:32:21 -04:00
LemonBoy
5e50d145d9 std: Limit the read/write size on Darwin
It turns out that the kernel won't read or write more than 0x7fffffff
bytes in a single call, failing with EINVAL when trying to do so.

Adjust the limit and curse whoever is responsible for this.

Closes #6332
2020-09-13 16:31:21 -04:00
Matt Knight
88dacd3b70
changed to opaque 2020-09-13 09:53:20 -07:00
Matt Knight
1afbf4fb21
removed all those kern aliases 2020-09-13 09:43:49 -07:00
Matt Knight
cb5f76bb1c
got booleans wrong 2020-09-13 09:24:56 -07:00
Matt Knight
b043a31889
added helper definitions 2020-09-12 23:45:35 -07:00
Andrew Kelley
af4cc20ce2 Merge remote-tracking branch 'origin/master' into stage2-zig-cc
Master branch added in the concept of library versioning being optional
to main.cpp. It will need to be re-added into this branch before merging
back into master.
2020-09-12 10:48:38 -07:00
Andrew Kelley
8374be1a1c std.cache_hash: bump up the base64 digest len to avoid '==' 2020-09-12 00:50:37 -07:00
LemonBoy
9fe4c89230 std: Add a gzip decoder 2020-09-11 20:02:41 -04:00
Andrew Kelley
1edf097871
Merge pull request #6323 from LemonBoy/fast-zlib
std: Make the DEFLATE decompression routine 3x faster
2020-09-11 17:25:18 -04:00
Andrew Kelley
1eaf0691f9
Merge pull request #6320 from ifreund/prctl
std: add prctl and securebits definitions for linux/C
2020-09-11 16:33:57 -04:00
Isaac Freund
1078810cef
std: add prctl definition when linking libc 2020-09-11 22:32:23 +02:00
Isaac Freund
b6f4601545
std: add securebits definitions for linux 2020-09-11 22:32:23 +02:00
Isaac Freund
744b73ab46
std: add prctl wrapper to std.os 2020-09-11 22:32:23 +02:00
Isaac Freund
7b961a876b
std: add prctl definition for linux 2020-09-11 22:32:22 +02:00
Andrew Kelley
42c32dbc7b
Merge pull request #6315 from ifreund/optional-shlib-version
std, stage1: make shared library versioning optional
2020-09-11 16:07:13 -04:00
LemonBoy
4b851c7255 Re-enable a compression test
It somehow got commented out...
2020-09-11 21:04:08 +02:00
LemonBoy
41f244bd2f std: Make the DEFLATE decompression routine 3x faster
A profiler run showed that the main bottleneck was the naive decoding of
the Huffman codes, replacing it with a nice trick borrowed by Zlib gave
a substantial speedup.
Replacing a `%` with a `and (mask-1)` gave another significant
improvement (yay for low hanging fruits).

A few numbers obtained by decompressing a 22M file:

Before:
```
./decompress  2,39s user 0,00s system 99% cpu 2,400 total
```

After:
```
./decompress  0,79s user 0,00s system 99% cpu 0,798 total
````
2020-09-11 21:00:15 +02:00
Isaac Freund
0a76a9bd73
std, stage1: make shared library versioning optional
This commit changes the behavior of stage1 to emit libfoo.so instead
of libfoo.so.0.0.0 when none of the --ver-major, --ver-minor, or
--ver-patch flags are set.

It also makes it possible to create unversioned shared libraries
using the zig build system, changing the version parameter of
addSharedLibrary() to a tagged union.
2020-09-11 10:46:37 +02:00
Isaac Freund
68bf29c31e std: allow overriding install dir of artifacts
This is necessary when, for example, writing a PAM module which should
be installed to lib/security/module_name.so.
2020-09-11 04:32:36 -04:00
Andrew Kelley
c41cd3e13a std.fs.File: fix typo in writeFileAll 2020-09-10 18:53:20 -07:00
Andrew Kelley
800c5de2ae update the stack trace test case for lines added to start.zig 2020-09-10 14:40:33 -07:00
Andrew Kelley
2315331d23
Merge pull request #5527 from shawnanastasio/ppc64le
Implement support for powerpc64{,le}
2020-09-10 15:56:27 -04:00
Christian Wesselhoeft
ea9b38c93c fs/file.zig: Update reader to use type alias
This is a minor cosmetic change which updates `reader` and `inStream`
to match `writer` and `outStream` below.
2020-09-10 15:42:24 -04:00
Isaac Freund
13b8c63834 std: complete C definitions for uid/gid setters
Also:
- correct the definition of setgid
- update the existing definitions to use the uid_t/gid_t types
2020-09-10 15:41:52 -04:00
Isaac Freund
bb9773f695 std: fix errorset of std.os.seteuid/setegid 2020-09-10 15:41:21 -04:00
Isaac Freund
486a774e5f std: add option to use single-threaded event loop
std.event.Loop does not yet work in single threaded builds. However,
using evented io on a single thread can be very convenient. This commit
allows settind @import("root").event_loop_mode to .single_threaded
in order to allow this without reimplementing the startup code in
start.zig
2020-09-10 15:40:38 -04:00
Isaac Freund
4170f3f77f std: fix bitrot in process.posixGetUserInfo() 2020-09-10 14:30:07 -04:00
Literally Void
78baa16da0 Fix issue #6303: iterating empty PriorityQueue crashes 2020-09-10 12:00:53 +03:00
Andrew Kelley
4056bb92e6 stage2: more progress moving zig cc to stage2
* std.cache_hash exposes Hasher type
 * std.cache_hash makes hasher_init a global const
 * std.cache_hash supports cloning so that clones can share the same
   open manifest dir handle as well as fork from shared hasher state
 * start to populate the cache_hash for stage2 builds
 * remove a footgun from std.cache_hash add function
 * get rid of std.Target.ObjectFormat.unknown
 * rework stage2 logic for resolving output artifact names by adding
   object_format as an optional parameter to std.zig.binNameAlloc
 * support -Denable-llvm in stage2 tests
 * Module supports the use case when there are no .zig files
 * introduce c_object_table and failed_c_objects to Module
 * propagate many new kinds of data from CLI into Module and into
   linker.Options
 * introduce -fLLVM, -fLLD, -fClang and their -fno- counterparts.
   closes #6251.
   - add logic for choosing when to use LLD or zig's self-hosted linker
 * stub code for implementing invoking Clang to build C objects
 * add -femit-h, -femit-h=foo, and -fno-emit-h CLI options
2020-09-09 09:28:05 -07:00
Jakub Konka
7d487a4162
Merge pull request #6289 from kubkon/macho
macho: fix bug where __text section would get overwritten
2020-09-09 09:00:24 +02:00
Zachary Meadows
edc40157eb Switch type of HashMap's count from usize to u32 (#6262) 2020-09-09 00:33:14 -04:00
Jakub Konka
9306dbd619
Fix bug where __text section would get overwritten
Fixes a bug where the last written load command would accidentally
override the beginning of the __text section. Also defines missing
MachO constants and relocation structs/enums.

Signed-off-by: Jakub Konka <kubkon@jakubkonka.com>
2020-09-08 20:55:06 +02:00
Mark Barbone
42b1b6be90 Add resize for arena allocator 2020-09-08 13:04:14 -04:00
Jens Goldberg
5bf3e54018 Add the Linux TCP socket options 2020-09-08 13:00:07 -04:00
Andrew Kelley
f6f0e09456
Merge pull request #6267 from mattnite/btf
BPF: add BTF
2020-09-08 12:59:31 -04:00
xackus
b8b68cb279 std: clean up bitrotten imports in crypto 2020-09-08 12:54:28 -04:00
Timon Kruiper
61cd9099e2 Zlib: Make deinit function public 2020-09-08 12:21:49 -04:00
LemonBoy
2328f40b7a std: Add DEFLATE and zlib decompressors 2020-09-07 14:59:46 -07:00
Matt Knight
a496f94be9 added map create, update, delete, and prog load 2020-09-07 15:42:41 -04:00
Matthew Knight
db7a238297
BPF: add some more documentation (#6268)
* added documentation for ringbuffers, which context type maps to which program type, and added some formatting
2020-09-07 15:41:29 -04:00
Andrew Kelley
f96f3265b5
Merge pull request #6265 from mattnite/bpf-finish-insns
BPF: Implement rest of pseudo instructions
2020-09-07 15:39:42 -04:00