Commit Graph

6363 Commits (87f632b08a04f4a03facfa220df1cbc466a5de39)

Author SHA1 Message Date
Andrew Kelley 87f632b08a fs.Dir.openDir: use empty object name for "." on Windows 2019-10-21 19:19:49 -04:00
Andrew Kelley ef67c49785
[wip] use NtDll APIs on Windows to implement std.fs.Dir 2019-10-21 14:18:01 -04:00
Andrew Kelley 5b1a492012
breaking: improve std.fs directory handling API
* Added `std.c.unlinkat` and `std.os.unlinkat`.
 * Removed `std.fs.MAX_BUF_BYTES` (this declaration never made it to
   master branch)
 * Added `std.fs.Dir.deleteTree` to be used on an open directory handle.
 * `std.fs.deleteTree` has better behavior for both relative and
   absolute paths. For absolute paths, it opens the base directory
   and uses that handle for subsequent operations. For relative paths,
   it does a similar strategy, using the cwd handle.
 * The error set of `std.fs.deleteTree` is improved to no longer have
   these possible errors:
   - OutOfMemory
   - FileTooBig
   - IsDir
   - DirNotEmpty
   - PathAlreadyExists
   - NoSpaceLeft
 * Added `std.fs.Dir.posix_cwd` which is a statically initialized
   directory representing the current working directory.
 * The error set of `std.Dir.open` is improved to no longer have these
   possible errors:
   - FileTooBig
   - IsDir
   - NoSpaceLeft
   - PathAlreadyExists
   - OutOfMemory
 * Added more alternative functions to `std.fs` for when the path
   parameter is a null terminated string. This can sometimes be more
   effecient on systems which have an ABI based on  null terminated
   strings.
 * Added `std.fs.Dir.openDir`, `std.fs.Dir.deleteFile`, and
   `std.fs.Dir.deleteDir` which all operate on an open directory handle.
 * `std.fs.Walker.Entry` now has a `dir` field, which can be used to do
   operations directly on `std.fs.Walker.Entry.basename`, avoiding
   `error.NameTooLong` for deeply nested paths.
 * Added more docs to `std.os.OpenError`

This commit does the POSIX components for these changes. I plan to
follow up shortly with a commit for Windows.
2019-10-20 21:48:23 -04:00
stratact e78d3750c5
Use 8192 sized buffers and remove allocator parameters 2019-10-19 14:04:51 -04:00
LemonBoy 6f7939a452 Prevent too eager constant-folding of switch expression
A pointer was wrongly assumed to be comptime-available causing the
analysis pass to assume its initial value was constant.

Fixes #3481
2019-10-19 13:48:26 -04:00
LemonBoy bab93e7561 Fix crash when generating constant unions with single field 2019-10-19 13:47:49 -04:00
Andrew Kelley 63dfca9715
Merge pull request #3482 from ziglang/mem-usage-report
add -DZIG_ENABLE_MEM_PROFILE option and -fmem-report flag
2019-10-19 12:42:32 -04:00
Andrew Kelley 4e98512344
reduce the size of IrInstruction by 8 bytes on 64 bit targets
This brings the std lib tests down from 3.51 GiB memory usage
to 3.41 GiB, by making two fields that were 64 bits 32 bits.
This is a small thing; the bigger wins will come from the strategy
outlined in the previous commit.
2019-10-19 03:51:04 -04:00
Andrew Kelley fa9f1d2396
add -DZIG_ENABLE_MEM_PROFILE option and -fmem-report flag
This can be used to find what's taking up memory in the compiler.
Here's an example of how to use it:

```
./zig test ../lib/std/std.zig --cache off -fmem-report
```

And here's the output I get for this on x86_64-linux-gnu today:

```
Const: 6462833 items, 152 bytes each, total 936.84 MiB
ConstGlobalRefs: 17236534 items, 24 bytes each, total 394.51 MiB
ResetResult: 1698108 items, 160 bytes each, total 259.11 MiB
ConstExprValue: 3118299 items, 80 bytes each, total 237.91 MiB
EndExpr: 1345395 items, 168 bytes each, total 215.56 MiB
Unknown_8: 27370821 items, 8 bytes each, total 208.82 MiB
VarPtr: 1127866 items, 168 bytes each, total 180.70 MiB
IrBasicBlock: 794834 items, 120 bytes each, total 90.96 MiB
LoadPtr: 554024 items, 160 bytes each, total 84.54 MiB
Unknown_64: 1245715 items, 64 bytes each, total 76.03 MiB
Unknown_40: 1879218 items, 40 bytes each, total 71.69 MiB
Unknown_72: 989117 items, 72 bytes each, total 67.92 MiB
Return: 423783 items, 160 bytes each, total 64.66 MiB
Unknown_168: 332480 items, 168 bytes each, total 53.27 MiB
Unknown_152: 336890 items, 152 bytes each, total 48.84 MiB
AddImplicitReturnType: 230819 items, 168 bytes each, total 36.98 MiB
Br: 217835 items, 168 bytes each, total 34.90 MiB
Unknown_184: 179529 items, 184 bytes each, total 31.50 MiB
FieldPtr: 179388 items, 184 bytes each, total 31.48 MiB
BinOp: 171004 items, 176 bytes each, total 28.70 MiB
LoadPtrGen: 173287 items, 168 bytes each, total 27.76 MiB
CondBr: 137864 items, 192 bytes each, total 25.24 MiB
Unknown_720: 30918 items, 720 bytes each, total 21.23 MiB
CallSrc: 99320 items, 216 bytes each, total 20.46 MiB
Unknown_160: 129243 items, 160 bytes each, total 19.72 MiB
Unknown_1: 19339456 items, 1 bytes each, total 18.44 MiB
CheckStatementIsVoid: 119838 items, 160 bytes each, total 18.29 MiB
Unknown_48: 371178 items, 48 bytes each, total 16.99 MiB
TestComptime: 101443 items, 160 bytes each, total 15.48 MiB
DeclVarSrc: 72578 items, 184 bytes each, total 12.74 MiB
StorePtr: 72776 items, 176 bytes each, total 12.22 MiB
ZigVar: 79201 items, 160 bytes each, total 12.09 MiB
Unknown_16: 770643 items, 16 bytes each, total 11.76 MiB
Phi: 60482 items, 184 bytes each, total 10.61 MiB
TestErrSrc: 66177 items, 168 bytes each, total 10.60 MiB
Unknown_240: 45164 items, 240 bytes each, total 10.34 MiB
ElemPtr: 58232 items, 184 bytes each, total 10.22 MiB
AllocaSrc: 60053 items, 176 bytes each, total 10.08 MiB
CallGen: 44873 items, 224 bytes each, total 9.59 MiB
SaveErrRetAddr: 63787 items, 152 bytes each, total 9.25 MiB
Unknown_112: 82283 items, 112 bytes each, total 8.79 MiB
AllocaGen: 51909 items, 176 bytes each, total 8.71 MiB
Unknown_24: 373599 items, 24 bytes each, total 8.55 MiB
ResultLocPeer: 113683 items, 72 bytes each, total 7.81 MiB
DeclRef: 36343 items, 168 bytes each, total 5.82 MiB
UnwrapErrPayload: 34603 items, 168 bytes each, total 5.54 MiB
Ref: 33414 items, 168 bytes each, total 5.35 MiB
Unknown_104: 53882 items, 104 bytes each, total 5.34 MiB
DeclVarGen: 32540 items, 168 bytes each, total 5.21 MiB
StructFieldPtr: 30449 items, 176 bytes each, total 5.11 MiB
UnwrapErrCode: 31508 items, 168 bytes each, total 5.05 MiB
Unknown_56: 90256 items, 56 bytes each, total 4.82 MiB
SpillBegin: 28722 items, 168 bytes each, total 4.60 MiB
SpillEnd: 28722 items, 160 bytes each, total 4.38 MiB
ResultLocReturn: 64573 items, 48 bytes each, total 2.96 MiB
PtrType: 14702 items, 184 bytes each, total 2.58 MiB
SliceType: 15005 items, 176 bytes each, total 2.52 MiB
Unknown_176: 13326 items, 176 bytes each, total 2.24 MiB
RefGen: 12881 items, 168 bytes each, total 2.06 MiB
UnOp: 12102 items, 176 bytes each, total 2.03 MiB
SwitchBr: 9453 items, 200 bytes each, total 1.80 MiB
TestErrGen: 11143 items, 160 bytes each, total 1.70 MiB
Unknown_32: 52359 items, 32 bytes each, total 1.60 MiB
CheckSwitchProngs: 9094 items, 184 bytes each, total 1.60 MiB
TypeOf: 9259 items, 160 bytes each, total 1.41 MiB
IntCast: 8772 items, 168 bytes each, total 1.41 MiB
OptionalUnwrapPtr: 8755 items, 168 bytes each, total 1.40 MiB
SwitchTarget: 9094 items, 160 bytes each, total 1.39 MiB
Cast: 8198 items, 176 bytes each, total 1.38 MiB
WidenOrShorten: 8448 items, 160 bytes each, total 1.29 MiB
ErrorUnion: 7613 items, 176 bytes each, total 1.28 MiB
SliceSrc: 6249 items, 192 bytes each, total 1.14 MiB
ErrWrapCode: 7133 items, 168 bytes each, total 1.14 MiB
TypeName: 7328 items, 160 bytes each, total 1.12 MiB
ImplicitCast: 5480 items, 176 bytes each, total 941.88 KiB
ResolveResult: 5638 items, 168 bytes each, total 924.98 KiB
ResultLocInstruction: 22696 items, 40 bytes each, total 886.56 KiB
BitCastSrc: 4947 items, 168 bytes each, total 811.62 KiB
CompileErr: 5148 items, 160 bytes each, total 804.38 KiB
ReturnPtr: 5305 items, 152 bytes each, total 787.46 KiB
Unreachable: 5038 items, 152 bytes each, total 747.83 KiB
TestNonNull: 4716 items, 160 bytes each, total 736.88 KiB
BitCastGen: 4431 items, 160 bytes each, total 692.34 KiB
PtrToInt: 4289 items, 160 bytes each, total 670.16 KiB
SliceGen: 3573 items, 192 bytes each, total 669.94 KiB
ArrayType: 4081 items, 168 bytes each, total 669.54 KiB
IntType: 3868 items, 168 bytes each, total 634.59 KiB
Unknown_88: 7213 items, 88 bytes each, total 619.87 KiB
Truncate: 3771 items, 168 bytes each, total 618.68 KiB
TypeInfo: 3740 items, 160 bytes each, total 584.38 KiB
SwitchVar: 3385 items, 176 bytes each, total 581.80 KiB
ContainerInitFields: 3223 items, 184 bytes each, total 579.13 KiB
ContainerInitList: 2309 items, 192 bytes each, total 432.94 KiB
PtrCastGen: 2626 items, 168 bytes each, total 430.83 KiB
BoolNot: 2457 items, 160 bytes each, total 383.91 KiB
FnProto: 2054 items, 184 bytes each, total 369.08 KiB
MergeErrSets: 1927 items, 176 bytes each, total 331.20 KiB
Unknown_136: 2486 items, 136 bytes each, total 330.17 KiB
Unknown_80: 4059 items, 80 bytes each, total 317.11 KiB
Bswap: 1670 items, 168 bytes each, total 273.98 KiB
TypeId: 1680 items, 160 bytes each, total 262.50 KiB
PtrCastSrc: 1371 items, 176 bytes each, total 235.64 KiB
ErrName: 1193 items, 160 bytes each, total 186.41 KiB
UnionTag: 1120 items, 160 bytes each, total 175.00 KiB
TagName: 1050 items, 160 bytes each, total 164.06 KiB
SizeOf: 942 items, 160 bytes each, total 147.19 KiB
MemberName: 871 items, 168 bytes each, total 142.90 KiB
Import: 881 items, 160 bytes each, total 137.66 KiB
PtrOfArrayToSlice: 758 items, 168 bytes each, total 124.36 KiB
UnionFieldPtr: 710 items, 176 bytes each, total 122.03 KiB
EnumToInt: 778 items, 160 bytes each, total 121.56 KiB
CheckRuntimeScope: 700 items, 168 bytes each, total 114.84 KiB
FieldParentPtr: 632 items, 184 bytes each, total 113.56 KiB
BoolToInt: 719 items, 160 bytes each, total 112.34 KiB
ResultLocPeerParent: 904 items, 104 bytes each, total 91.81 KiB
IntToPtr: 537 items, 168 bytes each, total 88.10 KiB
AlignOf: 561 items, 160 bytes each, total 87.66 KiB
AtomicRmw: 356 items, 208 bytes each, total 72.31 KiB
MemberCount: 441 items, 160 bytes each, total 68.91 KiB
Memset: 342 items, 176 bytes each, total 58.78 KiB
PopCount: 321 items, 168 bytes each, total 52.66 KiB
AlignCast: 251 items, 168 bytes each, total 41.18 KiB
IrInstruction *: 5230 items, 8 bytes each, total 40.86 KiB
IrBasicBlock *: 5230 items, 8 bytes each, total 40.86 KiB
TagType: 261 items, 160 bytes each, total 40.78 KiB
HasDecl: 234 items, 168 bytes each, total 38.39 KiB
OverflowOp: 191 items, 200 bytes each, total 37.30 KiB
Export: 209 items, 176 bytes each, total 35.92 KiB
SetCold: 219 items, 160 bytes each, total 34.22 KiB
ReturnAddress: 216 items, 152 bytes each, total 32.06 KiB
FromBytes: 178 items, 176 bytes each, total 30.59 KiB
SetRuntimeSafety: 188 items, 160 bytes each, total 29.38 KiB
OptionalWrap: 151 items, 168 bytes each, total 24.77 KiB
Clz: 143 items, 168 bytes each, total 23.46 KiB
ResizeSlice: 135 items, 168 bytes each, total 22.15 KiB
UnionInitNamedField: 106 items, 184 bytes each, total 19.05 KiB
Panic: 102 items, 160 bytes each, total 15.94 KiB
SwitchElseVar: 93 items, 168 bytes each, total 15.26 KiB
ToBytes: 89 items, 168 bytes each, total 14.60 KiB
IntToFloat: 78 items, 168 bytes each, total 12.80 KiB
Unknown_4360: 3 items, 4360 bytes each, total 12.77 KiB
ErrWrapPayload: 72 items, 168 bytes each, total 11.81 KiB
FloatOp: 62 items, 176 bytes each, total 10.66 KiB
FloatToInt: 47 items, 168 bytes each, total 7.71 KiB
FloatCast: 46 items, 168 bytes each, total 7.55 KiB
ErrToInt: 47 items, 160 bytes each, total 7.34 KiB
Asm: 33 items, 216 bytes each, total 6.96 KiB
ErrSetCast: 40 items, 168 bytes each, total 6.56 KiB
Memcpy: 34 items, 176 bytes each, total 5.84 KiB
AtomicLoad: 17 items, 184 bytes each, total 3.05 KiB
AwaitSrc: 16 items, 168 bytes each, total 2.62 KiB
Resume: 14 items, 160 bytes each, total 2.19 KiB
AwaitGen: 12 items, 176 bytes each, total 2.06 KiB
ArgType: 12 items, 168 bytes each, total 1.97 KiB
AnyFrameType: 12 items, 160 bytes each, total 1.88 KiB
SuspendFinish: 10 items, 160 bytes each, total 1.56 KiB
SuspendBegin: 10 items, 160 bytes each, total 1.56 KiB
Ctz: 9 items, 168 bytes each, total 1.48 KiB
FrameHandle: 8 items, 152 bytes each, total 1.19 KiB
SetEvalBranchQuota: 7 items, 160 bytes each, total 1.09 KiB
AssertZero: 7 items, 160 bytes each, total 1.09 KiB
UndeclaredIdent: 7 items, 160 bytes each, total 1.09 KiB
CmpxchgSrc: 5 items, 216 bytes each, total 1.05 KiB
CmpxchgGen: 5 items, 200 bytes each, total 1000.00 bytes
IntToEnum: 4 items, 168 bytes each, total 672.00 bytes
VectorType: 4 items, 168 bytes each, total 672.00 bytes
ErrorReturnTrace: 2 items, 160 bytes each, total 320.00 bytes
Breakpoint: 2 items, 152 bytes each, total 304.00 bytes
FrameAddress: 2 items, 152 bytes each, total 304.00 bytes
Unknown_4: 61 items, 4 bytes each, total 244.00 bytes
VectorToArray: 1 items, 168 bytes each, total 168.00 bytes
SetAlignStack: 1 items, 160 bytes each, total 160.00 bytes
Unknown_12: 2 items, 12 bytes each, total 24.00 bytes
ErrorTableEntry *: 0 items, 8 bytes each, total 0.00 bytes
AstNode *: 0 items, 8 bytes each, total 0.00 bytes
Total bytes used: 3.51 GiB
```

You can see that most of the memory is taken up by IR instructions,
as well as comptime values. This points toward 2 changes which will
greatly reduce memory usage:

 * Rework semantic analysis so that IR instructions can be freed.
   Currently the comptime value struct (ConstExprValue) is embedded
   directly into the IrInstruction struct. If this is made to be
   separate, at the very least pass 1 IR instructions can be freed.
   This includes `Const` which is the largest usage of memory currently.

 * Rework the ConstExprValue struct to no longer be a tagged union.
   For example, there's no need for an integer comptime value to be
   80 bytes.

From this you can also see that this eliminates some things from being
the culprit. Before doing this analysis, I considered whether doing
string interning would help. From the above output, you can see that all
strings in the compiler account for only 18 MiB, so string interning
would have been a dead end.
2019-10-19 03:23:23 -04:00
Andrew Kelley e42d86b657
Merge branch 'lun-4-progress-take-2'
closes #3362
2019-10-17 22:08:39 -04:00
Andrew Kelley 2d5b2bf1c9
improve progress reporting
* use erase rest of line escape code.
 * use `stderr.supportsAnsiEscapeCodes` rather than `isTty`.
 * respect `--color off`
 * avoid unnecessary recursion
 * add `Progress.log`
 * disable the progress std lib test since it's noisy and uses
   `time.sleep()`.
 * enable/integrate progress printing with the default test runner
2019-10-17 21:55:49 -04:00
Andrew Kelley 299991019d
rework the progress module and integrate with stage1 2019-10-17 20:20:22 -04:00
Luna a73c7bcaf9
add lib/std/progress.zig 2019-10-17 13:32:45 -04:00
Andrew Kelley 17aa8c3ee2
Merge branch 'musl-1.1.24'
closes #3463
2019-10-17 13:20:51 -04:00
Andrew Kelley 2aec35525d
mipsel: fix padding on stat struct 2019-10-17 13:20:30 -04:00
Andrew Kelley 88b9579488
update musl src to v1.1.24 2019-10-17 11:55:43 -04:00
Andrew Kelley 1135c9523b
update musl headers to v1.1.24 2019-10-17 11:47:57 -04:00
Andrew Kelley 0cb1c1696f
Merge pull request #3467 from ziglang/windows-static-linking
Fully static stage1 compiler on windows
2019-10-16 23:35:07 -04:00
Andrew Kelley 93295f3b13
remove -DZIG_FORCE_EXTERNAL_LLD=ON
Looks like the patch we need is in llvm trunk, not llvm 9.
2019-10-16 21:16:06 -04:00
Andrew Kelley f80c01f9d8
ref more math decls for better docs 2019-10-16 19:16:57 -04:00
Andrew Kelley c03dc6665f
update CI script for windows static builds 2019-10-16 19:08:41 -04:00
Sahnvour 63ecded3ce
msvc: reduce inlining aggressivity to workaround a codegen bug in cl.exe cf. #3024 2019-10-16 19:04:50 -04:00
Sahnvour 5b51f41cee
stage1: override cmake's default compilation flags to statically link the CRT on windows
we want to use /MT instead of /MD for a fully static executable
2019-10-16 19:04:50 -04:00
Andrew Kelley 7598a579ff
turn comments into doc comments 2019-10-16 18:33:12 -04:00
Sebsatian Keller 1b07ff7be0
Add 'missing' consts to math.zig
The C library defines constants for log2e and sqrt2 etc. Those should also be available in zig.
2019-10-16 18:31:22 -04:00
Emeka Nkurumeh 700bb19a90 fix std.mem.AsBytesReturnType (#3442)
closes #3334
2019-10-16 18:15:52 -04:00
Andrew Kelley 45e5841281
Merge pull request #3464 from nrdmn/uefi
UEFI improvements
2019-10-16 18:14:32 -04:00
Andrew Kelley 09abd09ab8
add docs for std.fs.File.updateTimes 2019-10-16 18:13:40 -04:00
LemonBoy 6a549a7f0c Fix compilation of updateTimes on 32bit platforms
Add a test to avoid regressions.

Fixes #3412
2019-10-16 18:10:32 -04:00
Andrew Kelley 5181970807
improve docs and field names of Statx struct 2019-10-16 17:24:42 -04:00
Andrew Kelley 312880f102
Merge pull request #3439 from LemonBoy/statx
Add support for the statx syscall
2019-10-16 17:11:22 -04:00
Andrew Kelley 3af2202ea4
add -I command line parameter, same as clang 2019-10-16 15:51:13 -04:00
Nick Erdmann e6eae25053
std/os/uefi: add documentation 2019-10-16 21:46:41 +02:00
Andrew Kelley 8cf3a4d586
[breaking] standardize std.os execve functions
* `std.os.execve` had the wrong name; it should have been
   `std.os.execvpe`. This is now corrected.
 * introduce `std.os.execveC` which does not look at PATH, and uses
   null terminated parameters, matching POSIX ABIs. It does not
   require an allocator.
 * fix typo nonsense doc comment in `std.fs.MAX_PATH_BYTES`.
 * introduce `std.os.execvpeC`, which is like `execvpe` except it
   uses null terminated parameters, matching POSIX ABIs, and thus
   does not require an allocator.
 * `std.os.execvpe` implementation is reworked to only convert
   parameters and then delegate to `std.os.execvpeC`.
 * `std.os.execvpeC` improved to handle `ENOTDIR`. See #3415
2019-10-16 15:24:48 -04:00
Nick Erdmann 77104da43e
std/os/uefi: add Guid format() 2019-10-16 19:49:51 +02:00
Andrew Kelley 10f6176f3d
only look for .defs when needed; remove samsrv.dll cross compiling support
previously zig would look for all the .defs even when not needed,
wasting time. also, we only had samsrv definitions for some architectures.
remove cross compiling support for this.
2019-10-16 13:12:35 -04:00
Andrew Kelley 5bfea3f1ea
add more cross compiling support for windows system dlls 2019-10-16 12:43:57 -04:00
Andrew Kelley f81d9d2c84
add cross compiling support for opengl32.dll 2019-10-16 12:16:35 -04:00
Andrew Kelley 1014cfdf3b
generated docs: progress towards generic types being useful
See #3406
2019-10-16 01:49:02 -04:00
Andrew Kelley 47dfaf3d17
merge analysis dumps tool handles errors 2019-10-15 18:40:26 -04:00
Andrew Kelley c7f994890c
Merge branch 'dtw-waleee-fix-help-modal'
closes #3455
2019-10-15 18:38:12 -04:00
Andrew Kelley e6ca61bdd1
generated docs: further clean up keyboard shortcut handling
now it's harder to introduce a bug when modifying keyboard shortcut
code.
2019-10-15 18:37:24 -04:00
dtw-waleee 725c873e81
fix small misstake, 'escape' v. 'Escape' 2019-10-15 18:19:13 -04:00
dtw-waleee 3e891c9c0b
Fix keyboard layout issue with help-modal
Some keyboard layouts produces a different ev.which value in firefox
for ? than 191, eg. the Swedish QWERTY one produces 171. Chrome/chromium
doesn't have this issue.
2019-10-15 18:19:13 -04:00
tgschultz 6a629d3a9f Replaced setTag hack in Deserialize with @unionInit 2019-10-15 18:14:13 -04:00
Andrew Kelley 9050cd847a
fix non-byte-aligned packed struct field...
...passed as generic fn parameter causing invalid LLVM IR.

closes #3460
2019-10-15 18:00:16 -04:00
Vexu 7495fd8cb9 fix struct align fmt 2019-10-15 17:58:02 -04:00
Andrew Kelley f7f3dedb1d
Merge pull request #3436 from LemonBoy/unpatch-lld
Assemble lib files using LLVM tools instead of lld
2019-10-13 17:55:36 -04:00
LemonBoy b164e0ae55 Fix stack iteration stop condition 2019-10-13 17:49:47 -04:00
Andrew Kelley 9439bf3809
Merge pull request #3446 from LemonBoy/riscv-things-x
More RISC-V stuff
2019-10-13 11:07:39 -04:00