Ekdohibs
/
zig
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

delete all stage1 c++ code not directly related to compiling stage2 

Deleted 16,000+ lines of c++ code, including:
 * an implementation of blake hashing
 * the cache hash system
 * compiler.cpp
 * all the linking code, and everything having to do with building
   glibc, musl, and mingw-w64
 * much of the stage1 compiler internals got slimmed down since it
   now assumes it is always outputting an object file.

More stuff:
 * stage1 is now built with a different strategy: we have a tiny
   zig0.cpp which is a slimmed down version of what stage1 main.cpp used
   to be. Its only purpose is to build stage2 zig code into an object
   file, which is then linked by the host build system (cmake) into
   stage1. zig0.cpp uses the same C API that stage2 now has access to,
   so that stage2 zig code can call into stage1 c++ code.
   - stage1.h is
   - stage2.h is
   - stage1.zig is the main entry point for the Zig/C++
     hybrid compiler. It has the functions exported from Zig, called
     in C++, and bindings for the functions exported from C++, called
     from Zig.
 * removed the memory profiling instrumentation from stage1.
   Abandon ship!
 * Re-added the sections to the README about how to build stage2 and
   stage3.
 * stage2 now knows as a comptime boolean whether it is being compiled
   as part of stage1 or as stage2.
   - TODO use this flag to call into stage1 for compiling zig code.
 * introduce -fdll-export-fns and -fno-dll-export-fns and clarify
   its relationship to link_mode (static/dynamic)
 * implement depending on LLVM to detect native target cpu features when
   LLVM extensions are enabled and zig lacks CPU feature detection for
   that target architecture.
 * C importing is broken, will need some stage2 support to function
   again.
master
Andrew Kelley 2020-09-17 18:29:38 -07:00
parent bc01887376
commit dc478687d9
47 changed files with 3448 additions and 12799 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

24
BRANCH_TODO
View File

@ -1,3 +1,21 @@
* `zig builtin`
* `zig translate-c`
* `zig test`
* `zig run`
* `zig init-lib`
* `zig init-exe`
* `zig build`
* `-ftime-report`
* -fstack-report print stack size diagnostics\n"
* -fdump-analysis write analysis.json file with type information\n"
* -femit-docs create a docs/ dir with html documentation\n"
* -fno-emit-docs do not produce docs/ dir with html documentation\n"
* -femit-asm output .s (assembly code)\n"
* -fno-emit-asm (default) do not output .s (assembly code)\n"
* -femit-llvm-ir produce a .ll file with LLVM IR\n"
* -fno-emit-llvm-ir (default) do not produce a .ll file with LLVM IR\n"
* --cache-dir [path] override the local cache directory\n"
* move main.cpp to stage2
* make sure zig cc works
- using it as a preprocessor (-E)
- try building some software
@ -6,26 +24,24 @@
- stage1 C++ code integration
* build & link againstn freestanding libc
* add CLI support for a way to pass extra flags to c source files
* implement the workaround for using LLVM to detect native CPU features
* self-host main.cpp
* capture lld stdout/stderr better
* musl
* mingw-w64
* use global zig-cache dir for crt files
* `zig translate-c`
* MachO LLD linking
* COFF LLD linking
* WASM LLD linking
* implement proper parsing of LLD stderr/stdout and exposing compile errors
* implement proper parsing of clang stderr/stdout and exposing compile errors
* implement proper compile errors for failing to build glibc crt files and shared libs
* self-host link.cpp and building libcs (#4313 and #4314). using the `zig cc` command will set a flag indicating a preference for the llvm backend, which will include linking with LLD. At least for now. If zig's self-hosted linker ever gets on par with the likes of ld and lld, we can make it always be used even for zig cc.
* improve the stage2 tests to support testing with LLVM extensions enabled
* multi-thread building C objects
* support cross compiling stage2 with `zig build`
* implement emit-h in stage2
* implement -fno-emit-bin
* audit the base cache hash
* audit the CLI options for stage2
* implement serialization/deserialization of incremental compilation metadata
* incremental compilation - implement detection of which source files changed
* improve the cache hash logic for c objects with respect to extra flags and file parameters

71
CMakeLists.txt
View File

@ -51,7 +51,6 @@ message("Configuring zig version ${ZIG_VERSION}")
set(ZIG_STATIC off CACHE BOOL "Attempt to build a static zig executable (not compatible with glibc)")
set(ZIG_STATIC_LLVM off CACHE BOOL "Prefer linking against static LLVM libraries")
set(ZIG_ENABLE_MEM_PROFILE off CACHE BOOL "Activate memory usage instrumentation")
set(ZIG_PREFER_CLANG_CPP_DYLIB off CACHE BOOL "Try to link against -lclang-cpp")
set(ZIG_WORKAROUND_4799 off CACHE BOOL "workaround for https://github.com/ziglang/zig/issues/4799")
set(ZIG_WORKAROUND_POLLY_SO off CACHE STRING "workaround for https://github.com/ziglang/zig/issues/4799")
@ -71,11 +70,6 @@ string(REGEX REPLACE "\\\\" "\\\\\\\\" ZIG_LIBC_INCLUDE_DIR_ESCAPED "${ZIG_LIBC_
option(ZIG_TEST_COVERAGE "Build Zig with test coverage instrumentation" OFF)
# Zig no longer has embedded LLD. This option is kept for package maintainers
# so that they don't have to update their scripts in case we ever re-introduce
# LLD to the tree. This option does nothing.
option(ZIG_FORCE_EXTERNAL_LLD "does nothing" OFF)
set(ZIG_TARGET_TRIPLE "native" CACHE STRING "arch-os-abi to output binaries for")
set(ZIG_TARGET_MCPU "baseline" CACHE STRING "-mcpu parameter to output binaries for")
set(ZIG_EXECUTABLE "" CACHE STRING "(when cross compiling) path to already-built zig binary")
@ -261,15 +255,11 @@ include_directories("${CMAKE_SOURCE_DIR}/deps/dbg-macro")
find_package(Threads)
# CMake doesn't let us create an empty executable, so we hang on to this one separately.
set(ZIG_MAIN_SRC "${CMAKE_SOURCE_DIR}/src/main.cpp")
# This is our shim which will be replaced by libstage2 written in Zig.
set(ZIG0_SHIM_SRC "${CMAKE_SOURCE_DIR}/src/stage2.cpp")
if(ZIG_ENABLE_MEM_PROFILE)
set(ZIG_SOURCES_MEM_PROFILE "${CMAKE_SOURCE_DIR}/src/mem_profile.cpp")
endif()
# This is our shim which will be replaced by stage1.zig.
set(ZIG0_SOURCES
"${CMAKE_SOURCE_DIR}/src/zig0.cpp"
"${CMAKE_SOURCE_DIR}/src/stage2.cpp"
)
set(ZIG_SOURCES
"${CMAKE_SOURCE_DIR}/src/analyze.cpp"
@ -277,37 +267,34 @@ set(ZIG_SOURCES
"${CMAKE_SOURCE_DIR}/src/bigfloat.cpp"
"${CMAKE_SOURCE_DIR}/src/bigint.cpp"
"${CMAKE_SOURCE_DIR}/src/buffer.cpp"
"${CMAKE_SOURCE_DIR}/src/cache_hash.cpp"
"${CMAKE_SOURCE_DIR}/src/codegen.cpp"
"${CMAKE_SOURCE_DIR}/src/compiler.cpp"
"${CMAKE_SOURCE_DIR}/src/dump_analysis.cpp"
"${CMAKE_SOURCE_DIR}/src/errmsg.cpp"
"${CMAKE_SOURCE_DIR}/src/error.cpp"
"${CMAKE_SOURCE_DIR}/src/glibc.cpp"
"${CMAKE_SOURCE_DIR}/src/heap.cpp"
"${CMAKE_SOURCE_DIR}/src/ir.cpp"
"${CMAKE_SOURCE_DIR}/src/ir_print.cpp"
"${CMAKE_SOURCE_DIR}/src/link.cpp"
"${CMAKE_SOURCE_DIR}/src/mem.cpp"
"${CMAKE_SOURCE_DIR}/src/os.cpp"
"${CMAKE_SOURCE_DIR}/src/parser.cpp"
"${CMAKE_SOURCE_DIR}/src/range_set.cpp"
"${CMAKE_SOURCE_DIR}/src/stage1.cpp"
"${CMAKE_SOURCE_DIR}/src/target.cpp"
"${CMAKE_SOURCE_DIR}/src/tokenizer.cpp"
"${CMAKE_SOURCE_DIR}/src/util.cpp"
"${CMAKE_SOURCE_DIR}/src/softfloat_ext.cpp"
"${ZIG_SOURCES_MEM_PROFILE}"
)
set(OPTIMIZED_C_SOURCES
"${CMAKE_SOURCE_DIR}/src/blake2b.c"
"${CMAKE_SOURCE_DIR}/src/parse_f128.c"
)
set(ZIG_CPP_SOURCES
# These are planned to stay even when we are self-hosted.
"${CMAKE_SOURCE_DIR}/src/zig_llvm.cpp"
"${CMAKE_SOURCE_DIR}/src/zig_clang.cpp"
"${CMAKE_SOURCE_DIR}/src/zig_clang_driver.cpp"
"${CMAKE_SOURCE_DIR}/src/zig_clang_cc1_main.cpp"
"${CMAKE_SOURCE_DIR}/src/zig_clang_cc1as_main.cpp"
# https://github.com/ziglang/zig/issues/6363
"${CMAKE_SOURCE_DIR}/src/windows_sdk.cpp"
)
@ -447,7 +434,7 @@ if(MSVC OR MINGW)
target_link_libraries(zigcompiler LINK_PUBLIC version)
endif()
add_executable(zig0 "${ZIG_MAIN_SRC}" "${ZIG0_SHIM_SRC}")
add_executable(zig0 ${ZIG0_SOURCES})
set_target_properties(zig0 PROPERTIES
COMPILE_FLAGS ${EXE_CFLAGS}
LINK_FLAGS ${EXE_LDFLAGS}
@ -455,37 +442,36 @@ set_target_properties(zig0 PROPERTIES
target_link_libraries(zig0 zigcompiler)
if(MSVC)
set(LIBSTAGE2 "${CMAKE_BINARY_DIR}/zigstage2.lib")
set(ZIG1_OBJECT "${CMAKE_BINARY_DIR}/zig1.obj")
else()
set(LIBSTAGE2 "${CMAKE_BINARY_DIR}/libzigstage2.a")
set(ZIG1_OBJECT "${CMAKE_BINARY_DIR}/zig1.o")
endif()
if("${CMAKE_BUILD_TYPE}" STREQUAL "Debug")
set(LIBSTAGE2_RELEASE_ARG "")
set(ZIG1_RELEASE_ARG "")
else()
set(LIBSTAGE2_RELEASE_ARG --release-fast --strip)
set(ZIG1_RELEASE_ARG --release-fast --strip)
endif()
set(BUILD_LIBSTAGE2_ARGS "build-lib"
"src-self-hosted/stage2.zig"
set(BUILD_ZIG1_ARGS
"src-self-hosted/stage1.zig"
-target "${ZIG_TARGET_TRIPLE}"
"-mcpu=${ZIG_TARGET_MCPU}"
--name zigstage2
--name zig1
--override-lib-dir "${CMAKE_SOURCE_DIR}/lib"
--cache on
--output-dir "${CMAKE_BINARY_DIR}"
${LIBSTAGE2_RELEASE_ARG}
--bundle-compiler-rt
-fPIC
${ZIG1_RELEASE_ARG}
-lc
--pkg-begin build_options "${ZIG_CONFIG_ZIG_OUT}"
--pkg-end
--pkg-begin compiler_rt "${CMAKE_SOURCE_DIR}/lib/std/special/compiler_rt.zig"
--pkg-end
)
if("${ZIG_TARGET_TRIPLE}" STREQUAL "native")
add_custom_target(zig_build_libstage2 ALL
COMMAND zig0 ${BUILD_LIBSTAGE2_ARGS}
add_custom_target(zig_build_zig1 ALL
COMMAND zig0 ${BUILD_ZIG1_ARGS}
DEPENDS zig0
BYPRODUCTS "${LIBSTAGE2}"
BYPRODUCTS "${ZIG1_OBJECT}"
WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}"
)
set(ZIG_EXECUTABLE "${zig_BINARY_DIR}/zig")
@ -493,26 +479,27 @@ if("${ZIG_TARGET_TRIPLE}" STREQUAL "native")
set(ZIG_EXECUTABLE "${ZIG_EXECUTABLE}.exe")
endif()
else()
add_custom_target(zig_build_libstage2 ALL
COMMAND "${ZIG_EXECUTABLE}" ${BUILD_LIBSTAGE2_ARGS}
BYPRODUCTS "${LIBSTAGE2}"
add_custom_target(zig_build_zig1 ALL
COMMAND "${ZIG_EXECUTABLE}" ${BUILD_ZIG1_ARGS}
BYPRODUCTS "${ZIG1_OBJECT}"
WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}"
)
endif()
add_executable(zig "${ZIG_MAIN_SRC}")
# cmake won't let us configure an executable without C sources.
add_executable(zig "${CMAKE_SOURCE_DIR}/src/empty.cpp")
set_target_properties(zig PROPERTIES
COMPILE_FLAGS ${EXE_CFLAGS}
LINK_FLAGS ${EXE_LDFLAGS}
)
target_link_libraries(zig zigcompiler "${LIBSTAGE2}")
target_link_libraries(zig zigcompiler "${ZIG1_OBJECT}")
if(MSVC)
target_link_libraries(zig ntdll.lib)
elseif(MINGW)
target_link_libraries(zig ntdll)
endif()
add_dependencies(zig zig_build_libstage2)
add_dependencies(zig zig_build_zig1)
install(TARGETS zig DESTINATION bin)

37
README.md
View File

@ -22,6 +22,8 @@ Note that you can
### Stage 1: Build Zig from C++ Source Code
This step must be repeated when you make changes to any of the C++ source code.
#### Dependencies
##### POSIX
@ -77,6 +79,41 @@ Hopefully this will be fixed upstream with LLVM 10.0.1.
See https://github.com/ziglang/zig/wiki/Building-Zig-on-Windows
### Stage 2: Build Self-Hosted Zig from Zig Source Code
Now we use the stage1 binary:
```
zig build --prefix $(pwd)/stage2 -Denable-llvm
```
This produces `stage2/bin/zig` which can be used for testing and development.
Once it is feature complete, it will be used to build stage 3 - the final compiler
binary.
### Stage 3: Rebuild Self-Hosted Zig Using the Self-Hosted Compiler
*Note: Stage 2 compiler is not yet able to build Stage 3. Building Stage 3 is
not yet supported.*
Once the self-hosted compiler can build itself, this will be the actual
compiler binary that we will install to the system. Until then, users should
use stage 1.
#### Debug / Development Build
```
stage2/bin/zig build
```
This produces `zig-cache/bin/zig`.
#### Release / Install Build
```
stage2/bin/zig build install -Drelease
```
## License
The ultimate goal of the Zig project is to serve users. As a first-order

2
build.zig
View File

@ -145,6 +145,7 @@ pub fn build(b: *Builder) !void {
exe.addBuildOption([]const []const u8, "log_scopes", log_scopes);
exe.addBuildOption([]const []const u8, "zir_dumps", zir_dumps);
exe.addBuildOption(bool, "enable_tracy", tracy != null);
exe.addBuildOption(bool, "is_stage1", false);
if (tracy) |tracy_path| {
const client_cpp = fs.path.join(
b.allocator,
@ -165,6 +166,7 @@ pub fn build(b: *Builder) !void {
const is_wasmtime_enabled = b.option(bool, "enable-wasmtime", "Use Wasmtime to enable and run WASI libstd tests") orelse false;
const glibc_multi_dir = b.option([]const u8, "enable-foreign-glibc", "Provide directory with glibc installations to run cross compiled tests that link glibc");
test_stage2.addBuildOption(bool, "is_stage1", false);
test_stage2.addBuildOption(bool, "have_llvm", enable_llvm);
test_stage2.addBuildOption(bool, "enable_qemu", is_qemu_enabled);
test_stage2.addBuildOption(bool, "enable_wine", is_wine_enabled);

18
src-self-hosted/Compilation.zig
View File

@ -244,6 +244,7 @@ pub const InitOptions = struct {
/// `null` means to not emit a C header file.
emit_h: ?EmitLoc = null,
link_mode: ?std.builtin.LinkMode = null,
dll_export_fns: ?bool = false,
object_format: ?std.builtin.ObjectFormat = null,
optimize_mode: std.builtin.Mode = .Debug,
keep_source_files_loaded: bool = false,
@ -340,9 +341,13 @@ pub fn create(gpa: *Allocator, options: InitOptions) !*Compilation {
return error.MachineCodeModelNotSupported;
}
const is_dyn_lib = switch (options.output_mode) {
.Obj, .Exe => false,
.Lib => (options.link_mode orelse .Static) == .Dynamic,
};
const is_exe_or_dyn_lib = switch (options.output_mode) {
.Obj => false,
.Lib => (options.link_mode orelse .Static) == .Dynamic,
.Lib => is_dyn_lib,
.Exe => true,
};
const must_dynamic_link = dl: {
@ -365,6 +370,8 @@ pub fn create(gpa: *Allocator, options: InitOptions) !*Compilation {
break :blk lm;
} else default_link_mode;
const dll_export_fns = if (options.dll_export_fns) |explicit| explicit else is_dyn_lib;
const libc_dirs = try detectLibCIncludeDirs(
arena,
options.zig_lib_directory.path.?,
@ -379,7 +386,12 @@ pub fn create(gpa: *Allocator, options: InitOptions) !*Compilation {
break :b true;
break :b link_mode == .Dynamic;
};
const pic = options.want_pic orelse must_pic;
const pic = if (options.want_pic) |explicit| pic: {
if (!explicit and must_pic) {
return error.TargetRequiresPIC;
}
break :pic explicit;
} else must_pic;
if (options.emit_h != null) fatal("-femit-h not supported yet", .{}); // TODO
@ -464,6 +476,7 @@ pub fn create(gpa: *Allocator, options: InitOptions) !*Compilation {
hash.add(valgrind);
hash.add(single_threaded);
hash.add(options.target.os.getVersionRange());
hash.add(dll_export_fns);
const digest = hash.final();
const artifact_sub_dir = try std.fs.path.join(arena, &[_][]const u8{ "o", &digest });
@ -601,6 +614,7 @@ pub fn create(gpa: *Allocator, options: InitOptions) !*Compilation {
.single_threaded = single_threaded,
.debug_link = options.debug_link,
.machine_code_model = options.machine_code_model,
.dll_export_fns = dll_export_fns,
});
errdefer bin_file.destroy();

66
src-self-hosted/libcxx.zig Normal file
View File

@ -0,0 +1,66 @@
//! TODO build libcxx and libcxxabi from source
pub const libcxxabi_files = [_][]const u8{
"src/abort_message.cpp",
"src/cxa_aux_runtime.cpp",
"src/cxa_default_handlers.cpp",
"src/cxa_demangle.cpp",
"src/cxa_exception.cpp",
"src/cxa_exception_storage.cpp",
"src/cxa_guard.cpp",
"src/cxa_handlers.cpp",
"src/cxa_noexception.cpp",
"src/cxa_personality.cpp",
"src/cxa_thread_atexit.cpp",
"src/cxa_unexpected.cpp",
"src/cxa_vector.cpp",
"src/cxa_virtual.cpp",
"src/fallback_malloc.cpp",
"src/private_typeinfo.cpp",
"src/stdlib_exception.cpp",
"src/stdlib_stdexcept.cpp",
"src/stdlib_typeinfo.cpp",
};
pub const libcxx_files = [_][]const u8{
"src/algorithm.cpp",
"src/any.cpp",
"src/bind.cpp",
"src/charconv.cpp",
"src/chrono.cpp",
"src/condition_variable.cpp",
"src/condition_variable_destructor.cpp",
"src/debug.cpp",
"src/exception.cpp",
"src/experimental/memory_resource.cpp",
"src/filesystem/directory_iterator.cpp",
"src/filesystem/operations.cpp",
"src/functional.cpp",
"src/future.cpp",
"src/hash.cpp",
"src/ios.cpp",
"src/iostream.cpp",
"src/locale.cpp",
"src/memory.cpp",
"src/mutex.cpp",
"src/mutex_destructor.cpp",
"src/new.cpp",
"src/optional.cpp",
"src/random.cpp",
"src/regex.cpp",
"src/shared_mutex.cpp",
"src/stdexcept.cpp",
"src/string.cpp",
"src/strstream.cpp",
"src/support/solaris/xlocale.cpp",
"src/support/win32/locale_win32.cpp",
"src/support/win32/support.cpp",
"src/support/win32/thread_win32.cpp",
"src/system_error.cpp",
"src/thread.cpp",
"src/typeinfo.cpp",
"src/utility.cpp",
"src/valarray.cpp",
"src/variant.cpp",
"src/vector.cpp",
};

1
src-self-hosted/link.zig
View File

@ -62,6 +62,7 @@ pub const Options = struct {
stack_check: bool,
single_threaded: bool,
debug_link: bool = false,
dll_export_fns: bool,
gc_sections: ?bool = null,
allow_shlib_undefined: ?bool = null,
linker_script: ?[]const u8 = null,

112
src-self-hosted/main.zig
View File

@ -92,7 +92,7 @@ pub fn log(
var general_purpose_allocator = std.heap.GeneralPurposeAllocator(.{}){};
pub fn main() !void {
pub fn main() anyerror!void {
const gpa = if (std.builtin.link_libc) std.heap.c_allocator else &general_purpose_allocator.allocator;
defer if (!std.builtin.link_libc) {
_ = general_purpose_allocator.deinit();
@ -102,7 +102,10 @@ pub fn main() !void {
const arena = &arena_instance.allocator;
const args = try process.argsAlloc(arena);
return mainArgs(gpa, arena, args);
}
pub fn mainArgs(gpa: *Allocator, arena: *Allocator, args: []const []const u8) !void {
if (args.len <= 1) {
std.log.info("{}", .{usage});
fatal("expected command argument", .{});
@ -131,7 +134,7 @@ pub fn main() !void {
} else if (mem.eql(u8, cmd, "libc")) {
return cmdLibC(gpa, cmd_args);
} else if (mem.eql(u8, cmd, "targets")) {
const info = try std.zig.system.NativeTargetInfo.detect(arena, .{});
const info = try detectNativeTargetInfo(arena, .{});
const stdout = io.getStdOut().outStream();
return @import("print_targets.zig").cmdTargets(arena, cmd_args, stdout, info.target);
} else if (mem.eql(u8, cmd, "version")) {
@ -183,7 +186,7 @@ const usage_build_generic =
\\ -mcmodel=[default|tiny| Limit range of code and data virtual addresses
\\ small|kernel|
\\ medium|large]
\\ --name [name] Override output name
\\ --name [name] Override root name (not a file path)
\\ --mode [mode] Set the build mode
\\ Debug (default) optimizations off, safety on
\\ ReleaseFast Optimizations on, safety off
@ -197,7 +200,9 @@ const usage_build_generic =
\\ -fno-sanitize-c Disable C undefined behavior detection in safe builds
\\ -fvalgrind Include valgrind client requests in release builds
\\ -fno-valgrind Omit valgrind client requests in debug builds
\\ --strip Exclude debug symbols
\\ -fdll-export-fns Mark exported functions as DLL exports (Windows)
\\ -fno-dll-export-fns Force-disable marking exported functions as DLL exports
\\ --strip Omit debug symbols
\\ --single-threaded Code assumes it is only used single-threaded
\\ -ofmt=[mode] Override target object format
\\ elf Executable and Linking Format
@ -262,6 +267,7 @@ pub fn buildOutputType(
var build_mode: std.builtin.Mode = .Debug;
var provided_name: ?[]const u8 = null;
var link_mode: ?std.builtin.LinkMode = null;
var dll_export_fns: ?bool = null;
var root_src_file: ?[]const u8 = null;
var version: std.builtin.Version = .{ .major = 0, .minor = 0, .patch = 0 };
var have_version = false;
@ -521,6 +527,10 @@ pub fn buildOutputType(
link_mode = .Dynamic;
} else if (mem.eql(u8, arg, "-static")) {
link_mode = .Static;
} else if (mem.eql(u8, arg, "-fdll-export-fns")) {
dll_export_fns = true;
} else if (mem.eql(u8, arg, "-fno-dll-export-fns")) {
dll_export_fns = false;
} else if (mem.eql(u8, arg, "--strip")) {
strip = true;
} else if (mem.eql(u8, arg, "--single-threaded")) {
@ -902,14 +912,7 @@ pub fn buildOutputType(
else => |e| return e,
};
const target_info = try std.zig.system.NativeTargetInfo.detect(gpa, cross_target);
if (target_info.cpu_detection_unimplemented) {
// TODO We want to just use detected_info.target but implementing
// CPU model & feature detection is todo so here we rely on LLVM.
// TODO The workaround to use LLVM to detect features needs to be used for
// `zig targets` as well.
fatal("CPU features detection is not yet available for this system without LLVM extensions", .{});
}
const target_info = try detectNativeTargetInfo(gpa, cross_target);
if (target_info.target.os.tag != .freestanding) {
if (ensure_libc_on_non_freestanding)
@ -1116,6 +1119,7 @@ pub fn buildOutputType(
.emit_bin = emit_bin_loc,
.emit_h = emit_h_loc,
.link_mode = link_mode,
.dll_export_fns = dll_export_fns,
.object_format = object_format,
.optimize_mode = build_mode,
.keep_source_files_loaded = zir_out_path != null,
@ -1974,3 +1978,87 @@ fn gimmeMoreOfThoseSweetSweetFileDescriptors() void {
test "fds" {
gimmeMoreOfThoseSweetSweetFileDescriptors();
}
fn detectNativeCpuWithLLVM(
arch: std.Target.Cpu.Arch,
llvm_cpu_name_z: ?[*:0]const u8,
llvm_cpu_features_opt: ?[*:0]const u8,
) !std.Target.Cpu {
var result = std.Target.Cpu.baseline(arch);
if (llvm_cpu_name_z) |cpu_name_z| {
const llvm_cpu_name = mem.spanZ(cpu_name_z);
for (arch.allCpuModels()) |model| {
const this_llvm_name = model.llvm_name orelse continue;
if (mem.eql(u8, this_llvm_name, llvm_cpu_name)) {
// Here we use the non-dependencies-populated set,
// so that subtracting features later in this function
// affect the prepopulated set.
result = std.Target.Cpu{
.arch = arch,
.model = model,
.features = model.features,
};
break;
}
}
}
const all_features = arch.allFeaturesList();
if (llvm_cpu_features_opt) |llvm_cpu_features| {
var it = mem.tokenize(mem.spanZ(llvm_cpu_features), ",");
while (it.next()) |decorated_llvm_feat| {
var op: enum {
add,
sub,
} = undefined;
var llvm_feat: []const u8 = undefined;
if (mem.startsWith(u8, decorated_llvm_feat, "+")) {
op = .add;
llvm_feat = decorated_llvm_feat[1..];
} else if (mem.startsWith(u8, decorated_llvm_feat, "-")) {
op = .sub;
llvm_feat = decorated_llvm_feat[1..];
} else {
return error.InvalidLlvmCpuFeaturesFormat;
}
for (all_features) |feature, index_usize| {
const this_llvm_name = feature.llvm_name orelse continue;
if (mem.eql(u8, llvm_feat, this_llvm_name)) {
const index = @intCast(std.Target.Cpu.Feature.Set.Index, index_usize);
switch (op) {
.add => result.features.addFeature(index),
.sub => result.features.removeFeature(index),
}
break;
}
}
}
}
result.features.populateDependencies(all_features);
return result;
}
fn detectNativeTargetInfo(gpa: *Allocator, cross_target: std.zig.CrossTarget) !std.zig.system.NativeTargetInfo {
var info = try std.zig.system.NativeTargetInfo.detect(gpa, cross_target);
if (info.cpu_detection_unimplemented) {
const arch = std.Target.current.cpu.arch;
// We want to just use detected_info.target but implementing
// CPU model & feature detection is todo so here we rely on LLVM.
// https://github.com/ziglang/zig/issues/4591
if (!build_options.have_llvm)
fatal("CPU features detection is not yet available for {} without LLVM extensions", .{@tagName(arch)});
const llvm = @import("llvm.zig");
const llvm_cpu_name = llvm.GetHostCPUName();
const llvm_cpu_features = llvm.GetNativeFeatures();
info.target.cpu = try detectNativeCpuWithLLVM(arch, llvm_cpu_name, llvm_cpu_features);
cross_target.updateCpuFeatures(&info.target.cpu.features);
info.target.cpu.arch = cross_target.getCpuArch();
}
return info;
}

1843
src-self-hosted/musl.zig Normal file
View File

File diff suppressed because it is too large Load Diff

495
src-self-hosted/stage1.zig Normal file
View File

@ -0,0 +1,495 @@
//! This is the main entry point for the Zig/C++ hybrid compiler (stage1).
//! It has the functions exported from Zig, called in C++, and bindings for
//! the functions exported from C++, called from Zig.
const std = @import("std");
const assert = std.debug.assert;
const mem = std.mem;
const build_options = @import("build_options");
const stage2 = @import("main.zig");
const fatal = stage2.fatal;
const CrossTarget = std.zig.CrossTarget;
const Target = std.Target;
comptime {
assert(std.builtin.link_libc);
assert(build_options.is_stage1);
_ = @import("compiler_rt");
}
pub const log = stage2.log;
pub const log_level = stage2.log_level;
pub export fn main(argc: c_int, argv: [*]const [*:0]const u8) c_int {
std.debug.maybeEnableSegfaultHandler();
const gpa = std.heap.c_allocator;
var arena_instance = std.heap.ArenaAllocator.init(gpa);
defer arena_instance.deinit();
const arena = &arena_instance.allocator;
const args = arena.alloc([]const u8, @intCast(usize, argc)) catch fatal("out of memory", .{});
for (args) |*arg, i| {
arg.* = mem.spanZ(argv[i]);
}
stage2.mainArgs(gpa, arena, args) catch |err| fatal("{}", .{err});
return 0;
}
// ABI warning
export fn stage2_panic(ptr: [*]const u8, len: usize) void {
@panic(ptr[0..len]);
}
// ABI warning
const Error = extern enum {
None,
OutOfMemory,
InvalidFormat,
SemanticAnalyzeFail,
AccessDenied,
Interrupted,
SystemResources,
FileNotFound,
FileSystem,
FileTooBig,
DivByZero,
Overflow,
PathAlreadyExists,
Unexpected,
ExactDivRemainder,
NegativeDenominator,
ShiftedOutOneBits,
CCompileErrors,
EndOfFile,
IsDir,
NotDir,
UnsupportedOperatingSystem,
SharingViolation,
PipeBusy,
PrimitiveTypeNotFound,
CacheUnavailable,
PathTooLong,
CCompilerCannotFindFile,
NoCCompilerInstalled,
ReadingDepFile,
InvalidDepFile,
MissingArchitecture,
MissingOperatingSystem,
UnknownArchitecture,
UnknownOperatingSystem,
UnknownABI,
InvalidFilename,
DiskQuota,
DiskSpace,
UnexpectedWriteFailure,
UnexpectedSeekFailure,
UnexpectedFileTruncationFailure,
Unimplemented,
OperationAborted,
BrokenPipe,
NoSpaceLeft,
NotLazy,
IsAsync,
ImportOutsidePkgPath,
UnknownCpuModel,
UnknownCpuFeature,
InvalidCpuFeatures,
InvalidLlvmCpuFeaturesFormat,
UnknownApplicationBinaryInterface,
ASTUnitFailure,
BadPathName,
SymLinkLoop,
ProcessFdQuotaExceeded,
SystemFdQuotaExceeded,
NoDevice,
DeviceBusy,
UnableToSpawnCCompiler,
CCompilerExitCode,
CCompilerCrashed,
CCompilerCannotFindHeaders,
LibCRuntimeNotFound,
LibCStdLibHeaderNotFound,
LibCKernel32LibNotFound,
UnsupportedArchitecture,
WindowsSdkNotFound,
UnknownDynamicLinkerPath,
TargetHasNoDynamicLinker,
InvalidAbiVersion,
InvalidOperatingSystemVersion,
UnknownClangOption,
NestedResponseFile,
ZigIsTheCCompiler,
FileBusy,
Locked,
};
// ABI warning
export fn stage2_attach_segfault_handler() void {
if (std.debug.runtime_safety and std.debug.have_segfault_handling_support) {
std.debug.attachSegfaultHandler();
}
}
// ABI warning
export fn stage2_progress_create() *std.Progress {
const ptr = std.heap.c_allocator.create(std.Progress) catch @panic("out of memory");
ptr.* = std.Progress{};
return ptr;
}
// ABI warning
export fn stage2_progress_destroy(progress: *std.Progress) void {
std.heap.c_allocator.destroy(progress);
}
// ABI warning
export fn stage2_progress_start_root(
progress: *std.Progress,
name_ptr: [*]const u8,
name_len: usize,
estimated_total_items: usize,
) *std.Progress.Node {
return progress.start(
name_ptr[0..name_len],
if (estimated_total_items == 0) null else estimated_total_items,
) catch @panic("timer unsupported");
}
// ABI warning
export fn stage2_progress_disable_tty(progress: *std.Progress) void {
progress.terminal = null;
}
// ABI warning
export fn stage2_progress_start(
node: *std.Progress.Node,
name_ptr: [*]const u8,
name_len: usize,
estimated_total_items: usize,
) *std.Progress.Node {
const child_node = std.heap.c_allocator.create(std.Progress.Node) catch @panic("out of memory");
child_node.* = node.start(
name_ptr[0..name_len],
if (estimated_total_items == 0) null else estimated_total_items,
);
child_node.activate();
return child_node;
}
// ABI warning
export fn stage2_progress_end(node: *std.Progress.Node) void {
node.end();
if (&node.context.root != node) {
std.heap.c_allocator.destroy(node);
}
}
// ABI warning
export fn stage2_progress_complete_one(node: *std.Progress.Node) void {
node.completeOne();
}
// ABI warning
export fn stage2_progress_update_node(node: *std.Progress.Node, done_count: usize, total_count: usize) void {
node.completed_items = done_count;
node.estimated_total_items = total_count;
node.activate();
node.context.maybeRefresh();
}
// ABI warning
const Stage2Target = extern struct {
arch: c_int,
vendor: c_int,
abi: c_int,
os: c_int,
is_native_os: bool,
is_native_cpu: bool,
llvm_cpu_name: ?[*:0]const u8,
llvm_cpu_features: ?[*:0]const u8,
cpu_builtin_str: ?[*:0]const u8,
os_builtin_str: ?[*:0]const u8,
dynamic_linker: ?[*:0]const u8,
llvm_cpu_features_asm_ptr: [*]const [*:0]const u8,
llvm_cpu_features_asm_len: usize,
fn fromTarget(self: *Stage2Target, cross_target: CrossTarget) !void {
const allocator = std.heap.c_allocator;
var dynamic_linker: ?[*:0]u8 = null;
const target = try crossTargetToTarget(cross_target, &dynamic_linker);
const generic_arch_name = target.cpu.arch.genericName();
var cpu_builtin_str_buffer = try std.ArrayListSentineled(u8, 0).allocPrint(allocator,
\\Cpu{{
\\ .arch = .{},
\\ .model = &Target.{}.cpu.{},
\\ .features = Target.{}.featureSet(&[_]Target.{}.Feature{{
\\
, .{
@tagName(target.cpu.arch),
generic_arch_name,
target.cpu.model.name,
generic_arch_name,
generic_arch_name,
});
defer cpu_builtin_str_buffer.deinit();
var llvm_features_buffer = try std.ArrayListSentineled(u8, 0).initSize(allocator, 0);
defer llvm_features_buffer.deinit();
// Unfortunately we have to do the work twice, because Clang does not support
// the same command line parameters for CPU features when assembling code as it does
// when compiling C code.
var asm_features_list = std.ArrayList([*:0]const u8).init(allocator);
defer asm_features_list.deinit();
for (target.cpu.arch.allFeaturesList()) |feature, index_usize| {
const index = @intCast(Target.Cpu.Feature.Set.Index, index_usize);
const is_enabled = target.cpu.features.isEnabled(index);
if (feature.llvm_name) |llvm_name| {
const plus_or_minus = "-+"[@boolToInt(is_enabled)];
try llvm_features_buffer.append(plus_or_minus);
try llvm_features_buffer.appendSlice(llvm_name);
try llvm_features_buffer.appendSlice(",");
}
if (is_enabled) {
// TODO some kind of "zig identifier escape" function rather than
// unconditionally using @"" syntax
try cpu_builtin_str_buffer.appendSlice(" .@\"");
try cpu_builtin_str_buffer.appendSlice(feature.name);
try cpu_builtin_str_buffer.appendSlice("\",\n");
}
}
switch (target.cpu.arch) {
.riscv32, .riscv64 => {
if (Target.riscv.featureSetHas(target.cpu.features, .relax)) {
try asm_features_list.append("-mrelax");
} else {
try asm_features_list.append("-mno-relax");
}
},
else => {
// TODO
// Argh, why doesn't the assembler accept the list of CPU features?!
// I don't see a way to do this other than hard coding everything.
},
}
try cpu_builtin_str_buffer.appendSlice(
\\ }),
\\};
\\
);
assert(mem.endsWith(u8, llvm_features_buffer.span(), ","));
llvm_features_buffer.shrink(llvm_features_buffer.len() - 1);
var os_builtin_str_buffer = try std.ArrayListSentineled(u8, 0).allocPrint(allocator,
\\Os{{
\\ .tag = .{},
\\ .version_range = .{{
, .{@tagName(target.os.tag)});
defer os_builtin_str_buffer.deinit();
// We'll re-use the OS version range builtin string for the cache hash.
const os_builtin_str_ver_start_index = os_builtin_str_buffer.len();
@setEvalBranchQuota(2000);
switch (target.os.tag) {
.freestanding,
.ananas,
.cloudabi,
.dragonfly,
.fuchsia,
.ios,
.kfreebsd,
.lv2,
.solaris,
.haiku,
.minix,
.rtems,
.nacl,
.cnk,
.aix,
.cuda,
.nvcl,
.amdhsa,
.ps4,
.elfiamcu,
.tvos,
.watchos,
.mesa3d,
.contiki,
.amdpal,
.hermit,
.hurd,
.wasi,
.emscripten,
.uefi,
.other,
=> try os_builtin_str_buffer.appendSlice(" .none = {} }\n"),
.freebsd,
.macosx,
.netbsd,
.openbsd,
=> try os_builtin_str_buffer.outStream().print(
\\ .semver = .{{
\\ .min = .{{
\\ .major = {},
\\ .minor = {},
\\ .patch = {},
\\ }},
\\ .max = .{{
\\ .major = {},
\\ .minor = {},
\\ .patch = {},
\\ }},
\\ }}}},
\\
, .{
target.os.version_range.semver.min.major,
target.os.version_range.semver.min.minor,
target.os.version_range.semver.min.patch,
target.os.version_range.semver.max.major,
target.os.version_range.semver.max.minor,
target.os.version_range.semver.max.patch,
}),
.linux => try os_builtin_str_buffer.outStream().print(
\\ .linux = .{{
\\ .range = .{{
\\ .min = .{{
\\ .major = {},
\\ .minor = {},
\\ .patch = {},
\\ }},
\\ .max = .{{
\\ .major = {},
\\ .minor = {},
\\ .patch = {},
\\ }},
\\ }},
\\ .glibc = .{{
\\ .major = {},
\\ .minor = {},
\\ .patch = {},
\\ }},
\\ }}}},
\\
, .{
target.os.version_range.linux.range.min.major,
target.os.version_range.linux.range.min.minor,
target.os.version_range.linux.range.min.patch,
target.os.version_range.linux.range.max.major,
target.os.version_range.linux.range.max.minor,
target.os.version_range.linux.range.max.patch,
target.os.version_range.linux.glibc.major,
target.os.version_range.linux.glibc.minor,
target.os.version_range.linux.glibc.patch,
}),
.windows => try os_builtin_str_buffer.outStream().print(
\\ .windows = .{{
\\ .min = {s},
\\ .max = {s},
\\ }}}},
\\
, .{
target.os.version_range.windows.min,
target.os.version_range.windows.max,
}),
}
try os_builtin_str_buffer.appendSlice("};\n");
const glibc_or_darwin_version = blk: {
if (target.isGnuLibC()) {
const stage1_glibc = try std.heap.c_allocator.create(Stage2SemVer);
const stage2_glibc = target.os.version_range.linux.glibc;
stage1_glibc.* = .{
.major = stage2_glibc.major,
.minor = stage2_glibc.minor,
.patch = stage2_glibc.patch,
};
break :blk stage1_glibc;
} else if (target.isDarwin()) {
const stage1_semver = try std.heap.c_allocator.create(Stage2SemVer);
const stage2_semver = target.os.version_range.semver.min;
stage1_semver.* = .{
.major = stage2_semver.major,
.minor = stage2_semver.minor,
.patch = stage2_semver.patch,
};
break :blk stage1_semver;
} else {
break :blk null;
}
};
const std_dl = target.standardDynamicLinkerPath();
const std_dl_z = if (std_dl.get()) |dl|
(try mem.dupeZ(std.heap.c_allocator, u8, dl)).ptr
else
null;
const asm_features = asm_features_list.toOwnedSlice();
self.* = .{
.arch = @enumToInt(target.cpu.arch) + 1, // skip over ZigLLVM_UnknownArch
.vendor = 0,
.os = @enumToInt(target.os.tag),
.abi = @enumToInt(target.abi),
.llvm_cpu_name = if (target.cpu.model.llvm_name) |s| s.ptr else null,
.llvm_cpu_features = llvm_features_buffer.toOwnedSlice().ptr,
.llvm_cpu_features_asm_ptr = asm_features.ptr,
.llvm_cpu_features_asm_len = asm_features.len,
.cpu_builtin_str = cpu_builtin_str_buffer.toOwnedSlice().ptr,
.os_builtin_str = os_builtin_str_buffer.toOwnedSlice().ptr,
.is_native_os = cross_target.isNativeOs(),
.is_native_cpu = cross_target.isNativeCpu(),
.glibc_or_darwin_version = glibc_or_darwin_version,
.dynamic_linker = dynamic_linker,
.standard_dynamic_linker_path = std_dl_z,
};
}
};
fn crossTargetToTarget(cross_target: CrossTarget, dynamic_linker_ptr: *?[*:0]u8) !Target {
var info = try std.zig.system.NativeTargetInfo.detect(std.heap.c_allocator, cross_target);
if (info.cpu_detection_unimplemented) {
// TODO We want to just use detected_info.target but implementing
// CPU model & feature detection is todo so here we rely on LLVM.
const llvm = @import("llvm.zig");
const llvm_cpu_name = llvm.GetHostCPUName();
const llvm_cpu_features = llvm.GetNativeFeatures();
const arch = Target.current.cpu.arch;
info.target.cpu = try detectNativeCpuWithLLVM(arch, llvm_cpu_name, llvm_cpu_features);
cross_target.updateCpuFeatures(&info.target.cpu.features);
info.target.cpu.arch = cross_target.getCpuArch();
}
if (info.dynamic_linker.get()) |dl| {
dynamic_linker_ptr.* = try mem.dupeZ(std.heap.c_allocator, u8, dl);
} else {
dynamic_linker_ptr.* = null;
}
return info.target;
}
// ABI warning
const Stage2SemVer = extern struct {
major: u32,
minor: u32,
patch: u32,
};

1054
src-self-hosted/stage2.zig
View File

File diff suppressed because it is too large Load Diff

135
src/all_types.hpp
View File

@ -10,7 +10,6 @@
#include "list.hpp"
#include "buffer.hpp"
#include "cache_hash.hpp"
#include "zig_llvm.h"
#include "hash_map.hpp"
#include "errmsg.hpp"
@ -1639,8 +1638,6 @@ struct ZigType {
size_t abi_size;
// Number of bits of information in this type. Known after ResolveStatusSizeKnown.
size_t size_in_bits;
bool gen_h_loop_flag;
};
enum FnAnalState {
@ -1976,67 +1973,16 @@ struct TimeEvent {
const char *name;
};
enum BuildMode {
BuildModeDebug,
BuildModeFastRelease,
BuildModeSafeRelease,
BuildModeSmallRelease,
};
enum CodeModel {
CodeModelDefault,
CodeModelTiny,
CodeModelSmall,
CodeModelKernel,
CodeModelMedium,
CodeModelLarge,
};
struct LinkLib {
Buf *name;
Buf *path;
ZigList<Buf *> symbols; // the list of symbols that we depend on from this lib
bool provided_explicitly;
};
enum ValgrindSupport {
ValgrindSupportAuto,
ValgrindSupportDisabled,
ValgrindSupportEnabled,
};
enum WantPIC {
WantPICAuto,
WantPICDisabled,
WantPICEnabled,
};
enum WantStackCheck {
WantStackCheckAuto,
WantStackCheckDisabled,
WantStackCheckEnabled,
};
enum WantCSanitize {
WantCSanitizeAuto,
WantCSanitizeDisabled,
WantCSanitizeEnabled,
};
enum OptionalBool {
OptionalBoolNull,
OptionalBoolFalse,
OptionalBoolTrue,
};
struct CFile {
ZigList<const char *> args;
const char *source_path;
const char *preprocessor_only_basename;
};
// When adding fields, check if they should be added to the hash computation in build_with_cache
struct CodeGen {
// Other code depends on this being first.
ZigStage1 stage1;
// arena allocator destroyed just prior to codegen emit
heap::ArenaAllocator *pass1_arena;
@ -2048,8 +1994,6 @@ struct CodeGen {
ZigLLVMDIBuilder *dbuilder;
ZigLLVMDICompileUnit *compile_unit;
ZigLLVMDIFile *compile_unit_file;
LinkLib *libc_link_lib;
LinkLib *libcpp_link_lib;
LLVMTargetDataRef target_data_ref;
LLVMTargetMachineRef target_machine;
ZigLLVMDIFile *dummy_di_file;
@ -2104,7 +2048,6 @@ struct CodeGen {
ZigList<ZigFn *> inline_fns;
ZigList<ZigFn *> test_fns;
ZigList<ErrorTableEntry *> errors_by_index;
ZigList<CacheHash *> caches_to_release;
size_t largest_err_name_len;
ZigList<ZigType *> type_resolve_stack;
@ -2173,7 +2116,6 @@ struct CodeGen {
Buf llvm_triple_str;
Buf global_asm;
Buf o_file_output_path;
Buf bin_file_output_path;
Buf asm_file_output_path;
Buf llvm_ir_file_output_path;
Buf *cache_dir;
@ -2184,7 +2126,7 @@ struct CodeGen {
const char **libc_include_dir_list;
size_t libc_include_dir_len;
Buf *zig_c_headers_dir; // Cannot be overridden; derived from zig_lib_dir.
Buf *builtin_zig_path;
Buf *zig_std_special_dir; // Cannot be overridden; derived from zig_lib_dir.
IrInstSrc *invalid_inst_src;
@ -2206,18 +2148,9 @@ struct CodeGen {
Stage2ProgressNode *main_progress_node;
Stage2ProgressNode *sub_progress_node;
WantPIC want_pic;
WantStackCheck want_stack_check;
WantCSanitize want_sanitize_c;
CacheHash cache_hash;
ErrColor err_color;
uint32_t next_unresolved_index;
unsigned pointer_size_bytes;
uint32_t target_os_index;
uint32_t target_arch_index;
uint32_t target_sub_arch_index;
uint32_t target_abi_index;
uint32_t target_oformat_index;
bool is_big_endian;
bool have_c_main;
bool have_winmain;
@ -2226,9 +2159,6 @@ struct CodeGen {
bool have_wwinmain_crt_startup;
bool have_dllmain_crt_startup;
bool have_err_ret_tracing;
bool link_eh_frame_hdr;
bool c_want_stdint;
bool c_want_stdbool;
bool verbose_tokenize;
bool verbose_ast;
bool verbose_link;
@ -2239,61 +2169,24 @@ struct CodeGen {
bool verbose_llvm_cpu_features;
bool error_during_imports;
bool generate_error_name_table;
bool enable_cache; // mutually exclusive with output_dir
bool enable_time_report;
bool enable_stack_report;
bool system_linker_hack;
bool reported_bad_link_libc_error;
bool is_dynamic; // shared library rather than static library. dynamic musl rather than static musl.
bool need_frame_size_prefix_data;
bool disable_c_depfile;
//////////////////////////// Participates in Input Parameter Cache Hash
/////// Note: there is a separate cache hash for builtin.zig, when adding fields,
/////// consider if they need to go into both.
ZigList<LinkLib *> link_libs_list;
// add -framework [name] args to linker
ZigList<Buf *> darwin_frameworks;
// add -rpath [name] args to linker
ZigList<Buf *> rpath_list;
ZigList<Buf *> forbidden_libs;
ZigList<Buf *> link_objects;
ZigList<Buf *> assembly_files;
ZigList<CFile *> c_source_files;
ZigList<const char *> lib_dirs;
ZigList<const char *> framework_dirs;
Stage2LibCInstallation *libc;
bool is_versioned;
size_t version_major;
size_t version_minor;
size_t version_patch;
const char *linker_script;
size_t stack_size_override;
bool link_libc;
bool link_libcpp;
BuildMode build_mode;
OutType out_type;
const ZigTarget *zig_target;
TargetSubsystem subsystem; // careful using this directly; see detect_subsystem
ValgrindSupport valgrind_support;
CodeModel code_model;
OptionalBool linker_gc_sections;
OptionalBool linker_allow_shlib_undefined;
OptionalBool linker_bind_global_refs_locally;
bool strip_debug_symbols;
bool is_test_build;
bool is_single_threaded;
bool want_single_threaded;
bool linker_rdynamic;
bool each_lib_rpath;
bool is_dummy_so;
bool disable_gen_h;
bool bundle_compiler_rt;
bool have_pic;
bool have_dynamic_link; // this is whether the final thing will be dynamically linked. see also is_dynamic
bool link_mode_dynamic;
bool dll_export_fns;
bool have_stack_probing;
bool have_sanitize_c;
bool function_sections;
bool enable_dump_analysis;
bool enable_doc_generation;
@ -2301,23 +2194,13 @@ struct CodeGen {
bool emit_asm;
bool emit_llvm_ir;
bool test_is_evented;
bool linker_z_nodelete;
bool linker_z_defs;
bool valgrind_enabled;
Buf *root_out_name;
Buf *test_filter;
Buf *test_name_prefix;
Buf *zig_lib_dir;
Buf *zig_std_dir;
Buf *version_script_path;
Buf *override_soname;
Buf *linker_optimization;
const char **llvm_argv;
size_t llvm_argv_len;
const char **clang_argv;
size_t clang_argv_len;
};
struct ZigVar {

49
src/analyze.cpp
View File

@ -3496,7 +3496,7 @@ void add_var_export(CodeGen *g, ZigVar *var, const char *symbol_name, GlobalLink
}
void add_fn_export(CodeGen *g, ZigFn *fn_table_entry, const char *symbol_name, GlobalLinkageId linkage, CallingConvention cc) {
if (cc == CallingConventionC && strcmp(symbol_name, "main") == 0 && g->libc_link_lib != nullptr) {
if (cc == CallingConventionC && strcmp(symbol_name, "main") == 0 && g->link_libc) {
g->have_c_main = true;
} else if (cc == CallingConventionStdcall && g->zig_target->os == OsWindows) {
if (strcmp(symbol_name, "WinMain") == 0) {
@ -7863,40 +7863,6 @@ const char *type_id_name(ZigTypeId id) {
zig_unreachable();
}
LinkLib *create_link_lib(Buf *name) {
LinkLib *link_lib = heap::c_allocator.create<LinkLib>();
link_lib->name = name;
return link_lib;
}
LinkLib *add_link_lib(CodeGen *g, Buf *name) {
bool is_libc = buf_eql_str(name, "c");
bool is_libcpp = buf_eql_str(name, "c++") || buf_eql_str(name, "c++abi");
if (is_libc && g->libc_link_lib != nullptr)
return g->libc_link_lib;
if (is_libcpp && g->libcpp_link_lib != nullptr)
return g->libcpp_link_lib;
for (size_t i = 0; i < g->link_libs_list.length; i += 1) {
LinkLib *existing_lib = g->link_libs_list.at(i);
if (buf_eql_buf(existing_lib->name, name)) {
return existing_lib;
}
}
LinkLib *link_lib = create_link_lib(name);
g->link_libs_list.append(link_lib);
if (is_libc)
g->libc_link_lib = link_lib;
if (is_libcpp)
g->libcpp_link_lib = link_lib;
return link_lib;
}
ZigType *get_align_amt_type(CodeGen *g) {
if (g->align_amt_type == nullptr) {
// according to LLVM the maximum alignment is 1 << 29.
@ -8017,12 +7983,13 @@ not_integer:
return ErrorNone;
}
Error file_fetch(CodeGen *g, Buf *resolved_path, Buf *contents) {
if (g->enable_cache) {
return cache_add_file_fetch(&g->cache_hash, resolved_path, contents);
} else {
return os_fetch_file_path(resolved_path, contents);
}
Error file_fetch(CodeGen *g, Buf *resolved_path, Buf *contents_buf) {
size_t len;
const char *contents = stage2_fetch_file(&g->stage1, buf_ptr(resolved_path), buf_len(resolved_path), &len);
if (contents == nullptr)
return ErrorNoMem;
buf_init_from_mem(contents_buf, contents, len);
return ErrorNone;
}
static X64CABIClass type_windows_abi_x86_64_class(CodeGen *g, ZigType *ty, size_t ty_size) {

3
src/analyze.hpp
View File

@ -200,8 +200,6 @@ ZigTypeId type_id_at_index(size_t index);
size_t type_id_len();
size_t type_id_index(ZigType *entry);
ZigType *get_generic_fn_type(CodeGen *g, FnTypeId *fn_type_id);
LinkLib *create_link_lib(Buf *name);
LinkLib *add_link_lib(CodeGen *codegen, Buf *lib);
bool optional_value_is_null(ZigValue *val);
uint32_t get_abi_alignment(CodeGen *g, ZigType *type_entry);
@ -256,7 +254,6 @@ Error ensure_const_val_repr(IrAnalyze *ira, CodeGen *codegen, AstNode *source_no
void typecheck_panic_fn(CodeGen *g, TldFn *tld_fn, ZigFn *panic_fn);
Buf *type_bare_name(ZigType *t);
Buf *type_h_name(ZigType *t);
Error create_c_object_cache(CodeGen *g, CacheHash **out_cache_hash, bool verbose);
LLVMTypeRef get_llvm_type(CodeGen *g, ZigType *type);
ZigLLVMDIType *get_llvm_di_type(CodeGen *g, ZigType *type);

196
src/blake2.h
View File

@ -1,196 +0,0 @@
/*
BLAKE2 reference source code package - reference C implementations
Copyright 2012, Samuel Neves <sneves@dei.uc.pt>. You may use this under the
terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at
your option. The terms of these licenses can be found at:
- CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0
- OpenSSL license : https://www.openssl.org/source/license.html
- Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0
More information about the BLAKE2 hash function can be found at
https://blake2.net.
*/
#ifndef BLAKE2_H
#define BLAKE2_H
#include <stddef.h>
#include <stdint.h>
#if defined(_MSC_VER)
#define BLAKE2_PACKED(x) __pragma(pack(push, 1)) x __pragma(pack(pop))
#else
#define BLAKE2_PACKED(x) x __attribute__((packed))
#endif
#if defined(__cplusplus)
extern "C" {
#endif
enum blake2s_constant
{
BLAKE2S_BLOCKBYTES = 64,
BLAKE2S_OUTBYTES = 32,
BLAKE2S_KEYBYTES = 32,
BLAKE2S_SALTBYTES = 8,
BLAKE2S_PERSONALBYTES = 8
};
enum blake2b_constant
{
BLAKE2B_BLOCKBYTES = 128,
BLAKE2B_OUTBYTES = 64,
BLAKE2B_KEYBYTES = 64,
BLAKE2B_SALTBYTES = 16,
BLAKE2B_PERSONALBYTES = 16
};
typedef struct blake2s_state__
{
uint32_t h[8];
uint32_t t[2];
uint32_t f[2];
uint8_t buf[BLAKE2S_BLOCKBYTES];
size_t buflen;
size_t outlen;
uint8_t last_node;
} blake2s_state;
typedef struct blake2b_state__
{
uint64_t h[8];
uint64_t t[2];
uint64_t f[2];
uint8_t buf[BLAKE2B_BLOCKBYTES];
size_t buflen;
size_t outlen;
uint8_t last_node;
} blake2b_state;
typedef struct blake2sp_state__
{
blake2s_state S[8][1];
blake2s_state R[1];
uint8_t buf[8 * BLAKE2S_BLOCKBYTES];
size_t buflen;
size_t outlen;
} blake2sp_state;
typedef struct blake2bp_state__
{
blake2b_state S[4][1];
blake2b_state R[1];
uint8_t buf[4 * BLAKE2B_BLOCKBYTES];
size_t buflen;
size_t outlen;
} blake2bp_state;
BLAKE2_PACKED(struct blake2s_param__
{
uint8_t digest_length; /* 1 */
uint8_t key_length; /* 2 */
uint8_t fanout; /* 3 */
uint8_t depth; /* 4 */
uint32_t leaf_length; /* 8 */
uint32_t node_offset; /* 12 */
uint16_t xof_length; /* 14 */
uint8_t node_depth; /* 15 */
uint8_t inner_length; /* 16 */
/* uint8_t reserved[0]; */
uint8_t salt[BLAKE2S_SALTBYTES]; /* 24 */
uint8_t personal[BLAKE2S_PERSONALBYTES]; /* 32 */
});
typedef struct blake2s_param__ blake2s_param;
BLAKE2_PACKED(struct blake2b_param__
{
uint8_t digest_length; /* 1 */
uint8_t key_length; /* 2 */
uint8_t fanout; /* 3 */
uint8_t depth; /* 4 */
uint32_t leaf_length; /* 8 */
uint32_t node_offset; /* 12 */
uint32_t xof_length; /* 16 */
uint8_t node_depth; /* 17 */
uint8_t inner_length; /* 18 */
uint8_t reserved[14]; /* 32 */
uint8_t salt[BLAKE2B_SALTBYTES]; /* 48 */
uint8_t personal[BLAKE2B_PERSONALBYTES]; /* 64 */
});
typedef struct blake2b_param__ blake2b_param;
typedef struct blake2xs_state__
{
blake2s_state S[1];
blake2s_param P[1];
} blake2xs_state;
typedef struct blake2xb_state__
{
blake2b_state S[1];
blake2b_param P[1];
} blake2xb_state;
/* Padded structs result in a compile-time error */
enum {
BLAKE2_DUMMY_1 = 1/(sizeof(blake2s_param) == BLAKE2S_OUTBYTES),
BLAKE2_DUMMY_2 = 1/(sizeof(blake2b_param) == BLAKE2B_OUTBYTES)
};
/* Streaming API */
int blake2s_init( blake2s_state *S, size_t outlen );
int blake2s_init_key( blake2s_state *S, size_t outlen, const void *key, size_t keylen );
int blake2s_init_param( blake2s_state *S, const blake2s_param *P );
int blake2s_update( blake2s_state *S, const void *in, size_t inlen );
int blake2s_final( blake2s_state *S, void *out, size_t outlen );
int blake2b_init( blake2b_state *S, size_t outlen );
int blake2b_init_key( blake2b_state *S, size_t outlen, const void *key, size_t keylen );
int blake2b_init_param( blake2b_state *S, const blake2b_param *P );
int blake2b_update( blake2b_state *S, const void *in, size_t inlen );
int blake2b_final( blake2b_state *S, void *out, size_t outlen );
int blake2sp_init( blake2sp_state *S, size_t outlen );
int blake2sp_init_key( blake2sp_state *S, size_t outlen, const void *key, size_t keylen );
int blake2sp_update( blake2sp_state *S, const void *in, size_t inlen );
int blake2sp_final( blake2sp_state *S, void *out, size_t outlen );
int blake2bp_init( blake2bp_state *S, size_t outlen );
int blake2bp_init_key( blake2bp_state *S, size_t outlen, const void *key, size_t keylen );
int blake2bp_update( blake2bp_state *S, const void *in, size_t inlen );
int blake2bp_final( blake2bp_state *S, void *out, size_t outlen );
/* Variable output length API */
int blake2xs_init( blake2xs_state *S, const size_t outlen );
int blake2xs_init_key( blake2xs_state *S, const size_t outlen, const void *key, size_t keylen );
int blake2xs_update( blake2xs_state *S, const void *in, size_t inlen );
int blake2xs_final(blake2xs_state *S, void *out, size_t outlen);
int blake2xb_init( blake2xb_state *S, const size_t outlen );
int blake2xb_init_key( blake2xb_state *S, const size_t outlen, const void *key, size_t keylen );
int blake2xb_update( blake2xb_state *S, const void *in, size_t inlen );
int blake2xb_final(blake2xb_state *S, void *out, size_t outlen);
/* Simple API */
int blake2s( void *out, size_t outlen, const void *in, size_t inlen, const void *key, size_t keylen );
int blake2b( void *out, size_t outlen, const void *in, size_t inlen, const void *key, size_t keylen );
int blake2sp( void *out, size_t outlen, const void *in, size_t inlen, const void *key, size_t keylen );
int blake2bp( void *out, size_t outlen, const void *in, size_t inlen, const void *key, size_t keylen );
int blake2xs( void *out, size_t outlen, const void *in, size_t inlen, const void *key, size_t keylen );
int blake2xb( void *out, size_t outlen, const void *in, size_t inlen, const void *key, size_t keylen );
/* This is simply an alias for blake2b */
int blake2( void *out, size_t outlen, const void *in, size_t inlen, const void *key, size_t keylen );
#if defined(__cplusplus)
}
#endif
#endif

539
src/blake2b.c
View File

@ -1,539 +0,0 @@
/*
BLAKE2 reference source code package - reference C implementations
Copyright 2012, Samuel Neves <sneves@dei.uc.pt>. You may use this under the
terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at
your option. The terms of these licenses can be found at:
- CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0
- OpenSSL license : https://www.openssl.org/source/license.html
- Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0
More information about the BLAKE2 hash function can be found at
https://blake2.net.
*/
#include <stdint.h>
#include <string.h>
#include <stdio.h>
#include "blake2.h"
/*
BLAKE2 reference source code package - reference C implementations
Copyright 2012, Samuel Neves <sneves@dei.uc.pt>. You may use this under the
terms of the CC0, the OpenSSL Licence, or the Apache Public License 2.0, at
your option. The terms of these licenses can be found at:
- CC0 1.0 Universal : http://creativecommons.org/publicdomain/zero/1.0
- OpenSSL license : https://www.openssl.org/source/license.html
- Apache 2.0 : http://www.apache.org/licenses/LICENSE-2.0
More information about the BLAKE2 hash function can be found at
https://blake2.net.
*/
#ifndef BLAKE2_IMPL_H
#define BLAKE2_IMPL_H
#include <stdint.h>
#include <string.h>
#if !defined(__cplusplus) && (!defined(__STDC_VERSION__) || __STDC_VERSION__ < 199901L)
#if defined(_MSC_VER)
#define BLAKE2_INLINE __inline
#elif defined(__GNUC__)
#define BLAKE2_INLINE __inline__
#else
#define BLAKE2_INLINE
#endif
#else
#define BLAKE2_INLINE inline
#endif
static BLAKE2_INLINE uint32_t load32( const void *src )
{
#if defined(NATIVE_LITTLE_ENDIAN)
uint32_t w;
memcpy(&w, src, sizeof w);
return w;
#else
const uint8_t *p = ( const uint8_t * )src;
return (( uint32_t )( p[0] ) << 0) |
(( uint32_t )( p[1] ) << 8) |
(( uint32_t )( p[2] ) << 16) |
(( uint32_t )( p[3] ) << 24) ;
#endif
}
static BLAKE2_INLINE uint64_t load64( const void *src )
{
#if defined(NATIVE_LITTLE_ENDIAN)
uint64_t w;
memcpy(&w, src, sizeof w);
return w;
#else
const uint8_t *p = ( const uint8_t * )src;
return (( uint64_t )( p[0] ) << 0) |
(( uint64_t )( p[1] ) << 8) |
(( uint64_t )( p[2] ) << 16) |
(( uint64_t )( p[3] ) << 24) |
(( uint64_t )( p[4] ) << 32) |
(( uint64_t )( p[5] ) << 40) |
(( uint64_t )( p[6] ) << 48) |
(( uint64_t )( p[7] ) << 56) ;
#endif
}
static BLAKE2_INLINE uint16_t load16( const void *src )
{
#if defined(NATIVE_LITTLE_ENDIAN)
uint16_t w;
memcpy(&w, src, sizeof w);
return w;
#else
const uint8_t *p = ( const uint8_t * )src;
return ( uint16_t )((( uint32_t )( p[0] ) << 0) |
(( uint32_t )( p[1] ) << 8));
#endif
}
static BLAKE2_INLINE void store16( void *dst, uint16_t w )
{
#if defined(NATIVE_LITTLE_ENDIAN)
memcpy(dst, &w, sizeof w);
#else
uint8_t *p = ( uint8_t * )dst;
*p++ = ( uint8_t )w; w >>= 8;
*p++ = ( uint8_t )w;
#endif
}
static BLAKE2_INLINE void store32( void *dst, uint32_t w )
{
#if defined(NATIVE_LITTLE_ENDIAN)
memcpy(dst, &w, sizeof w);
#else
uint8_t *p = ( uint8_t * )dst;
p[0] = (uint8_t)(w >> 0);
p[1] = (uint8_t)(w >> 8);
p[2] = (uint8_t)(w >> 16);
p[3] = (uint8_t)(w >> 24);
#endif
}
static BLAKE2_INLINE void store64( void *dst, uint64_t w )
{
#if defined(NATIVE_LITTLE_ENDIAN)
memcpy(dst, &w, sizeof w);
#else
uint8_t *p = ( uint8_t * )dst;
p[0] = (uint8_t)(w >> 0);
p[1] = (uint8_t)(w >> 8);
p[2] = (uint8_t)(w >> 16);
p[3] = (uint8_t)(w >> 24);
p[4] = (uint8_t)(w >> 32);
p[5] = (uint8_t)(w >> 40);
p[6] = (uint8_t)(w >> 48);
p[7] = (uint8_t)(w >> 56);
#endif
}
static BLAKE2_INLINE uint64_t load48( const void *src )
{
const uint8_t *p = ( const uint8_t * )src;
return (( uint64_t )( p[0] ) << 0) |
(( uint64_t )( p[1] ) << 8) |
(( uint64_t )( p[2] ) << 16) |
(( uint64_t )( p[3] ) << 24) |
(( uint64_t )( p[4] ) << 32) |
(( uint64_t )( p[5] ) << 40) ;
}
static BLAKE2_INLINE void store48( void *dst, uint64_t w )
{
uint8_t *p = ( uint8_t * )dst;
p[0] = (uint8_t)(w >> 0);
p[1] = (uint8_t)(w >> 8);
p[2] = (uint8_t)(w >> 16);
p[3] = (uint8_t)(w >> 24);
p[4] = (uint8_t)(w >> 32);
p[5] = (uint8_t)(w >> 40);
}
static BLAKE2_INLINE uint32_t rotr32( const uint32_t w, const unsigned c )
{
return ( w >> c ) | ( w << ( 32 - c ) );
}
static BLAKE2_INLINE uint64_t rotr64( const uint64_t w, const unsigned c )
{
return ( w >> c ) | ( w << ( 64 - c ) );
}
/* prevents compiler optimizing out memset() */
static BLAKE2_INLINE void secure_zero_memory(void *v, size_t n)
{
static void *(*const volatile memset_v)(void *, int, size_t) = &memset;
memset_v(v, 0, n);
}
#endif
static const uint64_t blake2b_IV[8] =
{
0x6a09e667f3bcc908ULL, 0xbb67ae8584caa73bULL,
0x3c6ef372fe94f82bULL, 0xa54ff53a5f1d36f1ULL,
0x510e527fade682d1ULL, 0x9b05688c2b3e6c1fULL,
0x1f83d9abfb41bd6bULL, 0x5be0cd19137e2179ULL
};
static const uint8_t blake2b_sigma[12][16] =
{
{ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 } ,
{ 14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3 } ,
{ 11, 8, 12, 0, 5, 2, 15, 13, 10, 14, 3, 6, 7, 1, 9, 4 } ,
{ 7, 9, 3, 1, 13, 12, 11, 14, 2, 6, 5, 10, 4, 0, 15, 8 } ,
{ 9, 0, 5, 7, 2, 4, 10, 15, 14, 1, 11, 12, 6, 8, 3, 13 } ,
{ 2, 12, 6, 10, 0, 11, 8, 3, 4, 13, 7, 5, 15, 14, 1, 9 } ,
{ 12, 5, 1, 15, 14, 13, 4, 10, 0, 7, 6, 3, 9, 2, 8, 11 } ,
{ 13, 11, 7, 14, 12, 1, 3, 9, 5, 0, 15, 4, 8, 6, 2, 10 } ,
{ 6, 15, 14, 9, 11, 3, 0, 8, 12, 2, 13, 7, 1, 4, 10, 5 } ,
{ 10, 2, 8, 4, 7, 6, 1, 5, 15, 11, 9, 14, 3, 12, 13 , 0 } ,
{ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 } ,
{ 14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3 }
};
static void blake2b_set_lastnode( blake2b_state *S )
{
S->f[1] = (uint64_t)-1;
}
/* Some helper functions, not necessarily useful */
static int blake2b_is_lastblock( const blake2b_state *S )
{
return S->f[0] != 0;
}
static void blake2b_set_lastblock( blake2b_state *S )
{
if( S->last_node ) blake2b_set_lastnode( S );
S->f[0] = (uint64_t)-1;
}
static void blake2b_increment_counter( blake2b_state *S, const uint64_t inc )
{
S->t[0] += inc;
S->t[1] += ( S->t[0] < inc );
}
static void blake2b_init0( blake2b_state *S )
{
size_t i;
memset( S, 0, sizeof( blake2b_state ) );
for( i = 0; i < 8; ++i ) S->h[i] = blake2b_IV[i];
}
/* init xors IV with input parameter block */
int blake2b_init_param( blake2b_state *S, const blake2b_param *P )
{
const uint8_t *p = ( const uint8_t * )( P );
size_t i;
blake2b_init0( S );
/* IV XOR ParamBlock */
for( i = 0; i < 8; ++i )
S->h[i] ^= load64( p + sizeof( S->h[i] ) * i );
S->outlen = P->digest_length;
return 0;
}
int blake2b_init( blake2b_state *S, size_t outlen )
{
blake2b_param P[1];
if ( ( !outlen ) || ( outlen > BLAKE2B_OUTBYTES ) ) return -1;
P->digest_length = (uint8_t)outlen;
P->key_length = 0;
P->fanout = 1;
P->depth = 1;
store32( &P->leaf_length, 0 );
store32( &P->node_offset, 0 );
store32( &P->xof_length, 0 );
P->node_depth = 0;
P->inner_length = 0;
memset( P->reserved, 0, sizeof( P->reserved ) );
memset( P->salt, 0, sizeof( P->salt ) );
memset( P->personal, 0, sizeof( P->personal ) );
return blake2b_init_param( S, P );
}
int blake2b_init_key( blake2b_state *S, size_t outlen, const void *key, size_t keylen )
{
blake2b_param P[1];
if ( ( !outlen ) || ( outlen > BLAKE2B_OUTBYTES ) ) return -1;
if ( !key || !keylen || keylen > BLAKE2B_KEYBYTES ) return -1;
P->digest_length = (uint8_t)outlen;
P->key_length = (uint8_t)keylen;
P->fanout = 1;
P->depth = 1;
store32( &P->leaf_length, 0 );
store32( &P->node_offset, 0 );
store32( &P->xof_length, 0 );
P->node_depth = 0;
P->inner_length = 0;
memset( P->reserved, 0, sizeof( P->reserved ) );
memset( P->salt, 0, sizeof( P->salt ) );
memset( P->personal, 0, sizeof( P->personal ) );
if( blake2b_init_param( S, P ) < 0 ) return -1;
{
uint8_t block[BLAKE2B_BLOCKBYTES];
memset( block, 0, BLAKE2B_BLOCKBYTES );
memcpy( block, key, keylen );
blake2b_update( S, block, BLAKE2B_BLOCKBYTES );
secure_zero_memory( block, BLAKE2B_BLOCKBYTES ); /* Burn the key from stack */
}
return 0;
}
#define G(r,i,a,b,c,d) \
do { \
a = a + b + m[blake2b_sigma[r][2*i+0]]; \
d = rotr64(d ^ a, 32); \
c = c + d; \
b = rotr64(b ^ c, 24); \
a = a + b + m[blake2b_sigma[r][2*i+1]]; \
d = rotr64(d ^ a, 16); \
c = c + d; \
b = rotr64(b ^ c, 63); \
} while(0)
#define ROUND(r) \
do { \
G(r,0,v[ 0],v[ 4],v[ 8],v[12]); \
G(r,1,v[ 1],v[ 5],v[ 9],v[13]); \
G(r,2,v[ 2],v[ 6],v[10],v[14]); \
G(r,3,v[ 3],v[ 7],v[11],v[15]); \
G(r,4,v[ 0],v[ 5],v[10],v[15]); \
G(r,5,v[ 1],v[ 6],v[11],v[12]); \
G(r,6,v[ 2],v[ 7],v[ 8],v[13]); \
G(r,7,v[ 3],v[ 4],v[ 9],v[14]); \
} while(0)
static void blake2b_compress( blake2b_state *S, const uint8_t block[BLAKE2B_BLOCKBYTES] )
{
uint64_t m[16];
uint64_t v[16];
size_t i;
for( i = 0; i < 16; ++i ) {
m[i] = load64( block + i * sizeof( m[i] ) );
}
for( i = 0; i < 8; ++i ) {
v[i] = S->h[i];
}
v[ 8] = blake2b_IV[0];
v[ 9] = blake2b_IV[1];
v[10] = blake2b_IV[2];
v[11] = blake2b_IV[3];
v[12] = blake2b_IV[4] ^ S->t[0];
v[13] = blake2b_IV[5] ^ S->t[1];
v[14] = blake2b_IV[6] ^ S->f[0];
v[15] = blake2b_IV[7] ^ S->f[1];
ROUND( 0 );
ROUND( 1 );
ROUND( 2 );
ROUND( 3 );
ROUND( 4 );
ROUND( 5 );
ROUND( 6 );
ROUND( 7 );
ROUND( 8 );
ROUND( 9 );
ROUND( 10 );
ROUND( 11 );
for( i = 0; i < 8; ++i ) {
S->h[i] = S->h[i] ^ v[i] ^ v[i + 8];
}
}
#undef G
#undef ROUND
int blake2b_update( blake2b_state *S, const void *pin, size_t inlen )
{
const unsigned char * in = (const unsigned char *)pin;
if( inlen > 0 )
{
size_t left = S->buflen;
size_t fill = BLAKE2B_BLOCKBYTES - left;
if( inlen > fill )
{
S->buflen = 0;
memcpy( S->buf + left, in, fill ); /* Fill buffer */
blake2b_increment_counter( S, BLAKE2B_BLOCKBYTES );
blake2b_compress( S, S->buf ); /* Compress */
in += fill; inlen -= fill;
while(inlen > BLAKE2B_BLOCKBYTES) {
blake2b_increment_counter(S, BLAKE2B_BLOCKBYTES);
blake2b_compress( S, in );
in += BLAKE2B_BLOCKBYTES;
inlen -= BLAKE2B_BLOCKBYTES;
}
}
memcpy( S->buf + S->buflen, in, inlen );
S->buflen += inlen;
}
return 0;
}
int blake2b_final( blake2b_state *S, void *out, size_t outlen )
{
uint8_t buffer[BLAKE2B_OUTBYTES] = {0};
size_t i;
if( out == NULL || outlen < S->outlen )
return -1;
if( blake2b_is_lastblock( S ) )
return -1;
blake2b_increment_counter( S, S->buflen );
blake2b_set_lastblock( S );
memset( S->buf + S->buflen, 0, BLAKE2B_BLOCKBYTES - S->buflen ); /* Padding */
blake2b_compress( S, S->buf );
for( i = 0; i < 8; ++i ) /* Output full hash to temp buffer */
store64( buffer + sizeof( S->h[i] ) * i, S->h[i] );
memcpy( out, buffer, S->outlen );
secure_zero_memory(buffer, sizeof(buffer));
return 0;
}
/* inlen, at least, should be uint64_t. Others can be size_t. */
int blake2b( void *out, size_t outlen, const void *in, size_t inlen, const void *key, size_t keylen )
{
blake2b_state S[1];
/* Verify parameters */
if ( NULL == in && inlen > 0 ) return -1;
if ( NULL == out ) return -1;
if( NULL == key && keylen > 0 ) return -1;
if( !outlen || outlen > BLAKE2B_OUTBYTES ) return -1;
if( keylen > BLAKE2B_KEYBYTES ) return -1;
if( keylen > 0 )
{
if( blake2b_init_key( S, outlen, key, keylen ) < 0 ) return -1;
}
else
{
if( blake2b_init( S, outlen ) < 0 ) return -1;
}
blake2b_update( S, ( const uint8_t * )in, inlen );
blake2b_final( S, out, outlen );
return 0;
}
int blake2( void *out, size_t outlen, const void *in, size_t inlen, const void *key, size_t keylen ) {
return blake2b(out, outlen, in, inlen, key, keylen);
}
#if defined(SUPERCOP)
int crypto_hash( unsigned char *out, unsigned char *in, unsigned long long inlen )
{
return blake2b( out, BLAKE2B_OUTBYTES, in, inlen, NULL, 0 );
}
#endif
#if defined(BLAKE2B_SELFTEST)
#include <string.h>
#include "blake2-kat.h"
int main( void )
{
uint8_t key[BLAKE2B_KEYBYTES];
uint8_t buf[BLAKE2_KAT_LENGTH];
size_t i, step;
for( i = 0; i < BLAKE2B_KEYBYTES; ++i )
key[i] = ( uint8_t )i;
for( i = 0; i < BLAKE2_KAT_LENGTH; ++i )
buf[i] = ( uint8_t )i;
/* Test simple API */
for( i = 0; i < BLAKE2_KAT_LENGTH; ++i )
{
uint8_t hash[BLAKE2B_OUTBYTES];
blake2b( hash, BLAKE2B_OUTBYTES, buf, i, key, BLAKE2B_KEYBYTES );
if( 0 != memcmp( hash, blake2b_keyed_kat[i], BLAKE2B_OUTBYTES ) )
{
goto fail;
}
}
/* Test streaming API */
for(step = 1; step < BLAKE2B_BLOCKBYTES; ++step) {
for (i = 0; i < BLAKE2_KAT_LENGTH; ++i) {
uint8_t hash[BLAKE2B_OUTBYTES];
blake2b_state S;
uint8_t * p = buf;
size_t mlen = i;
int err = 0;
if( (err = blake2b_init_key(&S, BLAKE2B_OUTBYTES, key, BLAKE2B_KEYBYTES)) < 0 ) {
goto fail;
}
while (mlen >= step) {
if ( (err = blake2b_update(&S, p, step)) < 0 ) {
goto fail;
}
mlen -= step;
p += step;
}
if ( (err = blake2b_update(&S, p, mlen)) < 0) {
goto fail;
}
if ( (err = blake2b_final(&S, hash, BLAKE2B_OUTBYTES)) < 0) {
goto fail;
}
if (0 != memcmp(hash, blake2b_keyed_kat[i], BLAKE2B_OUTBYTES)) {
goto fail;
}
}
}
puts( "ok" );
return 0;
fail:
puts("error");
return -1;
}
#endif

595
src/cache_hash.cpp
View File

@ -1,595 +0,0 @@
/*
* Copyright (c) 2018 Andrew Kelley
*
* This file is part of zig, which is MIT licensed.
* See http://opensource.org/licenses/MIT
*/
#include "stage2.h"
#include "cache_hash.hpp"
#include "all_types.hpp"
#include "buffer.hpp"
#include "os.hpp"
#include <stdio.h>
void cache_init(CacheHash *ch, Buf *manifest_dir) {
int rc = blake2b_init(&ch->blake, 48);
assert(rc == 0);
ch->files = {};
ch->manifest_dir = manifest_dir;
ch->manifest_file_path = nullptr;
ch->manifest_dirty = false;
ch->force_check_manifest = false;
ch->b64_digest = BUF_INIT;
}
void cache_mem(CacheHash *ch, const char *ptr, size_t len) {
assert(ch->manifest_file_path == nullptr);
assert(ptr != nullptr);
blake2b_update(&ch->blake, ptr, len);
}
void cache_slice(CacheHash *ch, Slice<const char> slice) {
// mix the length into the hash so that two juxtaposed cached slices can't collide
cache_usize(ch, slice.len);
cache_mem(ch, slice.ptr, slice.len);
}
void cache_str(CacheHash *ch, const char *ptr) {
// + 1 to include the null byte
cache_mem(ch, ptr, strlen(ptr) + 1);
}
void cache_int(CacheHash *ch, int x) {
assert(ch->manifest_file_path == nullptr);
// + 1 to include the null byte
uint8_t buf[sizeof(int) + 1];
memcpy(buf, &x, sizeof(int));
buf[sizeof(int)] = 0;
blake2b_update(&ch->blake, buf, sizeof(int) + 1);
}
void cache_usize(CacheHash *ch, size_t x) {
assert(ch->manifest_file_path == nullptr);
// + 1 to include the null byte
uint8_t buf[sizeof(size_t) + 1];
memcpy(buf, &x, sizeof(size_t));
buf[sizeof(size_t)] = 0;
blake2b_update(&ch->blake, buf, sizeof(size_t) + 1);
}
void cache_bool(CacheHash *ch, bool x) {
assert(ch->manifest_file_path == nullptr);
blake2b_update(&ch->blake, &x, 1);
}
void cache_buf(CacheHash *ch, Buf *buf) {
assert(ch->manifest_file_path == nullptr);
assert(buf != nullptr);
// + 1 to include the null byte
blake2b_update(&ch->blake, buf_ptr(buf), buf_len(buf) + 1);
}
void cache_buf_opt(CacheHash *ch, Buf *buf) {
assert(ch->manifest_file_path == nullptr);
if (buf == nullptr) {
cache_str(ch, "");
cache_str(ch, "");
} else {
cache_buf(ch, buf);
}
}
void cache_list_of_link_lib(CacheHash *ch, LinkLib **ptr, size_t len) {
assert(ch->manifest_file_path == nullptr);
for (size_t i = 0; i < len; i += 1) {
LinkLib *lib = ptr[i];
if (lib->provided_explicitly) {
cache_buf(ch, lib->name);
}
}
cache_str(ch, "");
}
void cache_list_of_buf(CacheHash *ch, Buf **ptr, size_t len) {
assert(ch->manifest_file_path == nullptr);
for (size_t i = 0; i < len; i += 1) {
Buf *buf = ptr[i];
cache_buf(ch, buf);
}
cache_str(ch, "");
}
void cache_list_of_file(CacheHash *ch, Buf **ptr, size_t len) {
assert(ch->manifest_file_path == nullptr);
for (size_t i = 0; i < len; i += 1) {
Buf *buf = ptr[i];
cache_file(ch, buf);
}
cache_str(ch, "");
}
void cache_list_of_str(CacheHash *ch, const char **ptr, size_t len) {
assert(ch->manifest_file_path == nullptr);
for (size_t i = 0; i < len; i += 1) {
const char *s = ptr[i];
cache_str(ch, s);
}
cache_str(ch, "");
}
void cache_file(CacheHash *ch, Buf *file_path) {
assert(ch->manifest_file_path == nullptr);
assert(file_path != nullptr);
Buf *resolved_path = buf_alloc();
*resolved_path = os_path_resolve(&file_path, 1);
CacheHashFile *chf = ch->files.add_one();
chf->path = resolved_path;
cache_buf(ch, resolved_path);
}
void cache_file_opt(CacheHash *ch, Buf *file_path) {
assert(ch->manifest_file_path == nullptr);
if (file_path == nullptr) {
cache_str(ch, "");
cache_str(ch, "");
} else {
cache_file(ch, file_path);
}
}
// Ported from std/base64.zig
static uint8_t base64_fs_alphabet[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_";
static void base64_encode(Slice<uint8_t> dest, Slice<uint8_t> source) {
size_t dest_len = ((source.len + 2) / 3) * 4;
assert(dest.len == dest_len);
size_t i = 0;
size_t out_index = 0;
for (; i + 2 < source.len; i += 3) {
dest.ptr[out_index] = base64_fs_alphabet[(source.ptr[i] >> 2) & 0x3f];
out_index += 1;
dest.ptr[out_index] = base64_fs_alphabet[((source.ptr[i] & 0x3) << 4) | ((source.ptr[i + 1] & 0xf0) >> 4)];
out_index += 1;
dest.ptr[out_index] = base64_fs_alphabet[((source.ptr[i + 1] & 0xf) << 2) | ((source.ptr[i + 2] & 0xc0) >> 6)];
out_index += 1;
dest.ptr[out_index] = base64_fs_alphabet[source.ptr[i + 2] & 0x3f];
out_index += 1;
}
// Assert that we never need pad characters.
assert(i == source.len);
}
// Ported from std/base64.zig
static Error base64_decode(Slice<uint8_t> dest, Slice<uint8_t> source) {
if (source.len % 4 != 0)
return ErrorInvalidFormat;
if (dest.len != (source.len / 4) * 3)
return ErrorInvalidFormat;
// In Zig this is comptime computed. In C++ it's not worth it to do that.
uint8_t char_to_index[256];
bool char_in_alphabet[256] = {0};
for (size_t i = 0; i < 64; i += 1) {
uint8_t c = base64_fs_alphabet[i];
assert(!char_in_alphabet[c]);
char_in_alphabet[c] = true;
char_to_index[c] = i;
}
size_t src_cursor = 0;
size_t dest_cursor = 0;
for (;src_cursor < source.len; src_cursor += 4) {
if (!char_in_alphabet[source.ptr[src_cursor + 0]]) return ErrorInvalidFormat;
if (!char_in_alphabet[source.ptr[src_cursor + 1]]) return ErrorInvalidFormat;
if (!char_in_alphabet[source.ptr[src_cursor + 2]]) return ErrorInvalidFormat;
if (!char_in_alphabet[source.ptr[src_cursor + 3]]) return ErrorInvalidFormat;
dest.ptr[dest_cursor + 0] = (char_to_index[source.ptr[src_cursor + 0]] << 2) | (char_to_index[source.ptr[src_cursor + 1]] >> 4);
dest.ptr[dest_cursor + 1] = (char_to_index[source.ptr[src_cursor + 1]] << 4) | (char_to_index[source.ptr[src_cursor + 2]] >> 2);
dest.ptr[dest_cursor + 2] = (char_to_index[source.ptr[src_cursor + 2]] << 6) | (char_to_index[source.ptr[src_cursor + 3]]);
dest_cursor += 3;
}
assert(src_cursor == source.len);
assert(dest_cursor == dest.len);
return ErrorNone;
}
static Error hash_file(uint8_t *digest, OsFile handle, Buf *contents) {
Error err;
if (contents) {
buf_resize(contents, 0);
}
blake2b_state blake;
int rc = blake2b_init(&blake, 48);
assert(rc == 0);
for (;;) {
uint8_t buf[4096];
size_t amt = 4096;
if ((err = os_file_read(handle, buf, &amt)))
return err;
if (amt == 0) {
rc = blake2b_final(&blake, digest, 48);
assert(rc == 0);
return ErrorNone;
}
blake2b_update(&blake, buf, amt);
if (contents) {
buf_append_mem(contents, (char*)buf, amt);
}
}
}
// If the wall clock time, rounded to the same precision as the
// mtime, is equal to the mtime, then we cannot rely on this mtime
// yet. We will instead save an mtime value that indicates the hash
// must be unconditionally computed.
static bool is_problematic_timestamp(const OsTimeStamp *fs_clock) {
OsTimeStamp wall_clock = os_timestamp_calendar();
// First make all the least significant zero bits in the fs_clock, also zero bits in the wall clock.
if (fs_clock->nsec == 0) {
wall_clock.nsec = 0;
if (fs_clock->sec == 0) {
wall_clock.sec = 0;
} else {
wall_clock.sec &= (-1ull) << ctzll(fs_clock->sec);
}
} else {
wall_clock.nsec &= (-1ull) << ctzll(fs_clock->nsec);
}
return wall_clock.nsec == fs_clock->nsec && wall_clock.sec == fs_clock->sec;
}
static Error populate_file_hash(CacheHash *ch, CacheHashFile *chf, Buf *contents) {
Error err;
assert(chf->path != nullptr);
OsFile this_file;
if ((err = os_file_open_r(chf->path, &this_file, &chf->attr)))
return err;
if (is_problematic_timestamp(&chf->attr.mtime)) {
chf->attr.mtime.sec = 0;
chf->attr.mtime.nsec = 0;
chf->attr.inode = 0;
}
if ((err = hash_file(chf->bin_digest, this_file, contents))) {
os_file_close(&this_file);
return err;
}
os_file_close(&this_file);
blake2b_update(&ch->blake, chf->bin_digest, 48);
return ErrorNone;
}
Error cache_hit(CacheHash *ch, Buf *out_digest) {
Error err;
uint8_t bin_digest[48];
int rc = blake2b_final(&ch->blake, bin_digest, 48);
assert(rc == 0);
buf_resize(&ch->b64_digest, 64);
base64_encode(buf_to_slice(&ch->b64_digest), {bin_digest, 48});
if (ch->files.length == 0 && !ch->force_check_manifest) {
buf_resize(out_digest, 64);
base64_encode(buf_to_slice(out_digest), {bin_digest, 48});
return ErrorNone;
}
rc = blake2b_init(&ch->blake, 48);
assert(rc == 0);
blake2b_update(&ch->blake, bin_digest, 48);
ch->manifest_file_path = buf_alloc();
os_path_join(ch->manifest_dir, &ch->b64_digest, ch->manifest_file_path);
buf_append_str(ch->manifest_file_path, ".txt");
if ((err = os_make_path(ch->manifest_dir)))
return err;
if ((err = os_file_open_lock_rw(ch->manifest_file_path, &ch->manifest_file)))
return err;
Buf line_buf = BUF_INIT;
buf_resize(&line_buf, 512);
if ((err = os_file_read_all(ch->manifest_file, &line_buf))) {
os_file_close(&ch->manifest_file);
return err;
}
size_t input_file_count = ch->files.length;
bool any_file_changed = false;
Error return_code = ErrorNone;
size_t file_i = 0;
SplitIterator line_it = memSplit(buf_to_slice(&line_buf), str("\n"));
for (;; file_i += 1) {
Optional<Slice<uint8_t>> opt_line = SplitIterator_next(&line_it);
CacheHashFile *chf;
if (file_i < input_file_count) {
chf = &ch->files.at(file_i);
} else if (any_file_changed) {
// cache miss.
// keep the manifest file open with the rw lock
// reset the hash
rc = blake2b_init(&ch->blake, 48);
assert(rc == 0);
blake2b_update(&ch->blake, bin_digest, 48);
ch->files.resize(input_file_count);
// bring the hash up to the input file hashes
for (file_i = 0; file_i < input_file_count; file_i += 1) {
blake2b_update(&ch->blake, ch->files.at(file_i).bin_digest, 48);
}
// caller can notice that out_digest is unmodified.
return return_code;
} else if (!opt_line.is_some) {
break;
} else {
chf = ch->files.add_one();
chf->path = nullptr;
}
if (!opt_line.is_some)
break;
SplitIterator it = memSplit(opt_line.value, str(" "));
Optional<Slice<uint8_t>> opt_inode = SplitIterator_next(&it);
if (!opt_inode.is_some) {
return_code = ErrorInvalidFormat;
break;
}
chf->attr.inode = strtoull((const char *)opt_inode.value.ptr, nullptr, 10);
Optional<Slice<uint8_t>> opt_mtime_sec = SplitIterator_next(&it);
if (!opt_mtime_sec.is_some) {
return_code = ErrorInvalidFormat;
break;
}
chf->attr.mtime.sec = strtoull((const char *)opt_mtime_sec.value.ptr, nullptr, 10);
Optional<Slice<uint8_t>> opt_mtime_nsec = SplitIterator_next(&it);
if (!opt_mtime_nsec.is_some) {
return_code = ErrorInvalidFormat;
break;
}
chf->attr.mtime.nsec = strtoull((const char *)opt_mtime_nsec.value.ptr, nullptr, 10);
Optional<Slice<uint8_t>> opt_digest = SplitIterator_next(&it);
if (!opt_digest.is_some) {
return_code = ErrorInvalidFormat;
break;
}
if ((err = base64_decode({chf->bin_digest, 48}, opt_digest.value))) {
return_code = ErrorInvalidFormat;
break;
}
Slice<uint8_t> file_path = SplitIterator_rest(&it);
if (file_path.len == 0) {
return_code = ErrorInvalidFormat;
break;
}
Buf *this_path = buf_create_from_slice(file_path);
if (chf->path != nullptr && !buf_eql_buf(this_path, chf->path)) {
return_code = ErrorInvalidFormat;
break;
}
chf->path = this_path;
// if the mtime matches we can trust the digest
OsFile this_file;
OsFileAttr actual_attr;
if ((err = os_file_open_r(chf->path, &this_file, &actual_attr))) {
fprintf(stderr, "Unable to open %s\n: %s", buf_ptr(chf->path), err_str(err));
os_file_close(&ch->manifest_file);
return ErrorCacheUnavailable;
}
if (chf->attr.mtime.sec == actual_attr.mtime.sec &&
chf->attr.mtime.nsec == actual_attr.mtime.nsec &&
chf->attr.inode == actual_attr.inode)
{
os_file_close(&this_file);
} else {
// we have to recompute the digest.
// later we'll rewrite the manifest with the new mtime/digest values
ch->manifest_dirty = true;
chf->attr = actual_attr;
if (is_problematic_timestamp(&actual_attr.mtime)) {
chf->attr.mtime.sec = 0;
chf->attr.mtime.nsec = 0;
chf->attr.inode = 0;
}
uint8_t actual_digest[48];
if ((err = hash_file(actual_digest, this_file, nullptr))) {
os_file_close(&this_file);
os_file_close(&ch->manifest_file);
return err;
}
os_file_close(&this_file);
if (memcmp(chf->bin_digest, actual_digest, 48) != 0) {
memcpy(chf->bin_digest, actual_digest, 48);
// keep going until we have the input file digests
any_file_changed = true;
}
}
if (!any_file_changed) {
blake2b_update(&ch->blake, chf->bin_digest, 48);
}
}
if (file_i < input_file_count || file_i == 0 || return_code != ErrorNone) {
// manifest file is empty or missing entries, so this is a cache miss
ch->manifest_dirty = true;
for (; file_i < input_file_count; file_i += 1) {
CacheHashFile *chf = &ch->files.at(file_i);
if ((err = populate_file_hash(ch, chf, nullptr))) {
fprintf(stderr, "Unable to hash %s: %s\n", buf_ptr(chf->path), err_str(err));
os_file_close(&ch->manifest_file);
return ErrorCacheUnavailable;
}
}
if (return_code != ErrorNone && return_code != ErrorInvalidFormat) {
os_file_close(&ch->manifest_file);
}
return return_code;
}
// Cache Hit
return cache_final(ch, out_digest);
}
Error cache_add_file_fetch(CacheHash *ch, Buf *resolved_path, Buf *contents) {
Error err;
assert(ch->manifest_file_path != nullptr);
CacheHashFile *chf = ch->files.add_one();
chf->path = resolved_path;
if ((err = populate_file_hash(ch, chf, contents))) {
os_file_close(&ch->manifest_file);
return err;
}
return ErrorNone;
}
Error cache_add_file(CacheHash *ch, Buf *path) {
Buf *resolved_path = buf_alloc();
*resolved_path = os_path_resolve(&path, 1);
return cache_add_file_fetch(ch, resolved_path, nullptr);
}
Error cache_add_dep_file(CacheHash *ch, Buf *dep_file_path, bool verbose) {
Error err;
Buf *contents = buf_alloc();
if ((err = os_fetch_file_path(dep_file_path, contents))) {
if (err == ErrorFileNotFound)
return err;
if (verbose) {
fprintf(stderr, "%s: unable to read .d file: %s\n", err_str(err), buf_ptr(dep_file_path));
}
return ErrorReadingDepFile;
}
auto it = stage2_DepTokenizer_init(buf_ptr(contents), buf_len(contents));
// skip first token: target
{
auto result = stage2_DepTokenizer_next(&it);
switch (result.type_id) {
case stage2_DepNextResult::error:
if (verbose) {
fprintf(stderr, "%s: failed processing .d file: %s\n", result.textz, buf_ptr(dep_file_path));
}
err = ErrorInvalidDepFile;
goto finish;
case stage2_DepNextResult::null:
err = ErrorNone;
goto finish;
case stage2_DepNextResult::target:
case stage2_DepNextResult::prereq:
err = ErrorNone;
break;
}
}
// Process 0+ preqreqs.
// clang is invoked in single-source mode so we never get more targets.
for (;;) {
auto result = stage2_DepTokenizer_next(&it);
switch (result.type_id) {
case stage2_DepNextResult::error:
if (verbose) {
fprintf(stderr, "%s: failed processing .d file: %s\n", result.textz, buf_ptr(dep_file_path));
}
err = ErrorInvalidDepFile;
goto finish;
case stage2_DepNextResult::null:
case stage2_DepNextResult::target:
err = ErrorNone;
goto finish;
case stage2_DepNextResult::prereq:
break;
}
auto textbuf = buf_alloc();
buf_init_from_str(textbuf, result.textz);
if ((err = cache_add_file(ch, textbuf))) {
if (verbose) {
fprintf(stderr, "unable to add %s to cache: %s\n", result.textz, err_str(err));
fprintf(stderr, "when processing .d file: %s\n", buf_ptr(dep_file_path));
}
goto finish;
}
}
finish:
stage2_DepTokenizer_deinit(&it);
return err;
}
static Error write_manifest_file(CacheHash *ch) {
Error err;
Buf contents = BUF_INIT;
buf_resize(&contents, 0);
uint8_t encoded_digest[65];
encoded_digest[64] = 0;
for (size_t i = 0; i < ch->files.length; i += 1) {
CacheHashFile *chf = &ch->files.at(i);
base64_encode({encoded_digest, 64}, {chf->bin_digest, 48});
buf_appendf(&contents, "%" ZIG_PRI_u64 " %" ZIG_PRI_u64 " %" ZIG_PRI_u64 " %s %s\n",
chf->attr.inode, chf->attr.mtime.sec, chf->attr.mtime.nsec, encoded_digest, buf_ptr(chf->path));
}
if ((err = os_file_overwrite(ch->manifest_file, &contents)))
return err;
return ErrorNone;
}
Error cache_final(CacheHash *ch, Buf *out_digest) {
assert(ch->manifest_file_path != nullptr);
// We don't close the manifest file yet, because we want to
// keep it locked until the API user is done using it.
// We also don't write out the manifest yet, because until
// cache_release is called we still might be working on creating
// the artifacts to cache.
uint8_t bin_digest[48];
int rc = blake2b_final(&ch->blake, bin_digest, 48);
assert(rc == 0);
buf_resize(out_digest, 64);
base64_encode(buf_to_slice(out_digest), {bin_digest, 48});
return ErrorNone;
}
void cache_release(CacheHash *ch) {
assert(ch->manifest_file_path != nullptr);
Error err;
if (ch->manifest_dirty) {
if ((err = write_manifest_file(ch))) {
fprintf(stderr, "Warning: Unable to write cache file '%s': %s\n",
buf_ptr(ch->manifest_file_path), err_str(err));
}
}
os_file_close(&ch->manifest_file);
}

83
src/cache_hash.hpp
View File

@ -1,83 +0,0 @@
/*
* Copyright (c) 2018 Andrew Kelley
*
* This file is part of zig, which is MIT licensed.
* See http://opensource.org/licenses/MIT
*/
#ifndef ZIG_CACHE_HASH_HPP
#define ZIG_CACHE_HASH_HPP
#include "blake2.h"
#include "os.hpp"
struct LinkLib;
struct CacheHashFile {
Buf *path;
OsFileAttr attr;
uint8_t bin_digest[48];
Buf *contents;
};
struct CacheHash {
blake2b_state blake;
ZigList<CacheHashFile> files;
Buf *manifest_dir;
Buf *manifest_file_path;
Buf b64_digest;
OsFile manifest_file;
bool manifest_dirty;
bool force_check_manifest;
};
// Always call this first to set up.
void cache_init(CacheHash *ch, Buf *manifest_dir);
// Next, use the hash population functions to add the initial parameters.
void cache_mem(CacheHash *ch, const char *ptr, size_t len);
void cache_slice(CacheHash *ch, Slice<const char> slice);
void cache_str(CacheHash *ch, const char *ptr);
void cache_int(CacheHash *ch, int x);
void cache_bool(CacheHash *ch, bool x);
void cache_usize(CacheHash *ch, size_t x);
void cache_buf(CacheHash *ch, Buf *buf);
void cache_buf_opt(CacheHash *ch, Buf *buf);
void cache_list_of_link_lib(CacheHash *ch, LinkLib **ptr, size_t len);
void cache_list_of_buf(CacheHash *ch, Buf **ptr, size_t len);
void cache_list_of_file(CacheHash *ch, Buf **ptr, size_t len);
void cache_list_of_str(CacheHash *ch, const char **ptr, size_t len);
void cache_file(CacheHash *ch, Buf *path);
void cache_file_opt(CacheHash *ch, Buf *path);
// Then call cache_hit when you're ready to see if you can skip the next step.
// out_b64_digest will be left unchanged if it was a cache miss.
// If you got a cache hit, the next step is cache_release.
// From this point on, there is a lock on the input params. Release
// the lock with cache_release.
// Set force_check_manifest if you plan to add files later, but have not
// added any files before calling cache_hit. CacheHash::b64_digest becomes
// available for use after this call, even in the case of a miss, and it
// is a hash of the input parameters only.
// If this function returns ErrorInvalidFormat, that error may be treated
// as a cache miss.
Error ATTRIBUTE_MUST_USE cache_hit(CacheHash *ch, Buf *out_b64_digest);
// If you did not get a cache hit, call this function for every file
// that is depended on, and then finish with cache_final.
Error ATTRIBUTE_MUST_USE cache_add_file(CacheHash *ch, Buf *path);
// This opens a file created by -MD -MF args to Clang
Error ATTRIBUTE_MUST_USE cache_add_dep_file(CacheHash *ch, Buf *path, bool verbose);
// This variant of cache_add_file returns the file contents.
// Also the file path argument must be already resolved.
Error ATTRIBUTE_MUST_USE cache_add_file_fetch(CacheHash *ch, Buf *resolved_path, Buf *contents);
// out_b64_digest will be the same thing that cache_hit returns if you got a cache hit
Error ATTRIBUTE_MUST_USE cache_final(CacheHash *ch, Buf *out_b64_digest);
// Until this function is called, no one will be able to get a lock on your input params.
void cache_release(CacheHash *ch);
#endif

2069
src/codegen.cpp
View File

File diff suppressed because it is too large Load Diff

35
src/codegen.hpp
View File

@ -16,47 +16,20 @@
#include <stdio.h>
CodeGen *codegen_create(Buf *main_pkg_path, Buf *root_src_path, const ZigTarget *target,
OutType out_type, BuildMode build_mode, Buf *zig_lib_dir,
Stage2LibCInstallation *libc, Buf *cache_dir, bool is_test_build, Stage2ProgressNode *progress_node);
BuildMode build_mode, Buf *zig_lib_dir,
bool is_test_build, Stage2ProgressNode *progress_node);
CodeGen *create_child_codegen(CodeGen *parent_gen, Buf *root_src_path, OutType out_type,
Stage2LibCInstallation *libc, const char *name, Stage2ProgressNode *progress_node);
void codegen_build_object(CodeGen *g);
void codegen_destroy(CodeGen *);
void codegen_set_clang_argv(CodeGen *codegen, const char **args, size_t len);
void codegen_set_llvm_argv(CodeGen *codegen, const char **args, size_t len);
void codegen_set_each_lib_rpath(CodeGen *codegen, bool each_lib_rpath);
void codegen_set_strip(CodeGen *codegen, bool strip);
void codegen_set_errmsg_color(CodeGen *codegen, ErrColor err_color);
void codegen_set_out_name(CodeGen *codegen, Buf *out_name);
void codegen_add_lib_dir(CodeGen *codegen, const char *dir);
void codegen_add_forbidden_lib(CodeGen *codegen, Buf *lib);
LinkLib *codegen_add_link_lib(CodeGen *codegen, Buf *lib);
void codegen_add_framework(CodeGen *codegen, const char *name);
void codegen_add_rpath(CodeGen *codegen, const char *name);
void codegen_set_rdynamic(CodeGen *g, bool rdynamic);
void codegen_set_linker_script(CodeGen *g, const char *linker_script);
void codegen_set_test_filter(CodeGen *g, Buf *filter);
void codegen_set_test_name_prefix(CodeGen *g, Buf *prefix);
void codegen_set_lib_version(CodeGen *g, bool is_versioned, size_t major, size_t minor, size_t patch);
void codegen_add_time_event(CodeGen *g, const char *name);
void codegen_print_timing_report(CodeGen *g, FILE *f);
void codegen_link(CodeGen *g);
void zig_link_add_compiler_rt(CodeGen *g, Stage2ProgressNode *progress_node);
void codegen_build_and_link(CodeGen *g);
ZigPackage *codegen_create_package(CodeGen *g, const char *root_src_dir, const char *root_src_path,
const char *pkg_path);
void codegen_add_assembly(CodeGen *g, Buf *path);
void codegen_add_object(CodeGen *g, Buf *object_path);
void codegen_translate_c(CodeGen *g, Buf *full_path);
Buf *codegen_generate_builtin_source(CodeGen *g);
TargetSubsystem detect_subsystem(CodeGen *g);
void codegen_release_caches(CodeGen *codegen);
bool codegen_fn_has_err_ret_tracing_arg(CodeGen *g, ZigType *return_type);
bool codegen_fn_has_err_ret_tracing_stack(CodeGen *g, ZigFn *fn, bool is_async);

196
src/compiler.cpp
View File

@ -1,196 +0,0 @@
#include "cache_hash.hpp"
#include "os.hpp"
#include "compiler.hpp"
#include <stdio.h>
Error get_compiler_id(Buf **result) {
static Buf saved_compiler_id = BUF_INIT;
if (saved_compiler_id.list.length != 0) {
*result = &saved_compiler_id;
return ErrorNone;
}
Error err;
Buf *manifest_dir = buf_alloc();
os_path_join(get_global_cache_dir(), buf_create_from_str("exe"), manifest_dir);
CacheHash cache_hash;
CacheHash *ch = &cache_hash;
cache_init(ch, manifest_dir);
Buf self_exe_path = BUF_INIT;
if ((err = os_self_exe_path(&self_exe_path)))
return err;
cache_file(ch, &self_exe_path);
buf_resize(&saved_compiler_id, 0);
if ((err = cache_hit(ch, &saved_compiler_id))) {
if (err != ErrorInvalidFormat)
return err;
}
if (buf_len(&saved_compiler_id) != 0) {
cache_release(ch);
*result = &saved_compiler_id;
return ErrorNone;
}
ZigList<Buf *> lib_paths = {};
if ((err = os_self_exe_shared_libs(lib_paths)))
return err;
#if defined(ZIG_OS_DARWIN)
// only add the self exe path on mac os
Buf *lib_path = lib_paths.at(0);
if ((err = cache_add_file(ch, lib_path)))
return err;
#else
for (size_t i = 0; i < lib_paths.length; i += 1) {
Buf *lib_path = lib_paths.at(i);
if ((err = cache_add_file(ch, lib_path)))
return err;
}
#endif
if ((err = cache_final(ch, &saved_compiler_id)))
return err;
cache_release(ch);
*result = &saved_compiler_id;
return ErrorNone;
}
static bool test_zig_install_prefix(Buf *test_path, Buf *out_zig_lib_dir) {
{
Buf *test_zig_dir = buf_sprintf("%s" OS_SEP "lib" OS_SEP "zig", buf_ptr(test_path));
Buf *test_index_file = buf_sprintf("%s" OS_SEP "std" OS_SEP "std.zig", buf_ptr(test_zig_dir));
int err;
bool exists;
if ((err = os_file_exists(test_index_file, &exists))) {
exists = false;
}
if (exists) {
buf_init_from_buf(out_zig_lib_dir, test_zig_dir);
return true;
}
}
// Also try without "zig"
{
Buf *test_zig_dir = buf_sprintf("%s" OS_SEP "lib", buf_ptr(test_path));
Buf *test_index_file = buf_sprintf("%s" OS_SEP "std" OS_SEP "std.zig", buf_ptr(test_zig_dir));
int err;
bool exists;
if ((err = os_file_exists(test_index_file, &exists))) {
exists = false;
}
if (exists) {
buf_init_from_buf(out_zig_lib_dir, test_zig_dir);
return true;
}
}
return false;
}
static int find_zig_lib_dir(Buf *out_path) {
int err;
Buf self_exe_path = BUF_INIT;
buf_resize(&self_exe_path, 0);
if (!(err = os_self_exe_path(&self_exe_path))) {
Buf *cur_path = &self_exe_path;
for (;;) {
Buf *test_dir = buf_alloc();
os_path_dirname(cur_path, test_dir);
if (buf_eql_buf(test_dir, cur_path)) {
break;
}
if (test_zig_install_prefix(test_dir, out_path)) {
return 0;
}
cur_path = test_dir;
}
}
return ErrorFileNotFound;
}
Buf *get_zig_lib_dir(void) {
static Buf saved_lib_dir = BUF_INIT;
if (saved_lib_dir.list.length != 0)
return &saved_lib_dir;
buf_resize(&saved_lib_dir, 0);
int err;
if ((err = find_zig_lib_dir(&saved_lib_dir))) {
fprintf(stderr, "Unable to find zig lib directory\n");
exit(EXIT_FAILURE);
}
return &saved_lib_dir;
}
Buf *get_zig_std_dir(Buf *zig_lib_dir) {
static Buf saved_std_dir = BUF_INIT;
if (saved_std_dir.list.length != 0)
return &saved_std_dir;
buf_resize(&saved_std_dir, 0);
os_path_join(zig_lib_dir, buf_create_from_str("std"), &saved_std_dir);
return &saved_std_dir;
}
Buf *get_zig_special_dir(Buf *zig_lib_dir) {
static Buf saved_special_dir = BUF_INIT;
if (saved_special_dir.list.length != 0)
return &saved_special_dir;
buf_resize(&saved_special_dir, 0);
os_path_join(get_zig_std_dir(zig_lib_dir), buf_sprintf("special"), &saved_special_dir);
return &saved_special_dir;
}
Buf *get_global_cache_dir(void) {
static Buf saved_global_cache_dir = BUF_INIT;
if (saved_global_cache_dir.list.length != 0)
return &saved_global_cache_dir;
buf_resize(&saved_global_cache_dir, 0);
Buf app_data_dir = BUF_INIT;
Error err;
if ((err = os_get_app_data_dir(&app_data_dir, "zig"))) {
fprintf(stderr, "Unable to get application data dir: %s\n", err_str(err));
exit(1);
}
os_path_join(&app_data_dir, buf_create_from_str("stage1"), &saved_global_cache_dir);
buf_deinit(&app_data_dir);
return &saved_global_cache_dir;
}
FileExt classify_file_ext(const char *filename_ptr, size_t filename_len) {
if (mem_ends_with_str(filename_ptr, filename_len, ".c")) {
return FileExtC;
} else if (mem_ends_with_str(filename_ptr, filename_len, ".C") ||
mem_ends_with_str(filename_ptr, filename_len, ".cc") ||
mem_ends_with_str(filename_ptr, filename_len, ".cpp") ||
mem_ends_with_str(filename_ptr, filename_len, ".cxx"))
{
return FileExtCpp;
} else if (mem_ends_with_str(filename_ptr, filename_len, ".ll")) {
return FileExtLLVMIr;
} else if (mem_ends_with_str(filename_ptr, filename_len, ".bc")) {
return FileExtLLVMBitCode;
} else if (mem_ends_with_str(filename_ptr, filename_len, ".s") ||
mem_ends_with_str(filename_ptr, filename_len, ".S"))
{
return FileExtAsm;
}
// TODO look for .so, .so.X, .so.X.Y, .so.X.Y.Z
return FileExtUnknown;
}

24
src/compiler.hpp
View File

@ -1,24 +0,0 @@
/*
* Copyright (c) 2018 Andrew Kelley
*
* This file is part of zig, which is MIT licensed.
* See http://opensource.org/licenses/MIT
*/
#ifndef ZIG_COMPILER_HPP
#define ZIG_COMPILER_HPP
#include "all_types.hpp"
Error get_compiler_id(Buf **result);
Buf *get_zig_lib_dir(void);
Buf *get_zig_special_dir(Buf *zig_lib_dir);
Buf *get_zig_std_dir(Buf *zig_lib_dir);
Buf *get_global_cache_dir(void);
FileExt classify_file_ext(const char *filename_ptr, size_t filename_len);
#endif

2
src/config.h.in
View File

@ -22,6 +22,4 @@
#define ZIG_LLVM_CONFIG_EXE "@LLVM_CONFIG_EXE@"
#define ZIG_DIA_GUIDS_LIB "@ZIG_DIA_GUIDS_LIB_ESCAPED@"
#cmakedefine ZIG_ENABLE_MEM_PROFILE
#endif

1
src/config.zig.in
View File

@ -3,3 +3,4 @@ pub const version: []const u8 = "@ZIG_VERSION@";
pub const log_scopes: []const []const u8 = &[_][]const u8{};
pub const zir_dumps: []const []const u8 = &[_][]const u8{};
pub const enable_tracy = false;
pub const is_stage1 = true;

12
src/dump_analysis.cpp
View File

@ -6,11 +6,11 @@
*/
#include "dump_analysis.hpp"
#include "compiler.hpp"
#include "analyze.hpp"
#include "config.h"
#include "ir.hpp"
#include "codegen.hpp"
#include "os.hpp"
enum JsonWriterState {
JsonWriterStateInvalid,
@ -1173,7 +1173,6 @@ static void anal_dump_fn(AnalDumpCtx *ctx, ZigFn *fn) {
}
void zig_print_analysis_dump(CodeGen *g, FILE *f, const char *one_indent, const char *nl) {
Error err;
AnalDumpCtx ctx = {};
ctx.g = g;
JsonWriter *jw = &ctx.jw;
@ -1199,15 +1198,6 @@ void zig_print_analysis_dump(CodeGen *g, FILE *f, const char *one_indent, const
jw_object_field(jw, "params");
jw_begin_object(jw);
{
jw_object_field(jw, "zigId");
Buf *compiler_id;
if ((err = get_compiler_id(&compiler_id))) {
fprintf(stderr, "Unable to determine compiler id: %s\n", err_str(err));
exit(1);
}
jw_string(jw, buf_ptr(compiler_id));
jw_object_field(jw, "zigVersion");
jw_string(jw, ZIG_VERSION_STRING);

0
src/empty.cpp Normal file
View File

7
src/errmsg.hpp
View File

@ -10,12 +10,7 @@
#include "buffer.hpp"
#include "list.hpp"
enum ErrColor {
ErrColorAuto,
ErrColorOff,
ErrColorOn,
};
#include "stage1.h"
struct ErrorMsg {
size_t line_start;

392
src/glibc.cpp
View File

@ -1,392 +0,0 @@
/*
* Copyright (c) 2019 Andrew Kelley
*
* This file is part of zig, which is MIT licensed.
* See http://opensource.org/licenses/MIT
*/
#include "glibc.hpp"
#include "compiler.hpp"
#include "cache_hash.hpp"
#include "codegen.hpp"
static const ZigGLibCLib glibc_libs[] = {
{"c", 6},
{"m", 6},
{"pthread", 0},
{"dl", 2},
{"rt", 1},
{"ld", 2},
{"util", 1},
};
Error glibc_load_metadata(ZigGLibCAbi **out_result, Buf *zig_lib_dir, bool verbose) {
Error err;
ZigGLibCAbi *glibc_abi = heap::c_allocator.create<ZigGLibCAbi>();
glibc_abi->vers_txt_path = buf_sprintf("%s" OS_SEP "libc" OS_SEP "glibc" OS_SEP "vers.txt", buf_ptr(zig_lib_dir));
glibc_abi->fns_txt_path = buf_sprintf("%s" OS_SEP "libc" OS_SEP "glibc" OS_SEP "fns.txt", buf_ptr(zig_lib_dir));
glibc_abi->abi_txt_path = buf_sprintf("%s" OS_SEP "libc" OS_SEP "glibc" OS_SEP "abi.txt", buf_ptr(zig_lib_dir));
glibc_abi->version_table.init(16);
Buf *vers_txt_contents = buf_alloc();
if ((err = os_fetch_file_path(glibc_abi->vers_txt_path, vers_txt_contents))) {
if (verbose) {
fprintf(stderr, "Unable to read %s: %s\n", buf_ptr(glibc_abi->vers_txt_path), err_str(err));
}
return err;
}
Buf *fns_txt_contents = buf_alloc();
if ((err = os_fetch_file_path(glibc_abi->fns_txt_path, fns_txt_contents))) {
if (verbose) {
fprintf(stderr, "Unable to read %s: %s\n", buf_ptr(glibc_abi->fns_txt_path), err_str(err));
}
return err;
}
Buf *abi_txt_contents = buf_alloc();
if ((err = os_fetch_file_path(glibc_abi->abi_txt_path, abi_txt_contents))) {
if (verbose) {
fprintf(stderr, "Unable to read %s: %s\n", buf_ptr(glibc_abi->abi_txt_path), err_str(err));
}
return err;
}
{
SplitIterator it = memSplit(buf_to_slice(vers_txt_contents), str("\r\n"));
for (;;) {
Optional<Slice<uint8_t>> opt_component = SplitIterator_next(&it);
if (!opt_component.is_some) break;
Buf *ver_buf = buf_create_from_slice(opt_component.value);
Stage2SemVer *this_ver = glibc_abi->all_versions.add_one();
if ((err = target_parse_glibc_version(this_ver, buf_ptr(ver_buf)))) {
if (verbose) {
fprintf(stderr, "Unable to parse glibc version '%s': %s\n", buf_ptr(ver_buf), err_str(err));
}
return err;
}
}
}
{
SplitIterator it = memSplit(buf_to_slice(fns_txt_contents), str("\r\n"));
for (;;) {
Optional<Slice<uint8_t>> opt_component = SplitIterator_next(&it);
if (!opt_component.is_some) break;
SplitIterator line_it = memSplit(opt_component.value, str(" "));
Optional<Slice<uint8_t>> opt_fn_name = SplitIterator_next(&line_it);
if (!opt_fn_name.is_some) {
if (verbose) {
fprintf(stderr, "%s: Expected function name\n", buf_ptr(glibc_abi->fns_txt_path));
}
return ErrorInvalidFormat;
}
Optional<Slice<uint8_t>> opt_lib_name = SplitIterator_next(&line_it);
if (!opt_lib_name.is_some) {
if (verbose) {
fprintf(stderr, "%s: Expected lib name\n", buf_ptr(glibc_abi->fns_txt_path));
}
return ErrorInvalidFormat;
}
Buf *this_fn_name = buf_create_from_slice(opt_fn_name.value);
Buf *this_lib_name = buf_create_from_slice(opt_lib_name.value);
glibc_abi->all_functions.append({ this_fn_name, glibc_lib_find(buf_ptr(this_lib_name)) });
}
}
{
SplitIterator it = memSplit(buf_to_slice(abi_txt_contents), str("\r\n"));
ZigGLibCVerList *ver_list_base = nullptr;
int line_num = 0;
for (;;) {
if (ver_list_base == nullptr) {
line_num += 1;
Optional<Slice<uint8_t>> opt_line = SplitIterator_next_separate(&it);
if (!opt_line.is_some) break;
ver_list_base = heap::c_allocator.allocate<ZigGLibCVerList>(glibc_abi->all_functions.length);
SplitIterator line_it = memSplit(opt_line.value, str(" "));
for (;;) {
ZigTarget *target = heap::c_allocator.create<ZigTarget>();
Optional<Slice<uint8_t>> opt_target = SplitIterator_next(&line_it);
if (!opt_target.is_some) break;
SplitIterator component_it = memSplit(opt_target.value, str("-"));
Optional<Slice<uint8_t>> opt_arch = SplitIterator_next(&component_it);
assert(opt_arch.is_some);
Optional<Slice<uint8_t>> opt_os = SplitIterator_next(&component_it);
assert(opt_os.is_some); // it's always "linux" so we ignore it
Optional<Slice<uint8_t>> opt_abi = SplitIterator_next(&component_it);
assert(opt_abi.is_some);
err = target_parse_arch(&target->arch, (char*)opt_arch.value.ptr, opt_arch.value.len);
assert(err == ErrorNone);
target->os = OsLinux;
err = target_parse_abi(&target->abi, (char*)opt_abi.value.ptr, opt_abi.value.len);
if (err != ErrorNone) {
fprintf(stderr, "Error parsing %s:%d: %s\n", buf_ptr(glibc_abi->abi_txt_path),
line_num, err_str(err));
fprintf(stderr, "arch: '%.*s', os: '%.*s', abi: '%.*s'\n",
(int)opt_arch.value.len, (const char*)opt_arch.value.ptr,
(int)opt_os.value.len, (const char*)opt_os.value.ptr,
(int)opt_abi.value.len, (const char*)opt_abi.value.ptr);
fprintf(stderr, "parsed from target: '%.*s'\n",
(int)opt_target.value.len, (const char*)opt_target.value.ptr);
fprintf(stderr, "parsed from line:\n%.*s\n", (int)opt_line.value.len, opt_line.value.ptr);
fprintf(stderr, "Zig installation appears to be corrupted.\n");
exit(1);
}
glibc_abi->version_table.put(target, ver_list_base);
}
continue;
}
for (size_t fn_i = 0; fn_i < glibc_abi->all_functions.length; fn_i += 1) {
ZigGLibCVerList *ver_list = &ver_list_base[fn_i];
line_num += 1;
Optional<Slice<uint8_t>> opt_line = SplitIterator_next_separate(&it);
assert(opt_line.is_some);
SplitIterator line_it = memSplit(opt_line.value, str(" "));
for (;;) {
Optional<Slice<uint8_t>> opt_ver = SplitIterator_next(&line_it);
if (!opt_ver.is_some) break;
assert(ver_list->len < 8); // increase the array len in the type
unsigned long ver_index = strtoul(buf_ptr(buf_create_from_slice(opt_ver.value)), nullptr, 10);
assert(ver_index < 255); // use a bigger integer in the type
ver_list->versions[ver_list->len] = ver_index;
ver_list->len += 1;
}
}
ver_list_base = nullptr;
}
}
*out_result = glibc_abi;
return ErrorNone;
}
Error glibc_build_dummies_and_maps(CodeGen *g, const ZigGLibCAbi *glibc_abi, const ZigTarget *target,
Buf **out_dir, bool verbose, Stage2ProgressNode *progress_node)
{
Error err;
Buf *cache_dir = get_global_cache_dir();
CacheHash *cache_hash = heap::c_allocator.create<CacheHash>();
Buf *manifest_dir = buf_sprintf("%s" OS_SEP CACHE_HASH_SUBDIR, buf_ptr(cache_dir));
cache_init(cache_hash, manifest_dir);
Buf *compiler_id;
if ((err = get_compiler_id(&compiler_id))) {
if (verbose) {
fprintf(stderr, "unable to get compiler id: %s\n", err_str(err));
}
return err;
}
cache_buf(cache_hash, compiler_id);
cache_int(cache_hash, target->arch);
cache_int(cache_hash, target->abi);
cache_int(cache_hash, target->glibc_or_darwin_version->major);
cache_int(cache_hash, target->glibc_or_darwin_version->minor);
cache_int(cache_hash, target->glibc_or_darwin_version->patch);
Buf digest = BUF_INIT;
buf_resize(&digest, 0);
if ((err = cache_hit(cache_hash, &digest))) {
// Treat an invalid format error as a cache miss.
if (err != ErrorInvalidFormat)
return err;
}
// We should always get a cache hit because there are no
// files in the input hash.
assert(buf_len(&digest) != 0);
Buf *dummy_dir = buf_alloc();
os_path_join(manifest_dir, &digest, dummy_dir);
if ((err = os_make_path(dummy_dir)))
return err;
Buf *test_if_exists_path = buf_alloc();
os_path_join(dummy_dir, buf_create_from_str("ok"), test_if_exists_path);
bool hit;
if ((err = os_file_exists(test_if_exists_path, &hit)))
return err;
if (hit) {
*out_dir = dummy_dir;
return ErrorNone;
}
ZigGLibCVerList *ver_list_base = glibc_abi->version_table.get(target);
uint8_t target_ver_index = 0;
for (;target_ver_index < glibc_abi->all_versions.length; target_ver_index += 1) {
const Stage2SemVer *this_ver = &glibc_abi->all_versions.at(target_ver_index);
if (this_ver->major == target->glibc_or_darwin_version->major &&
this_ver->minor == target->glibc_or_darwin_version->minor &&
this_ver->patch == target->glibc_or_darwin_version->patch)
{
break;
}
}
if (target_ver_index == glibc_abi->all_versions.length) {
if (verbose) {
fprintf(stderr, "Unrecognized glibc version: %d.%d.%d\n",
target->glibc_or_darwin_version->major,
target->glibc_or_darwin_version->minor,
target->glibc_or_darwin_version->patch);
}
return ErrorUnknownABI;
}
Buf *map_file_path = buf_sprintf("%s" OS_SEP "all.map", buf_ptr(dummy_dir));
Buf *map_contents = buf_alloc();
for (uint8_t ver_i = 0; ver_i < glibc_abi->all_versions.length; ver_i += 1) {
const Stage2SemVer *ver = &glibc_abi->all_versions.at(ver_i);
if (ver->patch == 0) {
buf_appendf(map_contents, "GLIBC_%d.%d { };\n", ver->major, ver->minor);
} else {
buf_appendf(map_contents, "GLIBC_%d.%d.%d { };\n", ver->major, ver->minor, ver->patch);
}
}
if ((err = os_write_file(map_file_path, map_contents))) {
if (verbose) {
fprintf(stderr, "unable to write %s: %s", buf_ptr(map_file_path), err_str(err));
}
return err;
}
for (size_t lib_i = 0; lib_i < array_length(glibc_libs); lib_i += 1) {
const ZigGLibCLib *lib = &glibc_libs[lib_i];
Buf *zig_file_path = buf_sprintf("%s" OS_SEP "%s.zig", buf_ptr(dummy_dir), lib->name);
Buf *zig_body = buf_alloc();
Buf *zig_footer = buf_alloc();
buf_appendf(zig_body, "comptime {\n");
buf_appendf(zig_body, " asm (\n");
for (size_t fn_i = 0; fn_i < glibc_abi->all_functions.length; fn_i += 1) {
const ZigGLibCFn *libc_fn = &glibc_abi->all_functions.at(fn_i);
if (libc_fn->lib != lib) continue;
ZigGLibCVerList *ver_list = &ver_list_base[fn_i];
// Pick the default symbol version:
// - If there are no versions, don't emit it
// - Take the greatest one <= than the target one
// - If none of them is <= than the
// specified one don't pick any default version
if (ver_list->len == 0) continue;
uint8_t chosen_def_ver_index = 255;
for (uint8_t ver_i = 0; ver_i < ver_list->len; ver_i += 1) {
uint8_t ver_index = ver_list->versions[ver_i];
if ((chosen_def_ver_index == 255 || ver_index > chosen_def_ver_index) &&
target_ver_index >= ver_index)
{
chosen_def_ver_index = ver_index;
}
}
for (uint8_t ver_i = 0; ver_i < ver_list->len; ver_i += 1) {
uint8_t ver_index = ver_list->versions[ver_i];
Buf *stub_name;
const Stage2SemVer *ver = &glibc_abi->all_versions.at(ver_index);
const char *sym_name = buf_ptr(libc_fn->name);
if (ver->patch == 0) {
stub_name = buf_sprintf("%s_%d_%d", sym_name, ver->major, ver->minor);
} else {
stub_name = buf_sprintf("%s_%d_%d_%d", sym_name, ver->major, ver->minor, ver->patch);
}
buf_appendf(zig_footer, "export fn %s() void {}\n", buf_ptr(stub_name));
// Default symbol version definition vs normal symbol version definition
const char *at_sign_str = (chosen_def_ver_index != 255 &&
ver_index == chosen_def_ver_index) ? "@@" : "@";
if (ver->patch == 0) {
buf_appendf(zig_body, " \\\\ .symver %s, %s%sGLIBC_%d.%d\n",
buf_ptr(stub_name), sym_name, at_sign_str, ver->major, ver->minor);
} else {
buf_appendf(zig_body, " \\\\ .symver %s, %s%sGLIBC_%d.%d.%d\n",
buf_ptr(stub_name), sym_name, at_sign_str, ver->major, ver->minor, ver->patch);
}
// Hide the stub to keep the symbol table clean
buf_appendf(zig_body, " \\\\ .hidden %s\n", buf_ptr(stub_name));
}
}
buf_appendf(zig_body, " );\n");
buf_appendf(zig_body, "}\n");
buf_append_buf(zig_body, zig_footer);
if ((err = os_write_file(zig_file_path, zig_body))) {
if (verbose) {
fprintf(stderr, "unable to write %s: %s", buf_ptr(zig_file_path), err_str(err));
}
return err;
}
bool is_ld = (strcmp(lib->name, "ld") == 0);
CodeGen *child_gen = create_child_codegen(g, zig_file_path, OutTypeLib, nullptr, lib->name, progress_node);
codegen_set_lib_version(child_gen, true, lib->sover, 0, 0);
child_gen->is_dynamic = true;
child_gen->is_dummy_so = true;
child_gen->version_script_path = map_file_path;
child_gen->enable_cache = false;
child_gen->output_dir = dummy_dir;
if (is_ld) {
assert(g->zig_target->standard_dynamic_linker_path != nullptr);
Buf *ld_basename = buf_alloc();
os_path_split(buf_create_from_str(g->zig_target->standard_dynamic_linker_path),
nullptr, ld_basename);
child_gen->override_soname = ld_basename;
}
codegen_build_and_link(child_gen);
}
if ((err = os_write_file(test_if_exists_path, buf_alloc()))) {
if (verbose) {
fprintf(stderr, "unable to write %s: %s", buf_ptr(test_if_exists_path), err_str(err));
}
return err;
}
*out_dir = dummy_dir;
return ErrorNone;
}
uint32_t hash_glibc_target(const ZigTarget *x) {
return x->arch * (uint32_t)3250106448 +
x->os * (uint32_t)542534372 +
x->abi * (uint32_t)59162639;
}
bool eql_glibc_target(const ZigTarget *a, const ZigTarget *b) {
return a->arch == b->arch &&
a->os == b->os &&
a->abi == b->abi;
}
size_t glibc_lib_count(void) {
return array_length(glibc_libs);
}
const ZigGLibCLib *glibc_lib_enum(size_t index) {
assert(index < array_length(glibc_libs));
return &glibc_libs[index];
}
const ZigGLibCLib *glibc_lib_find(const char *name) {
for (size_t i = 0; i < array_length(glibc_libs); i += 1) {
if (strcmp(glibc_libs[i].name, name) == 0) {
return &glibc_libs[i];
}
}
return nullptr;
}

50
src/glibc.hpp
View File

@ -1,50 +0,0 @@
/*
* Copyright (c) 2019 Andrew Kelley
*
* This file is part of zig, which is MIT licensed.
* See http://opensource.org/licenses/MIT
*/
#ifndef ZIG_GLIBC_HPP
#define ZIG_GLIBC_HPP
#include "all_types.hpp"
struct ZigGLibCLib {
const char *name;
uint8_t sover;
};
struct ZigGLibCFn {
Buf *name;
const ZigGLibCLib *lib;
};
struct ZigGLibCVerList {
uint8_t versions[8]; // 8 is just the max number, we know statically it's big enough
uint8_t len;
};
uint32_t hash_glibc_target(const ZigTarget *x);
bool eql_glibc_target(const ZigTarget *a, const ZigTarget *b);
struct ZigGLibCAbi {
Buf *abi_txt_path;
Buf *vers_txt_path;
Buf *fns_txt_path;
ZigList<Stage2SemVer> all_versions;
ZigList<ZigGLibCFn> all_functions;
// The value is a pointer to all_functions.length items and each item is an index
// into all_functions.
HashMap<const ZigTarget *, ZigGLibCVerList *, hash_glibc_target, eql_glibc_target> version_table;
};
Error glibc_load_metadata(ZigGLibCAbi **out_result, Buf *zig_lib_dir, bool verbose);
Error glibc_build_dummies_and_maps(CodeGen *codegen, const ZigGLibCAbi *glibc_abi, const ZigTarget *target,
Buf **out_dir, bool verbose, Stage2ProgressNode *progress_node);
size_t glibc_lib_count(void);
const ZigGLibCLib *glibc_lib_enum(size_t index);
const ZigGLibCLib *glibc_lib_find(const char *name);
#endif

66
src/heap.cpp
View File

@ -10,7 +10,6 @@
#include "config.h"
#include "heap.hpp"
#include "mem_profile.hpp"
namespace heap {
@ -48,21 +47,9 @@ void BootstrapAllocator::internal_deallocate(const mem::TypeInfo &info, void *pt
mem::os::free(ptr);
}
void CAllocator::init(const char *name) {
#ifdef ZIG_ENABLE_MEM_PROFILE
this->profile = bootstrap_allocator.create<mem::Profile>();
this->profile->init(name, "CAllocator");
#endif
}
void CAllocator::init(const char *name) { }
void CAllocator::deinit() {
#ifdef ZIG_ENABLE_MEM_PROFILE
assert(this->profile);
this->profile->deinit();
bootstrap_allocator.destroy(this->profile);
this->profile = nullptr;
#endif
}
void CAllocator::deinit() { }
CAllocator *CAllocator::construct(mem::Allocator *allocator, const char *name) {
auto p = new(allocator->create<CAllocator>()) CAllocator();
@ -75,23 +62,11 @@ void CAllocator::destruct(mem::Allocator *allocator) {
allocator->destroy(this);
}
#ifdef ZIG_ENABLE_MEM_PROFILE
void CAllocator::print_report(FILE *file) {
this->profile->print_report(file);
}
#endif
void *CAllocator::internal_allocate(const mem::TypeInfo &info, size_t count) {
#ifdef ZIG_ENABLE_MEM_PROFILE
this->profile->record_alloc(info, count);
#endif
return mem::os::calloc(count, info.size);
}
void *CAllocator::internal_allocate_nonzero(const mem::TypeInfo &info, size_t count) {
#ifdef ZIG_ENABLE_MEM_PROFILE
this->profile->record_alloc(info, count);
#endif
return mem::os::malloc(count * info.size);
}
@ -103,17 +78,10 @@ void *CAllocator::internal_reallocate(const mem::TypeInfo &info, void *old_ptr,
}
void *CAllocator::internal_reallocate_nonzero(const mem::TypeInfo &info, void *old_ptr, size_t old_count, size_t new_count) {
#ifdef ZIG_ENABLE_MEM_PROFILE
this->profile->record_dealloc(info, old_count);
this->profile->record_alloc(info, new_count);
#endif
return mem::os::realloc(old_ptr, new_count * info.size);
}
void CAllocator::internal_deallocate(const mem::TypeInfo &info, void *ptr, size_t count) {
#ifdef ZIG_ENABLE_MEM_PROFILE
this->profile->record_dealloc(info, count);
#endif
mem::os::free(ptr);
}
@ -249,10 +217,6 @@ void ArenaAllocator::Impl::track_object(Object object) {
}
void ArenaAllocator::init(Allocator *backing, const char *name) {
#ifdef ZIG_ENABLE_MEM_PROFILE
this->profile = bootstrap_allocator.create<mem::Profile>();
this->profile->init(name, "ArenaAllocator");
#endif
this->impl = bootstrap_allocator.create<Impl>();
{
auto &r = *this->impl;
@ -309,13 +273,6 @@ void ArenaAllocator::deinit() {
t = prev;
}
}
#ifdef ZIG_ENABLE_MEM_PROFILE
assert(this->profile);
this->profile->deinit();
bootstrap_allocator.destroy(this->profile);
this->profile = nullptr;
#endif
}
ArenaAllocator *ArenaAllocator::construct(mem::Allocator *allocator, mem::Allocator *backing, const char *name) {
@ -329,23 +286,11 @@ void ArenaAllocator::destruct(mem::Allocator *allocator) {
allocator->destroy(this);
}
#ifdef ZIG_ENABLE_MEM_PROFILE
void ArenaAllocator::print_report(FILE *file) {
this->profile->print_report(file);
}
#endif
void *ArenaAllocator::internal_allocate(const mem::TypeInfo &info, size_t count) {
#ifdef ZIG_ENABLE_MEM_PROFILE
this->profile->record_alloc(info, count);
#endif
return this->impl->allocate(info, count);
}
void *ArenaAllocator::internal_allocate_nonzero(const mem::TypeInfo &info, size_t count) {
#ifdef ZIG_ENABLE_MEM_PROFILE
this->profile->record_alloc(info, count);
#endif
return this->impl->allocate(info, count);
}
@ -354,17 +299,10 @@ void *ArenaAllocator::internal_reallocate(const mem::TypeInfo &info, void *old_p
}
void *ArenaAllocator::internal_reallocate_nonzero(const mem::TypeInfo &info, void *old_ptr, size_t old_count, size_t new_count) {
#ifdef ZIG_ENABLE_MEM_PROFILE
this->profile->record_dealloc(info, old_count);
this->profile->record_alloc(info, new_count);
#endif
return this->impl->reallocate(info, old_ptr, old_count, new_count);
}
void ArenaAllocator::internal_deallocate(const mem::TypeInfo &info, void *ptr, size_t count) {
#ifdef ZIG_ENABLE_MEM_PROFILE
this->profile->record_dealloc(info, count);
#endif
// noop
}

19
src/heap.hpp
View File

@ -12,12 +12,6 @@
#include "util_base.hpp"
#include "mem.hpp"
#ifdef ZIG_ENABLE_MEM_PROFILE
namespace mem {
struct Profile;
}
#endif
namespace heap {
struct BootstrapAllocator final : mem::Allocator {
@ -40,9 +34,6 @@ struct CAllocator final : mem::Allocator {
static CAllocator *construct(mem::Allocator *allocator, const char *name);
void destruct(mem::Allocator *allocator) final;
#ifdef ZIG_ENABLE_MEM_PROFILE
void print_report(FILE *file = nullptr);
#endif
private:
ATTRIBUTE_RETURNS_NOALIAS void *internal_allocate(const mem::TypeInfo &info, size_t count) final;
@ -51,9 +42,6 @@ private:
void *internal_reallocate_nonzero(const mem::TypeInfo &info, void *old_ptr, size_t old_count, size_t new_count) final;
void internal_deallocate(const mem::TypeInfo &info, void *ptr, size_t count) final;
#ifdef ZIG_ENABLE_MEM_PROFILE
mem::Profile *profile;
#endif
};
//
@ -71,9 +59,6 @@ struct ArenaAllocator final : mem::Allocator {
static ArenaAllocator *construct(mem::Allocator *allocator, mem::Allocator *backing, const char *name);
void destruct(mem::Allocator *allocator) final;
#ifdef ZIG_ENABLE_MEM_PROFILE
void print_report(FILE *file = nullptr);
#endif
private:
ATTRIBUTE_RETURNS_NOALIAS void *internal_allocate(const mem::TypeInfo &info, size_t count) final;
@ -82,10 +67,6 @@ private:
void *internal_reallocate_nonzero(const mem::TypeInfo &info, void *old_ptr, size_t old_count, size_t new_count) final;
void internal_deallocate(const mem::TypeInfo &info, void *ptr, size_t count) final;
#ifdef ZIG_ENABLE_MEM_PROFILE
mem::Profile *profile;
#endif
struct Impl;
Impl *impl;
};

1907
src/install_files.h
View File

File diff suppressed because it is too large Load Diff

181
src/ir.cpp
View File

@ -22570,38 +22570,12 @@ static IrInstGen *ir_analyze_container_field_ptr(IrAnalyze *ira, Buf *field_name
}
static void add_link_lib_symbol(IrAnalyze *ira, Buf *lib_name, Buf *symbol_name, AstNode *source_node) {
bool is_libc = target_is_libc_lib_name(ira->codegen->zig_target, buf_ptr(lib_name));
if (is_libc && ira->codegen->libc_link_lib == nullptr && !ira->codegen->reported_bad_link_libc_error) {
ir_add_error_node(ira, source_node,
buf_sprintf("dependency on library c must be explicitly specified in the build command"));
const char *msg = stage2_add_link_lib(&ira->codegen->stage1, buf_ptr(lib_name), buf_len(lib_name),
buf_ptr(symbol_name), buf_len(symbol_name));
if (msg != nullptr) {
ir_add_error_node(ira, source_node, buf_create_from_str(msg));
ira->codegen->reported_bad_link_libc_error = true;
}
LinkLib *link_lib = add_link_lib(ira->codegen, lib_name);
for (size_t i = 0; i < link_lib->symbols.length; i += 1) {
Buf *existing_symbol_name = link_lib->symbols.at(i);
if (buf_eql_buf(existing_symbol_name, symbol_name)) {
return;
}
}
if (!is_libc && !target_is_wasm(ira->codegen->zig_target) && !ira->codegen->have_pic && !ira->codegen->reported_bad_link_libc_error) {
ErrorMsg *msg = ir_add_error_node(ira, source_node,
buf_sprintf("dependency on dynamic library '%s' requires enabling Position Independent Code",
buf_ptr(lib_name)));
add_error_note(ira->codegen, msg, source_node,
buf_sprintf("fixed by `--library %s` or `-fPIC`", buf_ptr(lib_name)));
ira->codegen->reported_bad_link_libc_error = true;
}
for (size_t i = 0; i < ira->codegen->forbidden_libs.length; i += 1) {
Buf *forbidden_lib_name = ira->codegen->forbidden_libs.at(i);
if (buf_eql_buf(lib_name, forbidden_lib_name)) {
ir_add_error_node(ira, source_node,
buf_sprintf("linking against forbidden library '%s'", buf_ptr(symbol_name)));
}
}
link_lib->symbols.append(symbol_name);
}
static IrInstGen *ir_error_dependency_loop(IrAnalyze *ira, IrInst* source_instr) {
@ -26355,13 +26329,6 @@ static IrInstGen *ir_analyze_instruction_type_name(IrAnalyze *ira, IrInstSrcType
return result;
}
static void ir_cimport_cache_paths(Buf *cache_dir, Buf *tmp_c_file_digest, Buf *out_zig_dir, Buf *out_zig_path) {
buf_resize(out_zig_dir, 0);
buf_resize(out_zig_path, 0);
buf_appendf(out_zig_dir, "%s" OS_SEP "o" OS_SEP "%s",
buf_ptr(cache_dir), buf_ptr(tmp_c_file_digest));
buf_appendf(out_zig_path, "%s" OS_SEP "cimport.zig", buf_ptr(out_zig_dir));
}
static IrInstGen *ir_analyze_instruction_c_import(IrAnalyze *ira, IrInstSrcCImport *instruction) {
Error err;
AstNode *node = instruction->base.base.source_node;
@ -26393,145 +26360,7 @@ static IrInstGen *ir_analyze_instruction_c_import(IrAnalyze *ira, IrInstSrcCImpo
cimport_pkg->package_table.put(buf_create_from_str("std"), ira->codegen->std_package);
buf_init_from_buf(&cimport_pkg->pkg_path, namespace_name);
CacheHash *cache_hash;
if ((err = create_c_object_cache(ira->codegen, &cache_hash, false))) {
ir_add_error_node(ira, node, buf_sprintf("C import failed: unable to create cache: %s", err_str(err)));
return ira->codegen->invalid_inst_gen;
}
cache_buf(cache_hash, &cimport_scope->buf);
// Set this because we're not adding any files before checking for a hit.
cache_hash->force_check_manifest = true;
Buf tmp_c_file_digest = BUF_INIT;
buf_resize(&tmp_c_file_digest, 0);
if ((err = cache_hit(cache_hash, &tmp_c_file_digest))) {
if (err != ErrorInvalidFormat) {
ir_add_error_node(ira, node, buf_sprintf("C import failed: unable to check cache: %s", err_str(err)));
return ira->codegen->invalid_inst_gen;
}
}
ira->codegen->caches_to_release.append(cache_hash);
Buf *out_zig_dir = buf_alloc();
Buf *out_zig_path = buf_alloc();
if (buf_len(&tmp_c_file_digest) == 0 || cache_hash->files.length == 0) {
// Cache Miss
Buf *tmp_c_file_dir = buf_sprintf("%s" OS_SEP "o" OS_SEP "%s",
buf_ptr(ira->codegen->cache_dir), buf_ptr(&cache_hash->b64_digest));
Buf *resolve_paths[] = {
tmp_c_file_dir,
buf_create_from_str("cimport.h"),
};
Buf tmp_c_file_path = os_path_resolve(resolve_paths, 2);
if ((err = os_make_path(tmp_c_file_dir))) {
ir_add_error_node(ira, node, buf_sprintf("C import failed: unable to make dir: %s", err_str(err)));
return ira->codegen->invalid_inst_gen;
}
if ((err = os_write_file(&tmp_c_file_path, &cimport_scope->buf))) {
ir_add_error_node(ira, node, buf_sprintf("C import failed: unable to write .h file: %s", err_str(err)));
return ira->codegen->invalid_inst_gen;
}
if (ira->codegen->verbose_cimport) {
fprintf(stderr, "@cImport source: %s\n", buf_ptr(&tmp_c_file_path));
}
Buf *tmp_dep_file = buf_sprintf("%s.d", buf_ptr(&tmp_c_file_path));
ZigList<const char *> clang_argv = {0};
add_cc_args(ira->codegen, clang_argv, buf_ptr(tmp_dep_file), true, FileExtC);
clang_argv.append(buf_ptr(&tmp_c_file_path));
if (ira->codegen->verbose_cc) {
fprintf(stderr, "clang");
for (size_t i = 0; i < clang_argv.length; i += 1) {
fprintf(stderr, " %s", clang_argv.at(i));
}
fprintf(stderr, "\n");
}
clang_argv.append(nullptr); // to make the [start...end] argument work
Stage2ErrorMsg *errors_ptr;
size_t errors_len;
Stage2Ast *ast;
const char *resources_path = buf_ptr(ira->codegen->zig_c_headers_dir);
if ((err = stage2_translate_c(&ast, &errors_ptr, &errors_len,
&clang_argv.at(0), &clang_argv.last(), resources_path)))
{
if (err != ErrorCCompileErrors) {
ir_add_error_node(ira, node, buf_sprintf("C import failed: %s", err_str(err)));
return ira->codegen->invalid_inst_gen;
}
ErrorMsg *parent_err_msg = ir_add_error_node(ira, node, buf_sprintf("C import failed"));
if (ira->codegen->libc_link_lib == nullptr) {
add_error_note(ira->codegen, parent_err_msg, node,
buf_sprintf("libc headers not available; compilation does not link against libc"));
}
for (size_t i = 0; i < errors_len; i += 1) {
Stage2ErrorMsg *clang_err = &errors_ptr[i];
// Clang can emit "too many errors, stopping now", in which case `source` and `filename_ptr` are null
if (clang_err->source && clang_err->filename_ptr) {
ErrorMsg *err_msg = err_msg_create_with_offset(
clang_err->filename_ptr ?
buf_create_from_mem(clang_err->filename_ptr, clang_err->filename_len) : buf_alloc(),
clang_err->line, clang_err->column, clang_err->offset, clang_err->source,
buf_create_from_mem(clang_err->msg_ptr, clang_err->msg_len));
err_msg_add_note(parent_err_msg, err_msg);
}
}
return ira->codegen->invalid_inst_gen;
}
if (ira->codegen->verbose_cimport) {
fprintf(stderr, "@cImport .d file: %s\n", buf_ptr(tmp_dep_file));
}
if ((err = cache_add_dep_file(cache_hash, tmp_dep_file, false))) {
ir_add_error_node(ira, node, buf_sprintf("C import failed: unable to parse .d file: %s", err_str(err)));
return ira->codegen->invalid_inst_gen;
}
if ((err = cache_final(cache_hash, &tmp_c_file_digest))) {
ir_add_error_node(ira, node, buf_sprintf("C import failed: unable to finalize cache: %s", err_str(err)));
return ira->codegen->invalid_inst_gen;
}
ir_cimport_cache_paths(ira->codegen->cache_dir, &tmp_c_file_digest, out_zig_dir, out_zig_path);
if ((err = os_make_path(out_zig_dir))) {
ir_add_error_node(ira, node, buf_sprintf("C import failed: unable to make output dir: %s", err_str(err)));
return ira->codegen->invalid_inst_gen;
}
FILE *out_file = fopen(buf_ptr(out_zig_path), "wb");
if (out_file == nullptr) {
ir_add_error_node(ira, node,
buf_sprintf("C import failed: unable to open output file: %s", strerror(errno)));
return ira->codegen->invalid_inst_gen;
}
stage2_render_ast(ast, out_file);
if (fclose(out_file) != 0) {
ir_add_error_node(ira, node,
buf_sprintf("C import failed: unable to write to output file: %s", strerror(errno)));
return ira->codegen->invalid_inst_gen;
}
if (ira->codegen->verbose_cimport) {
fprintf(stderr, "@cImport output: %s\n", buf_ptr(out_zig_path));
}
} else {
// Cache Hit
ir_cimport_cache_paths(ira->codegen->cache_dir, &tmp_c_file_digest, out_zig_dir, out_zig_path);
if (ira->codegen->verbose_cimport) {
fprintf(stderr, "@cImport cache hit: %s\n", buf_ptr(out_zig_path));
}
}
Buf *out_zig_path = buf_create_from_str(stage2_cimport(&ira->codegen->stage1));
Buf *import_code = buf_alloc();
if ((err = file_fetch(ira->codegen, out_zig_path, import_code))) {

2985
src/link.cpp
View File

File diff suppressed because it is too large Load Diff

1474
src/main.cpp
View File

File diff suppressed because it is too large Load Diff

13
src/mem.cpp
View File

@ -7,7 +7,6 @@
#include "config.h"
#include "mem.hpp"
#include "mem_profile.hpp"
#include "heap.hpp"
namespace mem {
@ -22,16 +21,4 @@ void deinit() {
heap::bootstrap_allocator_state.deinit();
}
#ifdef ZIG_ENABLE_MEM_PROFILE
void print_report(FILE *file) {
heap::c_allocator_state.print_report(file);
intern_counters.print_report(file);
}
#endif
#ifdef ZIG_ENABLE_MEM_PROFILE
bool report_print = false;
FILE *report_file{nullptr};
#endif
} // namespace mem

9
src/mem.hpp
View File

@ -135,15 +135,6 @@ protected:
virtual void internal_deallocate(const TypeInfo &info, void *ptr, size_t count) = 0;
};
#ifdef ZIG_ENABLE_MEM_PROFILE
void print_report(FILE *file = nullptr);
// global memory report flag
extern bool report_print;
// global memory report default destination
extern FILE *report_file;
#endif
} // namespace mem
#endif

181
src/mem_profile.cpp
View File

@ -1,181 +0,0 @@
/*
* Copyright (c) 2020 Andrew Kelley
*
* This file is part of zig, which is MIT licensed.
* See http://opensource.org/licenses/MIT
*/
#include "config.h"
#ifdef ZIG_ENABLE_MEM_PROFILE
#include "mem.hpp"
#include "mem_list.hpp"
#include "mem_profile.hpp"
#include "heap.hpp"
namespace mem {
void Profile::init(const char *name, const char *kind) {
this->name = name;
this->kind = kind;
this->usage_table.init(heap::bootstrap_allocator, 1024);
}
void Profile::deinit() {
assert(this->name != nullptr);
if (mem::report_print)
this->print_report();
this->usage_table.deinit(heap::bootstrap_allocator);
this->name = nullptr;
}
void Profile::record_alloc(const TypeInfo &info, size_t count) {
if (count == 0) return;
auto existing_entry = this->usage_table.put_unique(
heap::bootstrap_allocator,
UsageKey{info.name_ptr, info.name_len},
Entry{info, 1, count, 0, 0} );
if (existing_entry != nullptr) {
assert(existing_entry->value.info.size == info.size); // allocated name does not match type
existing_entry->value.alloc.calls += 1;
existing_entry->value.alloc.objects += count;
}
}
void Profile::record_dealloc(const TypeInfo &info, size_t count) {
if (count == 0) return;
auto existing_entry = this->usage_table.maybe_get(UsageKey{info.name_ptr, info.name_len});
if (existing_entry == nullptr) {
fprintf(stderr, "deallocated name '");
for (size_t i = 0; i < info.name_len; ++i)
fputc(info.name_ptr[i], stderr);
zig_panic("' (size %zu) not found in allocated table; compromised memory usage stats", info.size);
}
if (existing_entry->value.info.size != info.size) {
fprintf(stderr, "deallocated name '");
for (size_t i = 0; i < info.name_len; ++i)
fputc(info.name_ptr[i], stderr);
zig_panic("' does not match expected type size %zu", info.size);
}
assert(existing_entry->value.alloc.calls - existing_entry->value.dealloc.calls > 0);
assert(existing_entry->value.alloc.objects - existing_entry->value.dealloc.objects >= count);
existing_entry->value.dealloc.calls += 1;
existing_entry->value.dealloc.objects += count;
}
static size_t entry_remain_total_bytes(const Profile::Entry *entry) {
return (entry->alloc.objects - entry->dealloc.objects) * entry->info.size;
}
static int entry_compare(const void *a, const void *b) {
size_t total_a = entry_remain_total_bytes(*reinterpret_cast<Profile::Entry *const *>(a));
size_t total_b = entry_remain_total_bytes(*reinterpret_cast<Profile::Entry *const *>(b));
if (total_a > total_b)
return -1;
if (total_a < total_b)
return 1;
return 0;
};
void Profile::print_report(FILE *file) {
if (!file) {
file = report_file;
if (!file)
file = stderr;
}
fprintf(file, "\n--- MEMORY PROFILE REPORT [%s]: %s ---\n", this->kind, this->name);
List<const Entry *> list;
auto it = this->usage_table.entry_iterator();
for (;;) {
auto entry = it.next();
if (!entry)
break;
list.append(&heap::bootstrap_allocator, &entry->value);
}
qsort(list.items, list.length, sizeof(const Entry *), entry_compare);
size_t total_bytes_alloc = 0;
size_t total_bytes_dealloc = 0;
size_t total_calls_alloc = 0;
size_t total_calls_dealloc = 0;
for (size_t i = 0; i < list.length; i += 1) {
const Entry *entry = list.at(i);
fprintf(file, " ");
for (size_t j = 0; j < entry->info.name_len; ++j)
fputc(entry->info.name_ptr[j], file);
fprintf(file, ": %zu bytes each", entry->info.size);
fprintf(file, ", alloc{ %zu calls, %zu objects, total ", entry->alloc.calls, entry->alloc.objects);
const auto alloc_num_bytes = entry->alloc.objects * entry->info.size;
zig_pretty_print_bytes(file, alloc_num_bytes);
fprintf(file, " }, dealloc{ %zu calls, %zu objects, total ", entry->dealloc.calls, entry->dealloc.objects);
const auto dealloc_num_bytes = entry->dealloc.objects * entry->info.size;
zig_pretty_print_bytes(file, dealloc_num_bytes);
fprintf(file, " }, remain{ %zu calls, %zu objects, total ",
entry->alloc.calls - entry->dealloc.calls,
entry->alloc.objects - entry->dealloc.objects );
const auto remain_num_bytes = alloc_num_bytes - dealloc_num_bytes;
zig_pretty_print_bytes(file, remain_num_bytes);
fprintf(file, " }\n");
total_bytes_alloc += alloc_num_bytes;
total_bytes_dealloc += dealloc_num_bytes;
total_calls_alloc += entry->alloc.calls;
total_calls_dealloc += entry->dealloc.calls;
}
fprintf(file, "\n Total bytes allocated: ");
zig_pretty_print_bytes(file, total_bytes_alloc);
fprintf(file, ", deallocated: ");
zig_pretty_print_bytes(file, total_bytes_dealloc);
fprintf(file, ", remaining: ");
zig_pretty_print_bytes(file, total_bytes_alloc - total_bytes_dealloc);
fprintf(file, "\n Total calls alloc: %zu, dealloc: %zu, remain: %zu\n",
total_calls_alloc, total_calls_dealloc, (total_calls_alloc - total_calls_dealloc));
list.deinit(&heap::bootstrap_allocator);
}
uint32_t Profile::usage_hash(UsageKey key) {
// FNV 32-bit hash
uint32_t h = 2166136261;
for (size_t i = 0; i < key.name_len; ++i) {
h = h ^ key.name_ptr[i];
h = h * 16777619;
}
return h;
}
bool Profile::usage_equal(UsageKey a, UsageKey b) {
return memcmp(a.name_ptr, b.name_ptr, a.name_len > b.name_len ? a.name_len : b.name_len) == 0;
}
void InternCounters::print_report(FILE *file) {
if (!file) {
file = report_file;
if (!file)
file = stderr;
}
fprintf(file, "\n--- IR INTERNING REPORT ---\n");
fprintf(file, " undefined: interned %zu times\n", intern_counters.x_undefined);
fprintf(file, " void: interned %zu times\n", intern_counters.x_void);
fprintf(file, " null: interned %zu times\n", intern_counters.x_null);
fprintf(file, " unreachable: interned %zu times\n", intern_counters.x_unreachable);
fprintf(file, " zero_byte: interned %zu times\n", intern_counters.zero_byte);
}
InternCounters intern_counters;
} // namespace mem
#endif

71
src/mem_profile.hpp
View File

@ -1,71 +0,0 @@
/*
* Copyright (c) 2020 Andrew Kelley
*
* This file is part of zig, which is MIT licensed.
* See http://opensource.org/licenses/MIT
*/
#ifndef ZIG_MEM_PROFILE_HPP
#define ZIG_MEM_PROFILE_HPP
#include "config.h"
#ifdef ZIG_ENABLE_MEM_PROFILE
#include <stdio.h>
#include "mem.hpp"
#include "mem_hash_map.hpp"
#include "util.hpp"
namespace mem {
struct Profile {
void init(const char *name, const char *kind);
void deinit();
void record_alloc(const TypeInfo &info, size_t count);
void record_dealloc(const TypeInfo &info, size_t count);
void print_report(FILE *file = nullptr);
struct Entry {
TypeInfo info;
struct Use {
size_t calls;
size_t objects;
} alloc, dealloc;
};
private:
const char *name;
const char *kind;
struct UsageKey {
const char *name_ptr;
size_t name_len;
};
static uint32_t usage_hash(UsageKey key);
static bool usage_equal(UsageKey a, UsageKey b);
HashMap<UsageKey, Entry, usage_hash, usage_equal> usage_table;
};
struct InternCounters {
size_t x_undefined;
size_t x_void;
size_t x_null;
size_t x_unreachable;
size_t zero_byte;
void print_report(FILE *file = nullptr);
};
extern InternCounters intern_counters;
} // namespace mem
#endif
#endif

109
src/mem_type_info.hpp
View File

@ -10,18 +10,8 @@
#include "config.h"
#ifndef ZIG_TYPE_INFO_IMPLEMENTATION
# ifdef ZIG_ENABLE_MEM_PROFILE
# define ZIG_TYPE_INFO_IMPLEMENTATION 1
# else
# define ZIG_TYPE_INFO_IMPLEMENTATION 0
# endif
#endif
namespace mem {
#if ZIG_TYPE_INFO_IMPLEMENTATION == 0
struct TypeInfo {
size_t size;
size_t alignment;
@ -32,105 +22,6 @@ struct TypeInfo {
}
};
#elif ZIG_TYPE_INFO_IMPLEMENTATION == 1
//
// A non-portable way to get a human-readable type-name compatible with
// non-RTTI C++ compiler mode; eg. `-fno-rtti`.
//
// Minimum requirements are c++11 and a compiler that has a constant for the
// current function's decorated name whereby a template-type name can be
// computed. eg. `__PRETTY_FUNCTION__` or `__FUNCSIG__`.
//
// given the following snippet:
//
// | #include <stdio.h>
// |
// | struct Top {};
// | namespace mynamespace {
// | using custom = unsigned int;
// | struct Foo {
// | struct Bar {};
// | };
// | };
// |
// | template <typename T>
// | void foobar() {
// | #ifdef _MSC_VER
// | fprintf(stderr, "--> %s\n", __FUNCSIG__);
// | #else
// | fprintf(stderr, "--> %s\n", __PRETTY_FUNCTION__);
// | #endif
// | }
// |
// | int main() {
// | foobar<Top>();
// | foobar<unsigned int>();
// | foobar<mynamespace::custom>();
// | foobar<mynamespace::Foo*>();
// | foobar<mynamespace::Foo::Bar*>();
// | }
//
// gcc 9.2.0 produces:
// --> void foobar() [with T = Top]
// --> void foobar() [with T = unsigned int]
// --> void foobar() [with T = unsigned int]
// --> void foobar() [with T = mynamespace::Foo*]
// --> void foobar() [with T = mynamespace::Foo::Bar*]
//
// xcode 11.3.1/clang produces:
// --> void foobar() [T = Top]
// --> void foobar() [T = unsigned int]
// --> void foobar() [T = unsigned int]
// --> void foobar() [T = mynamespace::Foo *]
// --> void foobar() [T = mynamespace::Foo::Bar *]
//
// VStudio 2019 16.5.0/msvc produces:
// --> void __cdecl foobar<struct Top>(void)
// --> void __cdecl foobar<unsigned int>(void)
// --> void __cdecl foobar<unsigned int>(void)
// --> void __cdecl foobar<structmynamespace::Foo*>(void)
// --> void __cdecl foobar<structmynamespace::Foo::Bar*>(void)
//
struct TypeInfo {
const char *name_ptr;
size_t name_len;
size_t size;
size_t alignment;
static constexpr TypeInfo to_type_info(const char *str, size_t start, size_t end, size_t size, size_t alignment) {
return TypeInfo{str + start, end - start, size, alignment};
}
static constexpr size_t index_of(const char *str, char c) {
return *str == c ? 0 : 1 + index_of(str + 1, c);
}
template <typename T>
static constexpr const char *decorated_name() {
#ifdef _MSC_VER
return __FUNCSIG__;
#else
return __PRETTY_FUNCTION__;
#endif
}
static constexpr TypeInfo extract(const char *decorated, size_t size, size_t alignment) {
#ifdef _MSC_VER
return to_type_info(decorated, index_of(decorated, '<') + 1, index_of(decorated, '>'), size, alignment);
#else
return to_type_info(decorated, index_of(decorated, '=') + 2, index_of(decorated, ']'), size, alignment);
#endif
}
template <typename T>
static constexpr TypeInfo make() {
return TypeInfo::extract(TypeInfo::decorated_name<T>(), sizeof(T), alignof(T));
}
};
#endif // ZIG_TYPE_INFO_IMPLEMENTATION
} // namespace mem
#endif

127
src/stage1.cpp Normal file
View File

@ -0,0 +1,127 @@
/*
* Copyright (c) 2020 Andrew Kelley
*
* This file is part of zig, which is MIT licensed.
* See http://opensource.org/licenses/MIT
*/
#include "stage1.h"
#include "os.hpp"
#include "all_types.hpp"
#include "codegen.hpp"
void zig_stage1_os_init(void) {
os_init();
mem::init();
init_all_targets();
}
struct ZigStage1 *zig_stage1_create(BuildMode optimize_mode,
const char *main_pkg_path_ptr, size_t main_pkg_path_len,
const char *root_src_path_ptr, size_t root_src_path_len,
const char *zig_lib_dir_ptr, size_t zig_lib_dir_len,
const ZigTarget *target, bool is_test_build, Stage2ProgressNode *progress_node)
{
Buf *main_pkg_path = buf_create_from_mem(main_pkg_path_ptr, main_pkg_path_len);
Buf *root_src_path = buf_create_from_mem(root_src_path_ptr, root_src_path_len);
Buf *zig_lib_dir = buf_create_from_mem(zig_lib_dir_ptr, zig_lib_dir_len);
CodeGen *g = codegen_create(main_pkg_path, root_src_path, target, optimize_mode,
zig_lib_dir, is_test_build, progress_node);
return &g->stage1;
}
void zig_stage1_destroy(struct ZigStage1 *stage1) {
CodeGen *codegen = reinterpret_cast<CodeGen *>(stage1);
codegen_destroy(codegen);
}
static void add_package(CodeGen *g, ZigStage1Pkg *stage1_pkg, ZigPackage *pkg) {
for (size_t i = 0; i < stage1_pkg->children_len; i += 1) {
ZigStage1Pkg *child_cli_pkg = stage1_pkg->children_ptr[i];
Buf *dirname = buf_alloc();
Buf *basename = buf_alloc();
os_path_split(buf_create_from_mem(child_cli_pkg->path_ptr, child_cli_pkg->path_len), dirname, basename);
ZigPackage *child_pkg = codegen_create_package(g, buf_ptr(dirname), buf_ptr(basename),
buf_ptr(buf_sprintf("%s.%.*s", buf_ptr(&pkg->pkg_path),
(int)child_cli_pkg->name_len, child_cli_pkg->name_ptr)));
auto entry = pkg->package_table.put_unique(
buf_create_from_mem(child_cli_pkg->name_ptr, child_cli_pkg->name_len),
child_pkg);
if (entry) {
ZigPackage *existing_pkg = entry->value;
Buf *full_path = buf_alloc();
os_path_join(&existing_pkg->root_src_dir, &existing_pkg->root_src_path, full_path);
fprintf(stderr, "Unable to add package '%.*s'->'%.*s': already exists as '%s'\n",
(int)child_cli_pkg->name_len, child_cli_pkg->name_ptr,
(int)child_cli_pkg->path_len, child_cli_pkg->path_ptr,
buf_ptr(full_path));
exit(EXIT_FAILURE);
}
add_package(g, child_cli_pkg, child_pkg);
}
}
void zig_stage1_build_object(struct ZigStage1 *stage1) {
CodeGen *g = reinterpret_cast<CodeGen *>(stage1);
g->root_out_name = buf_create_from_mem(stage1->root_name_ptr, stage1->root_name_len);
g->zig_lib_dir = buf_create_from_mem(stage1->zig_lib_dir_ptr, stage1->zig_lib_dir_len);
g->zig_std_dir = buf_create_from_mem(stage1->zig_std_dir_ptr, stage1->zig_std_dir_len);
g->output_dir = buf_create_from_mem(stage1->output_dir_ptr, stage1->output_dir_len);
if (stage1->builtin_zig_path_len != 0) {
g->builtin_zig_path = buf_create_from_mem(stage1->builtin_zig_path_ptr, stage1->builtin_zig_path_len);
}
if (stage1->test_filter_len != 0) {
g->test_filter = buf_create_from_mem(stage1->test_filter_ptr, stage1->test_filter_len);
}
if (stage1->test_name_prefix_len != 0) {
g->test_name_prefix = buf_create_from_mem(stage1->test_name_prefix_ptr, stage1->test_name_prefix_len);
}
g->link_mode_dynamic = stage1->link_mode_dynamic;
g->dll_export_fns = stage1->dll_export_fns;
g->have_pic = stage1->pic;
g->have_stack_probing = stage1->enable_stack_probing;
g->is_single_threaded = stage1->is_single_threaded;
g->valgrind_enabled = stage1->valgrind_enabled;
g->link_libc = stage1->link_libc;
g->link_libcpp = stage1->link_libcpp;
g->function_sections = stage1->function_sections;
g->subsystem = stage1->subsystem;
g->enable_time_report = stage1->enable_time_report;
g->enable_stack_report = stage1->enable_stack_report;
g->enable_dump_analysis = stage1->dump_analysis;
g->enable_doc_generation = stage1->enable_doc_generation;
g->emit_bin = stage1->emit_bin;
g->emit_asm = stage1->emit_asm;
g->emit_llvm_ir = stage1->emit_llvm_ir;
g->test_is_evented = stage1->test_is_evented;
g->verbose_tokenize = stage1->verbose_tokenize;
g->verbose_ast = stage1->verbose_ast;
g->verbose_link = stage1->verbose_link;
g->verbose_ir = stage1->verbose_ir;
g->verbose_llvm_ir = stage1->verbose_llvm_ir;
g->verbose_cimport = stage1->verbose_cimport;
g->verbose_cc = stage1->verbose_cc;
g->verbose_llvm_cpu_features = stage1->verbose_llvm_cpu_features;
g->err_color = stage1->err_color;
g->code_model = stage1->code_model;
{
g->strip_debug_symbols = stage1->strip;
if (!target_has_debug_info(g->zig_target)) {
g->strip_debug_symbols = true;
}
}
add_package(g, stage1->root_pkg, g->main_pkg);
codegen_build_object(g);
}

217
src/stage1.h Normal file
View File

@ -0,0 +1,217 @@
/*
* Copyright (c) 2020 Andrew Kelley
*
* This file is part of zig, which is MIT licensed.
* See http://opensource.org/licenses/MIT
*/
// This file deals with exposing stage1 C++ code to stage2 Zig code.
#ifndef ZIG_STAGE1_H
#define ZIG_STAGE1_H
#include "zig_llvm.h"
#include <stddef.h>
#ifdef __cplusplus
#define ZIG_EXTERN_C extern "C"
#else
#define ZIG_EXTERN_C
#endif
// ABI warning
enum ErrColor {
ErrColorAuto,
ErrColorOff,
ErrColorOn,
};
// ABI warning
enum CodeModel {
CodeModelDefault,
CodeModelTiny,
CodeModelSmall,
CodeModelKernel,
CodeModelMedium,
CodeModelLarge,
};
// ABI warning
enum TargetSubsystem {
TargetSubsystemConsole,
TargetSubsystemWindows,
TargetSubsystemPosix,
TargetSubsystemNative,
TargetSubsystemEfiApplication,
TargetSubsystemEfiBootServiceDriver,
TargetSubsystemEfiRom,
TargetSubsystemEfiRuntimeDriver,
// This means Zig should infer the subsystem.
// It's last so that the indexes of other items can line up
// with the enum in builtin.zig.
TargetSubsystemAuto
};
// ABI warning
// Synchronize with target.cpp::os_list
enum Os {
OsFreestanding,
OsAnanas,
OsCloudABI,
OsDragonFly,
OsFreeBSD,
OsFuchsia,
OsIOS,
OsKFreeBSD,
OsLinux,
OsLv2, // PS3
OsMacOSX,
OsNetBSD,
OsOpenBSD,
OsSolaris,
OsWindows,
OsHaiku,
OsMinix,
OsRTEMS,
OsNaCl, // Native Client
OsCNK, // BG/P Compute-Node Kernel
OsAIX,
OsCUDA, // NVIDIA CUDA
OsNVCL, // NVIDIA OpenCL
OsAMDHSA, // AMD HSA Runtime
OsPS4,
OsELFIAMCU,
OsTvOS, // Apple tvOS
OsWatchOS, // Apple watchOS
OsMesa3D,
OsContiki,
OsAMDPAL,
OsHermitCore,
OsHurd,
OsWASI,
OsEmscripten,
OsUefi,
OsOther,
};
// ABI warning
struct ZigTarget {
enum ZigLLVM_ArchType arch;
enum ZigLLVM_VendorType vendor;
enum ZigLLVM_EnvironmentType abi;
Os os;
bool is_native_os;
bool is_native_cpu;
const char *llvm_cpu_name;
const char *llvm_cpu_features;
const char *cpu_builtin_str;
const char *os_builtin_str;
const char *dynamic_linker;
const char **llvm_cpu_features_asm_ptr;
size_t llvm_cpu_features_asm_len;
};
// ABI warning
struct Stage2Progress;
// ABI warning
struct Stage2ProgressNode;
enum BuildMode {
BuildModeDebug,
BuildModeFastRelease,
BuildModeSafeRelease,
BuildModeSmallRelease,
};
struct ZigStage1Pkg {
const char *name_ptr;
size_t name_len;
const char *path_ptr;
size_t path_len;
struct ZigStage1Pkg **children_ptr;
size_t children_len;
struct ZigStage1Pkg *parent;
};
// This struct is used by both main.cpp and stage1.zig.
struct ZigStage1 {
const char *root_name_ptr;
size_t root_name_len;
const char *output_dir_ptr;
size_t output_dir_len;
const char *builtin_zig_path_ptr;
size_t builtin_zig_path_len;
const char *test_filter_ptr;
size_t test_filter_len;
const char *test_name_prefix_ptr;
size_t test_name_prefix_len;
const char *zig_lib_dir_ptr;
size_t zig_lib_dir_len;
const char *zig_std_dir_ptr;
size_t zig_std_dir_len;
void *userdata;
struct ZigStage1Pkg *root_pkg;
CodeModel code_model;
TargetSubsystem subsystem;
ErrColor err_color;
bool pic;
bool link_libc;
bool link_libcpp;
bool strip;
bool is_single_threaded;
bool dll_export_fns;
bool link_mode_dynamic;
bool valgrind_enabled;
bool function_sections;
bool enable_stack_probing;
bool enable_time_report;
bool enable_stack_report;
bool dump_analysis;
bool enable_doc_generation;
bool emit_bin;
bool emit_asm;
bool emit_llvm_ir;
bool test_is_evented;
bool verbose_tokenize;
bool verbose_ast;
bool verbose_link;
bool verbose_ir;
bool verbose_llvm_ir;
bool verbose_cimport;
bool verbose_cc;
bool verbose_llvm_cpu_features;
};
ZIG_EXTERN_C void zig_stage1_os_init(void);
ZIG_EXTERN_C struct ZigStage1 *zig_stage1_create(enum BuildMode optimize_mode,
const char *main_pkg_path_ptr, size_t main_pkg_path_len,
const char *root_src_path_ptr, size_t root_src_path_len,
const char *zig_lib_dir_ptr, size_t zig_lib_dir_len,
const ZigTarget *target, bool is_test_build, Stage2ProgressNode *progress_node);
ZIG_EXTERN_C void zig_stage1_build_object(struct ZigStage1 *);
ZIG_EXTERN_C void zig_stage1_destroy(struct ZigStage1 *);
#endif

131
src/stage2.cpp
View File

@ -5,40 +5,12 @@
#include "util.hpp"
#include "zig_llvm.h"
#include "target.hpp"
#include "buffer.hpp"
#include "os.hpp"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
Error stage2_translate_c(struct Stage2Ast **out_ast,
struct Stage2ErrorMsg **out_errors_ptr, size_t *out_errors_len,
const char **args_begin, const char **args_end, const char *resources_path)
{
const char *msg = "stage0 called stage2_translate_c";
stage2_panic(msg, strlen(msg));
}
void stage2_free_clang_errors(struct Stage2ErrorMsg *ptr, size_t len) {
const char *msg = "stage0 called stage2_free_clang_errors";
stage2_panic(msg, strlen(msg));
}
void stage2_zen(const char **ptr, size_t *len) {
const char *msg = "stage0 called stage2_zen";
stage2_panic(msg, strlen(msg));
}
int stage2_env(int argc, char** argv) {
const char *msg = "stage0 called stage2_env";
stage2_panic(msg, strlen(msg));
}
int stage2_cc(int argc, char** argv, bool is_cpp) {
const char *msg = "stage0 called stage2_cc";
stage2_panic(msg, strlen(msg));
}
void stage2_attach_segfault_handler(void) { }
void stage2_panic(const char *ptr, size_t len) {
fwrite(ptr, 1, len, stderr);
fprintf(stderr, "\n");
@ -46,32 +18,6 @@ void stage2_panic(const char *ptr, size_t len) {
abort();
}
void stage2_render_ast(struct Stage2Ast *ast, FILE *output_file) {
const char *msg = "stage0 called stage2_render_ast";
stage2_panic(msg, strlen(msg));
}
int stage2_fmt(int argc, char **argv) {
const char *msg = "stage0 called stage2_fmt";
stage2_panic(msg, strlen(msg));
}
stage2_DepTokenizer stage2_DepTokenizer_init(const char *input, size_t len) {
const char *msg = "stage0 called stage2_DepTokenizer_init";
stage2_panic(msg, strlen(msg));
}
void stage2_DepTokenizer_deinit(stage2_DepTokenizer *self) {
const char *msg = "stage0 called stage2_DepTokenizer_deinit";
stage2_panic(msg, strlen(msg));
}
stage2_DepNextResult stage2_DepTokenizer_next(stage2_DepTokenizer *self) {
const char *msg = "stage0 called stage2_DepTokenizer_next";
stage2_panic(msg, strlen(msg));
}
struct Stage2Progress {
int trash;
};
@ -196,10 +142,6 @@ static void get_native_target(ZigTarget *target) {
if (target->abi == ZigLLVM_UnknownEnvironment) {
target->abi = target_default_abi(target->arch, target->os);
}
if (target_is_glibc(target)) {
target->glibc_or_darwin_version = heap::c_allocator.create<Stage2SemVer>();
target_init_default_glibc_version(target);
}
}
Error stage2_target_parse(struct ZigTarget *target, const char *zig_triple, const char *mcpu,
@ -217,13 +159,11 @@ Error stage2_target_parse(struct ZigTarget *target, const char *zig_triple, cons
if (mcpu == nullptr) {
target->llvm_cpu_name = ZigLLVMGetHostCPUName();
target->llvm_cpu_features = ZigLLVMGetNativeFeatures();
target->cache_hash = "native\n\n";
} else if (strcmp(mcpu, "baseline") == 0) {
target->is_native_os = false;
target->is_native_cpu = false;
target->llvm_cpu_name = "";
target->llvm_cpu_features = "";
target->cache_hash = "baseline\n\n";
} else {
const char *msg = "stage0 can't handle CPU/features in the target";
stage2_panic(msg, strlen(msg));
@ -264,11 +204,8 @@ Error stage2_target_parse(struct ZigTarget *target, const char *zig_triple, cons
const char *msg = "stage0 can't handle CPU/features in the target";
stage2_panic(msg, strlen(msg));
}
target->cache_hash = "\n\n";
}
target->cache_hash_len = strlen(target->cache_hash);
if (dynamic_linker != nullptr) {
target->dynamic_linker = dynamic_linker;
}
@ -276,49 +213,29 @@ Error stage2_target_parse(struct ZigTarget *target, const char *zig_triple, cons
return ErrorNone;
}
int stage2_cmd_targets(const char *zig_triple, const char *mcpu, const char *dynamic_linker) {
const char *msg = "stage0 called stage2_cmd_targets";
const char *stage2_fetch_file(struct ZigStage1 *stage1, const char *path_ptr, size_t path_len,
size_t *result_len)
{
Error err;
Buf contents_buf = BUF_INIT;
Buf path_buf = BUF_INIT;
buf_init_from_mem(&path_buf, path_ptr, path_len);
if ((err = os_fetch_file_path(&path_buf, &contents_buf))) {
return nullptr;
}
*result_len = buf_len(&contents_buf);
return buf_ptr(&contents_buf);
}
const char *stage2_cimport(struct ZigStage1 *stage1) {
const char *msg = "stage0 called stage2_cimport";
stage2_panic(msg, strlen(msg));
}
enum Error stage2_libc_parse(struct Stage2LibCInstallation *libc, const char *libc_file) {
libc->include_dir = "/dummy/include";
libc->include_dir_len = strlen(libc->include_dir);
libc->sys_include_dir = "/dummy/sys/include";
libc->sys_include_dir_len = strlen(libc->sys_include_dir);
libc->crt_dir = "";
libc->crt_dir_len = strlen(libc->crt_dir);
libc->msvc_lib_dir = "";
libc->msvc_lib_dir_len = strlen(libc->msvc_lib_dir);
libc->kernel32_lib_dir = "";
libc->kernel32_lib_dir_len = strlen(libc->kernel32_lib_dir);
return ErrorNone;
const char *stage2_add_link_lib(struct ZigStage1 *stage1,
const char *lib_name_ptr, size_t lib_name_len,
const char *symbol_name_ptr, size_t symbol_name_len)
{
return nullptr;
}
enum Error stage2_libc_render(struct Stage2LibCInstallation *self, FILE *file) {
const char *msg = "stage0 called stage2_libc_render";
stage2_panic(msg, strlen(msg));
}
enum Error stage2_libc_find_native(struct Stage2LibCInstallation *libc) {
const char *msg = "stage0 called stage2_libc_find_native";
stage2_panic(msg, strlen(msg));
}
enum Error stage2_detect_native_paths(struct Stage2NativePaths *native_paths) {
native_paths->include_dirs_ptr = nullptr;
native_paths->include_dirs_len = 0;
native_paths->lib_dirs_ptr = nullptr;
native_paths->lib_dirs_len = 0;
native_paths->rpaths_ptr = nullptr;
native_paths->rpaths_len = 0;
native_paths->warnings_ptr = nullptr;
native_paths->warnings_len = 0;
return ErrorNone;
}
const bool stage2_is_zig0 = true;

182
src/stage2.h
View File

@ -5,6 +5,8 @@
* See http://opensource.org/licenses/MIT
*/
// This file deals with exposing stage2 Zig code to stage1 C++ code.
#ifndef ZIG_STAGE2_H
#define ZIG_STAGE2_H
@ -12,7 +14,7 @@
#include <stdint.h>
#include <stdio.h>
#include "zig_llvm.h"
#include "stage1.h"
#ifdef __cplusplus
#define ZIG_EXTERN_C extern "C"
@ -27,7 +29,7 @@
#endif
// ABI warning: the types and declarations in this file must match both those in
// stage2.cpp and src-self-hosted/stage2.zig.
// stage2.cpp and src-self-hosted/stage1.zig.
// ABI warning
enum Error {
@ -124,74 +126,9 @@ struct Stage2ErrorMsg {
unsigned offset; // byte offset into source
};
// ABI warning
struct Stage2Ast;
// ABI warning
ZIG_EXTERN_C enum Error stage2_translate_c(struct Stage2Ast **out_ast,
struct Stage2ErrorMsg **out_errors_ptr, size_t *out_errors_len,
const char **args_begin, const char **args_end, const char *resources_path);
// ABI warning
ZIG_EXTERN_C void stage2_free_clang_errors(struct Stage2ErrorMsg *ptr, size_t len);
// ABI warning
ZIG_EXTERN_C void stage2_render_ast(struct Stage2Ast *ast, FILE *output_file);
// ABI warning
ZIG_EXTERN_C void stage2_zen(const char **ptr, size_t *len);
// ABI warning
ZIG_EXTERN_C int stage2_env(int argc, char **argv);
// ABI warning
ZIG_EXTERN_C int stage2_cc(int argc, char **argv, bool is_cpp);
// ABI warning
ZIG_EXTERN_C void stage2_attach_segfault_handler(void);
// ABI warning
ZIG_EXTERN_C ZIG_ATTRIBUTE_NORETURN void stage2_panic(const char *ptr, size_t len);
// ABI warning
ZIG_EXTERN_C int stage2_fmt(int argc, char **argv);
// ABI warning
struct stage2_DepTokenizer {
void *handle;
};
// ABI warning
struct stage2_DepNextResult {
enum TypeId {
error,
null,
target,
prereq,
};
TypeId type_id;
// when ent == error --> error text
// when ent == null --> undefined
// when ent == target --> target pathname
// when ent == prereq --> prereq pathname
const char *textz;
};
// ABI warning
ZIG_EXTERN_C stage2_DepTokenizer stage2_DepTokenizer_init(const char *input, size_t len);
// ABI warning
ZIG_EXTERN_C void stage2_DepTokenizer_deinit(stage2_DepTokenizer *self);
// ABI warning
ZIG_EXTERN_C stage2_DepNextResult stage2_DepTokenizer_next(stage2_DepTokenizer *self);
// ABI warning
struct Stage2Progress;
// ABI warning
struct Stage2ProgressNode;
// ABI warning
ZIG_EXTERN_C Stage2Progress *stage2_progress_create(void);
// ABI warning
@ -212,69 +149,6 @@ ZIG_EXTERN_C void stage2_progress_complete_one(Stage2ProgressNode *node);
ZIG_EXTERN_C void stage2_progress_update_node(Stage2ProgressNode *node,
size_t completed_count, size_t estimated_total_items);
// ABI warning
struct Stage2LibCInstallation {
const char *include_dir;
size_t include_dir_len;
const char *sys_include_dir;
size_t sys_include_dir_len;
const char *crt_dir;
size_t crt_dir_len;
const char *msvc_lib_dir;
size_t msvc_lib_dir_len;
const char *kernel32_lib_dir;
size_t kernel32_lib_dir_len;
};
// ABI warning
ZIG_EXTERN_C enum Error stage2_libc_parse(struct Stage2LibCInstallation *libc, const char *libc_file);
// ABI warning
ZIG_EXTERN_C enum Error stage2_libc_render(struct Stage2LibCInstallation *self, FILE *file);
// ABI warning
ZIG_EXTERN_C enum Error stage2_libc_find_native(struct Stage2LibCInstallation *libc);
// ABI warning
// Synchronize with target.cpp::os_list
enum Os {
OsFreestanding,
OsAnanas,
OsCloudABI,
OsDragonFly,
OsFreeBSD,
OsFuchsia,
OsIOS,
OsKFreeBSD,
OsLinux,
OsLv2, // PS3
OsMacOSX,
OsNetBSD,
OsOpenBSD,
OsSolaris,
OsWindows,
OsHaiku,
OsMinix,
OsRTEMS,
OsNaCl, // Native Client
OsCNK, // BG/P Compute-Node Kernel
OsAIX,
OsCUDA, // NVIDIA CUDA
OsNVCL, // NVIDIA OpenCL
OsAMDHSA, // AMD HSA Runtime
OsPS4,
OsELFIAMCU,
OsTvOS, // Apple tvOS
OsWatchOS, // Apple watchOS
OsMesa3D,
OsContiki,
OsAMDPAL,
OsHermitCore,
OsHurd,
OsWASI,
OsEmscripten,
OsUefi,
OsOther,
};
// ABI warning
struct Stage2SemVer {
uint32_t major;
@ -282,56 +156,20 @@ struct Stage2SemVer {
uint32_t patch;
};
// ABI warning
struct ZigTarget {
enum ZigLLVM_ArchType arch;
enum ZigLLVM_VendorType vendor;
enum ZigLLVM_EnvironmentType abi;
Os os;
bool is_native_os;
bool is_native_cpu;
// null means default. this is double-purposed to be darwin min version
struct Stage2SemVer *glibc_or_darwin_version;
const char *llvm_cpu_name;
const char *llvm_cpu_features;
const char *cpu_builtin_str;
const char *cache_hash;
size_t cache_hash_len;
const char *os_builtin_str;
const char *dynamic_linker;
const char *standard_dynamic_linker_path;
const char **llvm_cpu_features_asm_ptr;
size_t llvm_cpu_features_asm_len;
};
// ABI warning
ZIG_EXTERN_C enum Error stage2_target_parse(struct ZigTarget *target, const char *zig_triple, const char *mcpu,
const char *dynamic_linker);
// ABI warning
ZIG_EXTERN_C int stage2_cmd_targets(const char *zig_triple, const char *mcpu, const char *dynamic_linker);
ZIG_EXTERN_C const char *stage2_fetch_file(struct ZigStage1 *stage1, const char *path_ptr, size_t path_len,
size_t *result_len);
// ABI warning
struct Stage2NativePaths {
const char **include_dirs_ptr;
size_t include_dirs_len;
const char **lib_dirs_ptr;
size_t lib_dirs_len;
const char **rpaths_ptr;
size_t rpaths_len;
const char **warnings_ptr;
size_t warnings_len;
};
// ABI warning
ZIG_EXTERN_C enum Error stage2_detect_native_paths(struct Stage2NativePaths *native_paths);
ZIG_EXTERN_C const char *stage2_cimport(struct ZigStage1 *stage1);
// ABI warning
ZIG_EXTERN_C const bool stage2_is_zig0;
ZIG_EXTERN_C const char *stage2_add_link_lib(struct ZigStage1 *stage1,
const char *lib_name_ptr, size_t lib_name_len,
const char *symbol_name_ptr, size_t symbol_name_len);
#endif

127
src/target.cpp
View File

@ -10,8 +10,6 @@
#include "target.hpp"
#include "util.hpp"
#include "os.hpp"
#include "compiler.hpp"
#include "glibc.hpp"
#include <stdio.h>
@ -346,33 +344,6 @@ const char *target_abi_name(ZigLLVM_EnvironmentType abi) {
return ZigLLVMGetEnvironmentTypeName(abi);
}
Error target_parse_glibc_version(Stage2SemVer *glibc_ver, const char *text) {
glibc_ver->major = 2;
glibc_ver->minor = 0;
glibc_ver->patch = 0;
SplitIterator it = memSplit(str(text), str("GLIBC_."));
{
Optional<Slice<uint8_t>> opt_component = SplitIterator_next(&it);
if (!opt_component.is_some) return ErrorUnknownABI;
glibc_ver->major = strtoul(buf_ptr(buf_create_from_slice(opt_component.value)), nullptr, 10);
}
{
Optional<Slice<uint8_t>> opt_component = SplitIterator_next(&it);
if (!opt_component.is_some) return ErrorNone;
glibc_ver->minor = strtoul(buf_ptr(buf_create_from_slice(opt_component.value)), nullptr, 10);
}
{
Optional<Slice<uint8_t>> opt_component = SplitIterator_next(&it);
if (!opt_component.is_some) return ErrorNone;
glibc_ver->patch = strtoul(buf_ptr(buf_create_from_slice(opt_component.value)), nullptr, 10);
}
return ErrorNone;
}
void target_init_default_glibc_version(ZigTarget *target) {
*target->glibc_or_darwin_version = {2, 17, 0};
}
Error target_parse_arch(ZigLLVM_ArchType *out_arch, const char *arch_ptr, size_t arch_len) {
*out_arch = ZigLLVM_UnknownArch;
for (size_t arch_i = 0; arch_i < array_length(arch_list); arch_i += 1) {
@ -756,66 +727,6 @@ const char *target_llvm_ir_file_ext(const ZigTarget *target) {
return ".ll";
}
const char *target_exe_file_ext(const ZigTarget *target) {
if (target->os == OsWindows) {
return ".exe";
} else if (target->os == OsUefi) {
return ".efi";
} else if (target_is_wasm(target)) {
return ".wasm";
} else {
return "";
}
}
const char *target_lib_file_prefix(const ZigTarget *target) {
if ((target->os == OsWindows && !target_abi_is_gnu(target->abi)) ||
target->os == OsUefi ||
target_is_wasm(target))
{
return "";
} else {
return "lib";
}
}
const char *target_lib_file_ext(const ZigTarget *target, bool is_static, bool is_versioned,
size_t version_major, size_t version_minor, size_t version_patch)
{
if (target_is_wasm(target)) {
return ".wasm";
}
if (target->os == OsWindows || target->os == OsUefi) {
if (is_static) {
if (target->os == OsWindows && target_abi_is_gnu(target->abi)) {
return ".a";
} else {
return ".lib";
}
} else {
return ".dll";
}
} else {
if (is_static) {
return ".a";
} else if (target_os_is_darwin(target->os)) {
if (is_versioned) {
return buf_ptr(buf_sprintf(".%" ZIG_PRI_usize ".%" ZIG_PRI_usize ".%" ZIG_PRI_usize ".dylib",
version_major, version_minor, version_patch));
} else {
return ".dylib";
}
} else {
if (is_versioned) {
return buf_ptr(buf_sprintf(".so.%" ZIG_PRI_usize ".%" ZIG_PRI_usize ".%" ZIG_PRI_usize,
version_major, version_minor, version_patch));
} else {
return ".so";
}
}
}
}
bool target_is_android(const ZigTarget *target) {
return target->abi == ZigLLVM_Android;
}
@ -992,40 +903,6 @@ bool target_os_requires_libc(Os os) {
return (target_os_is_darwin(os) || os == OsFreeBSD || os == OsNetBSD || os == OsDragonFly);
}
bool target_supports_fpic(const ZigTarget *target) {
// This is not whether the target supports Position Independent Code, but whether the -fPIC
// C compiler argument is valid.
return target->os != OsWindows;
}
bool target_supports_clang_march_native(const ZigTarget *target) {
// Whether clang supports -march=native on this target.
// Arguably it should always work, but in reality it gives:
// error: the clang compiler does not support '-march=native'
// If we move CPU detection logic into Zig itelf, we will not need this,
// instead we will always pass target features and CPU configuration explicitly.
return target->arch != ZigLLVM_aarch64 &&
target->arch != ZigLLVM_aarch64_be;
}
bool target_supports_stack_probing(const ZigTarget *target) {
return target->os != OsWindows && target->os != OsUefi && (target->arch == ZigLLVM_x86 || target->arch == ZigLLVM_x86_64);
}
bool target_supports_sanitize_c(const ZigTarget *target) {
return true;
}
bool target_requires_pic(const ZigTarget *target, bool linking_libc) {
// This function returns whether non-pic code is completely invalid on the given target.
return target_is_android(target) || target->os == OsWindows || target->os == OsUefi || target_os_requires_libc(target->os) ||
(linking_libc && target_is_glibc(target));
}
bool target_requires_pie(const ZigTarget *target) {
return target_is_android(target);
}
bool target_is_glibc(const ZigTarget *target) {
return target->os == OsLinux && target_abi_is_gnu(target->abi);
}
@ -1038,10 +915,6 @@ bool target_is_wasm(const ZigTarget *target) {
return target->arch == ZigLLVM_wasm32 || target->arch == ZigLLVM_wasm64;
}
bool target_is_single_threaded(const ZigTarget *target) {
return target_is_wasm(target);
}
ZigLLVM_EnvironmentType target_default_abi(ZigLLVM_ArchType arch, Os os) {
if (arch == ZigLLVM_wasm32 || arch == ZigLLVM_wasm64) {
return ZigLLVM_Musl;

30
src/target.hpp
View File

@ -12,22 +12,6 @@
struct Buf;
enum TargetSubsystem {
TargetSubsystemConsole,
TargetSubsystemWindows,
TargetSubsystemPosix,
TargetSubsystemNative,
TargetSubsystemEfiApplication,
TargetSubsystemEfiBootServiceDriver,
TargetSubsystemEfiRom,
TargetSubsystemEfiRuntimeDriver,
// This means Zig should infer the subsystem.
// It's last so that the indexes of other items can line up
// with the enum in builtin.zig.
TargetSubsystemAuto
};
enum CIntType {
CIntTypeShort,
CIntTypeUShort,
@ -46,9 +30,6 @@ Error target_parse_arch(ZigLLVM_ArchType *arch, const char *arch_ptr, size_t arc
Error target_parse_os(Os *os, const char *os_ptr, size_t os_len);
Error target_parse_abi(ZigLLVM_EnvironmentType *abi, const char *abi_ptr, size_t abi_len);
Error target_parse_glibc_version(Stage2SemVer *out, const char *text);
void target_init_default_glibc_version(ZigTarget *target);
size_t target_arch_count(void);
ZigLLVM_ArchType target_arch_enum(size_t index);
const char *target_arch_name(ZigLLVM_ArchType arch);
@ -85,10 +66,6 @@ uint32_t target_c_type_size_in_bits(const ZigTarget *target, CIntType id);
const char *target_o_file_ext(const ZigTarget *target);
const char *target_asm_file_ext(const ZigTarget *target);
const char *target_llvm_ir_file_ext(const ZigTarget *target);
const char *target_exe_file_ext(const ZigTarget *target);
const char *target_lib_file_prefix(const ZigTarget *target);
const char *target_lib_file_ext(const ZigTarget *target, bool is_static, bool is_versioned,
size_t version_major, size_t version_minor, size_t version_patch);
bool target_can_exec(const ZigTarget *host_target, const ZigTarget *guest_target);
ZigLLVM_OSType get_llvm_os_type(Os os_type);
@ -104,10 +81,6 @@ bool target_can_build_libc(const ZigTarget *target);
const char *target_libc_generic_name(const ZigTarget *target);
bool target_is_libc_lib_name(const ZigTarget *target, const char *name);
bool target_is_libcpp_lib_name(const ZigTarget *target, const char *name);
bool target_supports_fpic(const ZigTarget *target);
bool target_supports_clang_march_native(const ZigTarget *target);
bool target_requires_pic(const ZigTarget *target, bool linking_libc);
bool target_requires_pie(const ZigTarget *target);
bool target_abi_is_gnu(ZigLLVM_EnvironmentType abi);
bool target_abi_is_musl(ZigLLVM_EnvironmentType abi);
bool target_is_glibc(const ZigTarget *target);
@ -115,9 +88,6 @@ bool target_is_musl(const ZigTarget *target);
bool target_is_wasm(const ZigTarget *target);
bool target_is_riscv(const ZigTarget *target);
bool target_is_android(const ZigTarget *target);
bool target_is_single_threaded(const ZigTarget *target);
bool target_supports_stack_probing(const ZigTarget *target);
bool target_supports_sanitize_c(const ZigTarget *target);
bool target_has_debug_info(const ZigTarget *target);
const char *target_arch_musl_name(ZigLLVM_ArchType arch);

307
src/zig0.cpp Normal file
View File

@ -0,0 +1,307 @@
/*
* Copyright (c) 2015 Andrew Kelley
*
* This file is part of zig, which is MIT licensed.
* See http://opensource.org/licenses/MIT
*/
// This file is the entry point for zig0, which is *only* used to build
// stage2, the self-hosted compiler, into an object file, which is then
// linked by the same build system (cmake) that linked this binary.
#include "stage1.h"
#include "heap.hpp"
#include "stage2.h"
#include "target.hpp"
#include "error.hpp"
#include <stdio.h>
#include <string.h>
static int print_error_usage(const char *arg0) {
fprintf(stderr, "See `%s --help` for detailed usage information\n", arg0);
return EXIT_FAILURE;
}
static int print_full_usage(const char *arg0, FILE *file, int return_code) {
fprintf(file,
"Usage: %s [options] builds an object file\n"
"\n"
"Options:\n"
" --color [auto|off|on] enable or disable colored error messages\n"
" --name [name] override output name\n"
" --output-dir [dir] override output directory (defaults to cwd)\n"
" --pkg-begin [name] [path] make pkg available to import and push current pkg\n"
" --pkg-end pop current pkg\n"
" --main-pkg-path set the directory of the root package\n"
" --release-fast build with optimizations on and safety off\n"
" --release-safe build with optimizations on and safety on\n"
" --release-small build with size optimizations on and safety off\n"
" --single-threaded source may assume it is only used single-threaded\n"
" -dynamic create a shared library (.so; .dll; .dylib)\n"
" --strip exclude debug symbols\n"
" -target [name] <arch>-<os>-<abi> see the targets command\n"
" -mcpu [cpu] specify target CPU and feature set\n"
" --verbose-tokenize enable compiler debug output for tokenization\n"
" --verbose-ast enable compiler debug output for AST parsing\n"
" --verbose-link enable compiler debug output for linking\n"
" --verbose-ir enable compiler debug output for Zig IR\n"
" --verbose-llvm-ir enable compiler debug output for LLVM IR\n"
" --verbose-cimport enable compiler debug output for C imports\n"
" --verbose-cc enable compiler debug output for C compilation\n"
" --verbose-llvm-cpu-features enable compiler debug output for LLVM CPU features\n"
"\n"
, arg0);
return return_code;
}
static bool str_starts_with(const char *s1, const char *s2) {
size_t s2_len = strlen(s2);
if (strlen(s1) < s2_len) {
return false;
}
return memcmp(s1, s2, s2_len) == 0;
}
int main_exit(Stage2ProgressNode *root_progress_node, int exit_code) {
if (root_progress_node != nullptr) {
stage2_progress_end(root_progress_node);
}
return exit_code;
}
int main(int argc, char **argv) {
zig_stage1_os_init();
char *arg0 = argv[0];
Error err;
const char *in_file = nullptr;
const char *output_dir = nullptr;
bool strip = false;
const char *out_name = nullptr;
bool verbose_tokenize = false;
bool verbose_ast = false;
bool verbose_link = false;
bool verbose_ir = false;
bool verbose_llvm_ir = false;
bool verbose_cimport = false;
bool verbose_cc = false;
bool verbose_llvm_cpu_features = false;
ErrColor color = ErrColorAuto;
const char *dynamic_linker = nullptr;
bool link_libc = false;
bool link_libcpp = false;
const char *target_string = nullptr;
ZigStage1Pkg *cur_pkg = heap::c_allocator.create<ZigStage1Pkg>();
BuildMode build_mode = BuildModeDebug;
TargetSubsystem subsystem = TargetSubsystemAuto;
const char *override_lib_dir = nullptr;
const char *main_pkg_path = nullptr;
const char *mcpu = nullptr;
for (int i = 1; i < argc; i += 1) {
char *arg = argv[i];
if (arg[0] == '-') {
if (strcmp(arg, "--") == 0) {
fprintf(stderr, "Unexpected end-of-parameter mark: %s\n", arg);
} else if (strcmp(arg, "--release-fast") == 0) {
build_mode = BuildModeFastRelease;
} else if (strcmp(arg, "--release-safe") == 0) {
build_mode = BuildModeSafeRelease;
} else if (strcmp(arg, "--release-small") == 0) {
build_mode = BuildModeSmallRelease;
} else if (strcmp(arg, "--help") == 0) {
return print_full_usage(arg0, stdout, EXIT_SUCCESS);
} else if (strcmp(arg, "--strip") == 0) {
strip = true;
} else if (strcmp(arg, "--verbose-tokenize") == 0) {
verbose_tokenize = true;
} else if (strcmp(arg, "--verbose-ast") == 0) {
verbose_ast = true;
} else if (strcmp(arg, "--verbose-link") == 0) {
verbose_link = true;
} else if (strcmp(arg, "--verbose-ir") == 0) {
verbose_ir = true;
} else if (strcmp(arg, "--verbose-llvm-ir") == 0) {
verbose_llvm_ir = true;
} else if (strcmp(arg, "--verbose-cimport") == 0) {
verbose_cimport = true;
} else if (strcmp(arg, "--verbose-cc") == 0) {
verbose_cc = true;
} else if (strcmp(arg, "--verbose-llvm-cpu-features") == 0) {
verbose_llvm_cpu_features = true;
} else if (arg[1] == 'l' && arg[2] != 0) {
// alias for --library
const char *l = &arg[2];
if (strcmp(l, "c") == 0) {
link_libc = true;
} else if (strcmp(l, "c++") == 0 || strcmp(l, "stdc++") == 0) {
link_libcpp = true;
}
} else if (strcmp(arg, "--pkg-begin") == 0) {
if (i + 2 >= argc) {
fprintf(stderr, "Expected 2 arguments after --pkg-begin\n");
return print_error_usage(arg0);
}
ZigStage1Pkg *new_cur_pkg = heap::c_allocator.create<ZigStage1Pkg>();
i += 1;
new_cur_pkg->name_ptr = argv[i];
new_cur_pkg->name_len = strlen(argv[i]);
i += 1;
new_cur_pkg->path_ptr = argv[i];
new_cur_pkg->path_len = strlen(argv[i]);
new_cur_pkg->parent = cur_pkg;
cur_pkg->children_ptr = heap::c_allocator.reallocate<ZigStage1Pkg *>(cur_pkg->children_ptr,
cur_pkg->children_len, cur_pkg->children_len + 1);
cur_pkg->children_ptr[cur_pkg->children_len] = new_cur_pkg;
cur_pkg->children_len += 1;
cur_pkg = new_cur_pkg;
} else if (strcmp(arg, "--pkg-end") == 0) {
if (cur_pkg->parent == nullptr) {
fprintf(stderr, "Encountered --pkg-end with no matching --pkg-begin\n");
return EXIT_FAILURE;
}
cur_pkg = cur_pkg->parent;
} else if (str_starts_with(arg, "-mcpu=")) {
mcpu = arg + strlen("-mcpu=");
} else if (i + 1 >= argc) {
fprintf(stderr, "Expected another argument after %s\n", arg);
return print_error_usage(arg0);
} else {
i += 1;
if (strcmp(arg, "--output-dir") == 0) {
output_dir = argv[i];
} else if (strcmp(arg, "--color") == 0) {
if (strcmp(argv[i], "auto") == 0) {
color = ErrColorAuto;
} else if (strcmp(argv[i], "on") == 0) {
color = ErrColorOn;
} else if (strcmp(argv[i], "off") == 0) {
color = ErrColorOff;
} else {
fprintf(stderr, "--color options are 'auto', 'on', or 'off'\n");
return print_error_usage(arg0);
}
} else if (strcmp(arg, "--name") == 0) {
out_name = argv[i];
} else if (strcmp(arg, "--dynamic-linker") == 0) {
dynamic_linker = argv[i];
} else if (strcmp(arg, "--override-lib-dir") == 0) {
override_lib_dir = argv[i];
} else if (strcmp(arg, "--main-pkg-path") == 0) {
main_pkg_path = argv[i];
} else if (strcmp(arg, "--library") == 0 || strcmp(arg, "-l") == 0) {
if (strcmp(argv[i], "c") == 0) {
link_libc = true;
} else if (strcmp(argv[i], "c++") == 0 || strcmp(argv[i], "stdc++") == 0) {
link_libcpp = true;
}
} else if (strcmp(arg, "-target") == 0) {
target_string = argv[i];
} else if (strcmp(arg, "--subsystem") == 0) {
if (strcmp(argv[i], "console") == 0) {
subsystem = TargetSubsystemConsole;
} else if (strcmp(argv[i], "windows") == 0) {
subsystem = TargetSubsystemWindows;
} else if (strcmp(argv[i], "posix") == 0) {
subsystem = TargetSubsystemPosix;
} else if (strcmp(argv[i], "native") == 0) {
subsystem = TargetSubsystemNative;
} else if (strcmp(argv[i], "efi_application") == 0) {
subsystem = TargetSubsystemEfiApplication;
} else if (strcmp(argv[i], "efi_boot_service_driver") == 0) {
subsystem = TargetSubsystemEfiBootServiceDriver;
} else if (strcmp(argv[i], "efi_rom") == 0) {
subsystem = TargetSubsystemEfiRom;
} else if (strcmp(argv[i], "efi_runtime_driver") == 0) {
subsystem = TargetSubsystemEfiRuntimeDriver;
} else {
fprintf(stderr, "invalid: --subsystem %s\n"
"Options are:\n"
" console\n"
" windows\n"
" posix\n"
" native\n"
" efi_application\n"
" efi_boot_service_driver\n"
" efi_rom\n"
" efi_runtime_driver\n"
, argv[i]);
return EXIT_FAILURE;
}
} else if (strcmp(arg, "-mcpu") == 0) {
mcpu = argv[i];
} else {
fprintf(stderr, "Invalid argument: %s\n", arg);
return print_error_usage(arg0);
}
}
} else if (!in_file) {
in_file = arg;
} else {
fprintf(stderr, "Unexpected extra parameter: %s\n", arg);
return print_error_usage(arg0);
}
}
if (cur_pkg->parent != nullptr) {
fprintf(stderr, "Unmatched --pkg-begin\n");
return EXIT_FAILURE;
}
Stage2Progress *progress = stage2_progress_create();
Stage2ProgressNode *root_progress_node = stage2_progress_start_root(progress, "", 0, 0);
if (color == ErrColorOff) stage2_progress_disable_tty(progress);
ZigTarget target;
if ((err = target_parse_triple(&target, target_string, mcpu, dynamic_linker))) {
fprintf(stderr, "invalid target: %s\n", err_str(err));
return print_error_usage(arg0);
}
if (in_file == nullptr) {
fprintf(stderr, "missing zig file\n");
return print_error_usage(arg0);
}
if (out_name == nullptr) {
fprintf(stderr, "missing --name\n");
return print_error_usage(arg0);
}
ZigStage1 *stage1 = zig_stage1_create(build_mode,
main_pkg_path, (main_pkg_path == nullptr) ? 0 : strlen(main_pkg_path),
in_file, strlen(in_file),
override_lib_dir, strlen(override_lib_dir), &target, false,
root_progress_node);
stage1->root_name_ptr = out_name;
stage1->root_name_len = strlen(out_name);
stage1->strip = strip;
stage1->verbose_tokenize = verbose_tokenize;
stage1->verbose_ast = verbose_ast;
stage1->verbose_link = verbose_link;
stage1->verbose_ir = verbose_ir;
stage1->verbose_llvm_ir = verbose_llvm_ir;
stage1->verbose_cimport = verbose_cimport;
stage1->verbose_cc = verbose_cc;
stage1->verbose_llvm_cpu_features = verbose_llvm_cpu_features;
stage1->output_dir_ptr = output_dir;
stage1->output_dir_len = strlen(output_dir);
stage1->root_pkg = cur_pkg;
stage1->err_color = color;
stage1->link_libc = link_libc;
stage1->link_libcpp = link_libcpp;
stage1->subsystem = subsystem;
stage1->pic = true;
stage1->emit_bin = true;
zig_stage1_build_object(stage1);
zig_stage1_destroy(stage1);
return main_exit(root_progress_node, EXIT_SUCCESS);
}