The large diff in boot/menhir/parser_menhir.ml comes from the fact
that the token list is now included in it, instead of being merely
a reference to the yacc parsers' Parser.token type.
The goal of this change is to avoid conflicts encountered by
compiler-libs users that would also use their own MenhirLib runtime
for their own parsers.
I first tried to implement a solution to this module-name-conflict
issue using module aliases and -open, but this proven too fragile and
too difficult to get right.
Uses the new $symbolstartpos feature of Menhir
to get locations identical to the OCamlYacc ones.
REBASE POINT: at the point of this commit, using a diff program
on parser.mly and parser_menhir.mlyp should give identical results
after the header code. If you rebase the Menhir-parser patchset
against a newer ocamlyacc parser (parser.mly), those two files
will have diverged, and you need to merge the parser.mly change
back into parser_menhir.mlyp -- and then deal with them in the
rest of the patch series.
- Introduce `-dcamlprimc`, to keep the generated C file containing the primitive list
- Use `-fdebug-prefix-map` for compiling temporary C files when this option is supported
This commit renames a few C compiler related build variables so that
they are reserved for the build system. They will then be re-introduced,
but this time as user varialbes whose value can be freely customized
when compiling the package, without risking to conflict with those
command-line flags that are required by the build system itself.
Here are the variables this commit renames:
- CFLAGS -> OC_CFLAGS
- CPPFLAGS -> OC_CPPFLAGS
- LDFLAGS -> OC_LDFLAGS
Note: before this commit the compilation of scheduler.c in
otherlibs/threads was relying on make's implicit rule to compile C files.
Since this commit stops using the standard variables for flags,
it is necessary to introduce an explicit rule to compile C files
and that makes use of the newly introduced variables.
* Remove the now obsolete comments at the beginning of the main Makefile
* Move and fix documentation of the bootstrap from INSTALL to
HACKING-bootstrap.adoc
* Update install instructions
This commit contains the following fixes and enhancements to the
root Makefile:
* coreall: add dependency on runtime
* all: before this commit, this target was depending on runtime and started
by making coreall. This commit simplifies this and just makes this target
depend on coreall, which is both simpler and semantically more accurate.
* compare: let this target exit if the comparison fails
* bootstrap: this commit removes the final invocation to make compare.
The comparison was actually done twice, (1) as the final step of coreboot
and (2) as the final step of bootstrap. The motivation for that was to
ensure that the comparison messages were not lost and did appear last.
Actually, it turns out that if the comparison fails, it makes no
sense to continue the bootstrap process. That's why compare now exits
immediately when it fails and there is thus no need to perform it twice.
* Get rid of backup and restore targets which have become obsolete
with the use of a revision control system
This tool is used to compile ocamlc. Given that make_opcodes is itself
recompiled as part of the bootstrap process, it needs to be run on the new
(rather than old) runtime. In other words, make_opcodes needs to be run
using byterun/ocamlrun rather than boot/ocamlrun.
This commit fixes this.
The compiler's bootstrap procedure involves calling the promote-cross
target of the root Makefile.
When this target is invoked e.g. in the context of an executable
magic number update, it is actually not possible to use the stripdebug
tool.
Thus, this commit makes sure the promote-cross target uses cp
to promote the bytecode tools, rather than stripdebug.
This is not an issue because in the promote rule used later
during the bootstrap the stripdebug tool can be used, so that the
bytecode executables one gets after a complete bootstrap will remain
small and won't contain any debugging symbol.
In environments where the executables compiled to native code,
such as ocamlopt.opt, are always used in preference to the bytecode
versions then space can be saved by not installing the latter.
This patch provides a configure option to do such. It is relatively lightly
engineered; in particular, it won't complain if the native code executables
aren't themselves being built; but given this is an option for knowledgeable
users we think that it is reasonable.
I can observe weird performance bottlenecks on my machine caused by
the use of 'cp' in the 'install' scripts of OCaml. When installing
into a directory that is already populated by an existing
installation, 'make install' can routinely take 10s on my machine¹. After this
change it reliably takes 1.5s, independently of whether the
destination is already populated or not.
¹: a brtfs filesystem on an old-ish SSD
Why I care
----------
An extra 10s delay due to 'make install' can be noticeable in tight
change-build-install-test feedback loops for a compiler change where
we change the compiler, have a fast 'make world.opt' due to
incremental builds, install the change and test it -- possibly after
installing a couple opam packages, which can be fairly quick.
Partial diagnosis
-----------------
The performance issue seems to be caused by the fact that 'cp' (at
least the GNU coreutils version), when the file already exists,
replaces it by opening it in writeonly+truncate mode and writing the
file content ('strace' shows that the delay is caused within an
'openat' call). In particular, using the --remove-destination option
(which changes 'cp' to just remove the destination file before
copying) removes the performance issue, but this option seems missing
from the BSD/OSX 'cp' so it could cause portability issue.
Change
------
The present commit rewrites the 'install' targets of all Makefiles to
use the 'install' command instead. 'install' by default gives
executable-like permission to the destination file, instead of reusing
the source file's permissions, so we specify manually the permission
modes, depending on whether the installed file is an executable (or
dynamically-linked library) or just data (including other compiled
object files).
Testing
-------
I checked manually that the permissions of the installed files are
identical to the ones of the current 'cp'-using targets, except for
some '.mli' file in middle_end which currently have +x bits enabled
for no good reason.
Remark: To test this, playing with the DESTDIR variable is very useful
(this lets you install to a new directory (or the same as before)
without having to re-run the configure script). I used the following,
fairly slow shell script to collect permissions:
for f in $(find $DESTDIR); do \
echo $(basename $f) $(ls -l $f | cut -d' ' -f1); \
done | sort
Remark: it is important to run `sync` in-between 'make install' runs
to avoid timing effects due to filesystem or disk caching
strategies. I believe that this corresponds to the natural time delay
(and unrelated disk activity) that would occur in realistic
change-install-test feedback loops.
octachron
Gabriel Scherer
Drup
Xavier Clerc
Gabriel Radanne
Valentin Gatien-Baron
Sébastien Hinderer
David Allsopp
Thomas Refis
Mark Shinwell
Leo White
Pierre Chambart
Nicolás Ojeda Bär
Jeremie Dimino