Now that programs are built with their $(EXE) suffix, their installation
rules can be simplified a bit because most of the programs get installed
under the name they have been built with.
This commit touches neither boot/ocamlc nor boot/ocamllex
It has the side-effect of fixing the cleanup rules which did not use the
$(EXE) extension when removing a file although it was produced with the
$(EXE) extension.
When building for the first time, the only requirement is that generated
header files have been built (jumptbl.h, version.h and opnames.h).
Detailed dependency information is only required when headers have been
edited.
COMPUTE_DEPS in Makefile.config controls whether C dependency
information should be generated on a per-file basis. This variable is
controlled by a new --disable-dependency-generation in configure which
is enabled for Git checkouts and disabled for tarballs (i.e. releases).
The Microsoft C compiler (cl) cannot generate dependencies in a
consistent way which we can consume, so for a Git checkout configure
searches for an additional C compiler in order to compute dependencies.
This is obviously not required for a user-build.
As a result, the MSVC port can now safely run make alldepend, since only
OCaml dependency information is committed to the repo after this change.
CI does not need to waste time testing the dependency information,
because it only tests a single build. A single Travis job has been added
which tests the build system code to generate the dependency information
(and provides a single `make -j` run in CI, although Inria's CI also
tests parallel building continuously).
This moves the configure-generated parts of Makefile.common to a
separate (generated) Makefile, allowing Makefile.common to be a normal
Makefile.
OCaml's build system Makefile's now include Makefile.build_config (which
itself includes Makefile.config) but Makefile.config is still installed
as before. This allows configure to generate variables which are
specific to the build process and are not intended to be exported to the
installation.
Before this PR, dynlink_compilerlibs/foo.cmi would not depend on
dynlink_compilerlibs/foo.mli, resulting in object files not being
properly recomputed in incremental-rebuild scenarios.
Before dynlink_compilerlibs/foo.mli depends on
dynlink_compilerlibs/.depend which in turn depends on all copied
source files, the present change has the impact of having the .depend
being regenerated much more often. We change its generator from
$(CAMLRUN) boot/ocamlc to $(BEST_OCAMLDEP) (using ocamlc.opt
when available); on my machine, when ocamlc.opt is available, the
.depend step goes from 0.6s to 0.2s.
Dynlink does a weird dance of copying compiler files from various
places. The 'depend' target does not perform this copying, and instead
just 'touch'es the files that should be present to run -- defined in
DEPEND_DUMMY_FILES: if you run 'depend' after a build, you get the
real content of the files, but if you run without building you get
empty files.
This is (dubious but apparently) correct for most of those files which
should not generate dependencies anyway, but not for
dynlink_platform_intf.mli which contains important dependencies (as
in: removing them breaks the parallel build) and should be properly
built before 'depend' runs.
* Various file moves in the middle end: this is the first stage of improving separation between the middle end and backend.
* Creation of file_formats/ directory (with associated file moves) to hold the definitions of compilation artifact formats.
* Creation of lambda/ directory (with associated file moves) to hold Lambda language definition files, transformation passes and construction passes from Typedtree.
* Disable (hopefully temporarily) dynlink, debugger and ocamldoc for the dune build.
Persistent_env is a new module that handles the relation between the
type-checking state and the "persistent" typing information laying in
.cmi files on the filesystem. In particular, it handles the collection
and production of CRC information for the .cmi files being read and
written to the filesystem; the using modules (in our case, only Env)
are in charge of turning the cmi files into higher-level information
(components and signatures).
Persistent_env exposes a type `'a t` of a persistent environment,
which acts as a mutable store of `'a` values. There is no global state
in the module itself: while Env (and thus the OCaml type-checker) uses
a single global persistent environment, it should be possible to
create several independent environments to represent, for example,
several independent type-checking sessions.
- Add a Load_path module which caches files lookup
- Instead of falling back to the external environment, allow to
declare in the environment that a module comes from the external
world. This allows persistent structures to shadows non-persistent
ones
In order to prepare the transition to autoconf, this commit moves the
configuration Makefile out of the config directory which will disappear
and gives it the name it will have once intstalled, namely Makefile.config.
The proposed behavior of `-config-var s` is as follows:
- if `s` is an existing configuration variable, print its value as
a string and exit with a success return value (0)
- if `s` is not an existing configuration variable, print nothing
and exit with a failure return value (non-0)
Note that we do not print a newline after the value of the
configuration variable. In particular, if the value is an empty
string, the output is undistinguishable from the output for
non-existing variables, the return value has to be considered instead.
The following alternative behaviors were considered:
- We could print a newline after the configuration value, which
would let users distinguish empty values from non-existing variables
by counting the lines of output, and would also be more pleasant for
users invoking the option from the command-line. However, the way
bash works on Windows means that $(ocamlc -config-var foo) would keep
a trailing \r in its output, and portable scripts would have to use
$(ocamlc -config-var foo | tr -d '\r') instead, which is a pain.
(This issue was pointed out by David Allsopp)
- We could print a message on the error output if the configuration
variable does not exist. This is clearer to a human user, but it is
annoying for scripts if they forget to silence the error output and
get their output mixed with our error messages. The main use of this
new feature is for scripting purposes.
I can observe weird performance bottlenecks on my machine caused by
the use of 'cp' in the 'install' scripts of OCaml. When installing
into a directory that is already populated by an existing
installation, 'make install' can routinely take 10s on my machine¹. After this
change it reliably takes 1.5s, independently of whether the
destination is already populated or not.
¹: a brtfs filesystem on an old-ish SSD
Why I care
----------
An extra 10s delay due to 'make install' can be noticeable in tight
change-build-install-test feedback loops for a compiler change where
we change the compiler, have a fast 'make world.opt' due to
incremental builds, install the change and test it -- possibly after
installing a couple opam packages, which can be fairly quick.
Partial diagnosis
-----------------
The performance issue seems to be caused by the fact that 'cp' (at
least the GNU coreutils version), when the file already exists,
replaces it by opening it in writeonly+truncate mode and writing the
file content ('strace' shows that the delay is caused within an
'openat' call). In particular, using the --remove-destination option
(which changes 'cp' to just remove the destination file before
copying) removes the performance issue, but this option seems missing
from the BSD/OSX 'cp' so it could cause portability issue.
Change
------
The present commit rewrites the 'install' targets of all Makefiles to
use the 'install' command instead. 'install' by default gives
executable-like permission to the destination file, instead of reusing
the source file's permissions, so we specify manually the permission
modes, depending on whether the installed file is an executable (or
dynamically-linked library) or just data (including other compiled
object files).
Testing
-------
I checked manually that the permissions of the installed files are
identical to the ones of the current 'cp'-using targets, except for
some '.mli' file in middle_end which currently have +x bits enabled
for no good reason.
Remark: To test this, playing with the DESTDIR variable is very useful
(this lets you install to a new directory (or the same as before)
without having to re-run the configure script). I used the following,
fairly slow shell script to collect permissions:
for f in $(find $DESTDIR); do \
echo $(basename $f) $(ls -l $f | cut -d' ' -f1); \
done | sort
Remark: it is important to run `sync` in-between 'make install' runs
to avoid timing effects due to filesystem or disk caching
strategies. I believe that this corresponds to the natural time delay
(and unrelated disk activity) that would occur in realistic
change-install-test feedback loops.
Thomas Refis
Nicolás Ojeda Bär
Sébastien Hinderer
Gabriel Scherer
David Allsopp
Stephen Dolan
Greta Yorsh
Jeremie Dimino
Mark Shinwell
Leo White
Xavier Clerc
Gabriel Radanne
Runhang Li
hendriktews
Damien Doligez
Alain Frisch
Mark Shinwell