I can observe weird performance bottlenecks on my machine caused by
the use of 'cp' in the 'install' scripts of OCaml. When installing
into a directory that is already populated by an existing
installation, 'make install' can routinely take 10s on my machine¹. After this
change it reliably takes 1.5s, independently of whether the
destination is already populated or not.
¹: a brtfs filesystem on an old-ish SSD
Why I care
----------
An extra 10s delay due to 'make install' can be noticeable in tight
change-build-install-test feedback loops for a compiler change where
we change the compiler, have a fast 'make world.opt' due to
incremental builds, install the change and test it -- possibly after
installing a couple opam packages, which can be fairly quick.
Partial diagnosis
-----------------
The performance issue seems to be caused by the fact that 'cp' (at
least the GNU coreutils version), when the file already exists,
replaces it by opening it in writeonly+truncate mode and writing the
file content ('strace' shows that the delay is caused within an
'openat' call). In particular, using the --remove-destination option
(which changes 'cp' to just remove the destination file before
copying) removes the performance issue, but this option seems missing
from the BSD/OSX 'cp' so it could cause portability issue.
Change
------
The present commit rewrites the 'install' targets of all Makefiles to
use the 'install' command instead. 'install' by default gives
executable-like permission to the destination file, instead of reusing
the source file's permissions, so we specify manually the permission
modes, depending on whether the installed file is an executable (or
dynamically-linked library) or just data (including other compiled
object files).
Testing
-------
I checked manually that the permissions of the installed files are
identical to the ones of the current 'cp'-using targets, except for
some '.mli' file in middle_end which currently have +x bits enabled
for no good reason.
Remark: To test this, playing with the DESTDIR variable is very useful
(this lets you install to a new directory (or the same as before)
without having to re-run the configure script). I used the following,
fairly slow shell script to collect permissions:
for f in $(find $DESTDIR); do \
echo $(basename $f) $(ls -l $f | cut -d' ' -f1); \
done | sort
Remark: it is important to run `sync` in-between 'make install' runs
to avoid timing effects due to filesystem or disk caching
strategies. I believe that this corresponds to the natural time delay
(and unrelated disk activity) that would occur in realistic
change-install-test feedback loops.
This commit adds the following options to ocamldebug:
* -no-prompt: suppress all prompts
* -no-version: do not print version at startup
* -no-time: do not print times
* -no-breakpoint: do not print message at breakpoint setup and removal
* -topdirs-path: set path to the directory containing topdirs.cmi
If you type list at a step where the source info is not available, ocamldebug could crash
due to the uncaught exception. This patch catches the exception and return an user friendy
error message instead.
to warn when the argument is matches against a constant pattern.
This attributes is applied on predefined exception constructors
which take an purely informational (with no stability guarantee)
message.
git-svn-id: http://caml.inria.fr/svn/ocaml/trunk@16502 f963ae5c-01c2-4b8c-9fe0-0dff7051ff02
This should cover all places involving filenames in the compiler.
There are a few more paths still using Latin-1 in other ways,
e.g. in ocamldoc.
From: Peter Zotov <whitequark@whitequark.org>
git-svn-id: http://caml.inria.fr/svn/ocaml/trunk@15727 f963ae5c-01c2-4b8c-9fe0-0dff7051ff02
"print a" in ocamldebug when a is a forced lazy value segfaults on all
OCaml versions I tested it on. This comes from the fact that the
debugcomm.ml re-implementation of the Obj module (talking over the
debugging socket) implements Obj.tag without special-casing integers
(that is, testing is_block first), and always assumes its input is
a block.
git-svn-id: http://caml.inria.fr/svn/ocaml/trunk@15658 f963ae5c-01c2-4b8c-9fe0-0dff7051ff02
(patch by Josh Watzman)
Add absolute directory names to bytecode format for ocamldebug to use
The need for a long list of -I directives makes interactively using
ocamldebug a pain in the butt. Many folks have solved this with various
`find` invocations or even Python wrappers, but those lead to other
problems when it might include files you weren't expecting (or miss
things you were). But all of this is really annoying since the tooling
should be able to figure out itself, even heuristically, where your
source files are -- gdb gets this right, why can't we?
This patch implements one of the more important heuristics from gdb: you
typically debug on the same machine you built on, so looking for the
source files and built artifacts in the absolute paths where they were
during compilation is a good first try. We write out absolute paths into
a new structure at the beginning of the debug section and then
automatically append those directories into the load path.
This means mean that if you happen to be debugging on a machine
where the original source and build artifacts are *not* available in
their original absolute locations, things will work as before, using the
standard load path mechanism. You can also explicitly use -I to prepend
directories to the load path and override the defaults located by this
new mechanism.
I personally find this makes using ocamldebug much more pleasant :)
git-svn-id: http://caml.inria.fr/svn/ocaml/trunk@14533 f963ae5c-01c2-4b8c-9fe0-0dff7051ff02
There seems to be a problematic interaction with backtrace collection. To test this,
make world.opt
cd testsuite
make one DIR=tests/backtrace/
This needs to be sorted out before the patch can go in again.
git-svn-id: http://caml.inria.fr/svn/ocaml/trunk@14510 f963ae5c-01c2-4b8c-9fe0-0dff7051ff02
Noticed that I had to bootstrap to test on ARM, so I commit a new bootstrap
compiler.
git-svn-id: http://caml.inria.fr/svn/ocaml/trunk@14479 f963ae5c-01c2-4b8c-9fe0-0dff7051ff02
14278
14277
14276
14176
14175
14173
14172
14171
14169
14168
14167
These changes need to mature on their own branch.
git-svn-id: http://caml.inria.fr/svn/ocaml/trunk@14329 f963ae5c-01c2-4b8c-9fe0-0dff7051ff02
This script was built from ocamlcomp.sh.in through sed and is called
instead of "ocamlc" (for instance).
It makes it possible to switch from "ocamlc" to "ocamlc.opt" without
changing anything in the Makefiles, only calling sed.
I couldn't cleanly make it handle both a compiler for the target and for
the build. Instead I'm replacing it and doing as much as possible
directly in the Makefiles.
I hoped it would reduce the number of shell invocations, which would
speed things up quite a lot on Windows but I still had to have at least
one since it's not possible to update a make variable from inside a make
rule: i.e. it's not possible to do X=a, build a.opt and update X to be
a.opt.
git-svn-id: http://caml.inria.fr/svn/ocaml/trunk@14168 f963ae5c-01c2-4b8c-9fe0-0dff7051ff02
(Patch by Adrien Nader!)
Several commits ago, I made a change that replaced includes for
"otherlibs/unix" with "otherlibs/$(UNIXLIB)", making it possible to
include "win32unix".
Apparently I forgot to do it for debugger/Makefile or skipped it on
purpose since it only matters for cross-compilation and cross-compiling
the debugger is not supported (yet).
Change it now rather than forget it later.
git-svn-id: http://caml.inria.fr/svn/ocaml/trunk@14164 f963ae5c-01c2-4b8c-9fe0-0dff7051ff02
- Reset most of the fields of Env.t when saving bin-annot files
- Move debugger/envaux.ml to typing/, and add a function to
recover environements from bin-annot files.
- Move tools/typedtreeIter.ml to typing/
- Move the code of typing/typedtreeMap.ml from cmt_format.ml
git-svn-id: http://caml.inria.fr/svn/ocaml/trunk@12702 f963ae5c-01c2-4b8c-9fe0-0dff7051ff02
Nicolás Ojeda Bär
Gabriel Radanne
Sébastien Hinderer
Thomas Refis
Gabriel Scherer
Damien Doligez
Fourchaux
Hugo Herbelin
David Allsopp
Xavier Leroy
Daniel
Yunxing Dai
Alain Frisch
Runhang Li
Fabrice Le Fessant
Jacques Garrigue
Christophe Troestler
Nicolas Ojeda Bar
Mark Shinwell
Mark Shinwell
lijunsong
Jérémie Dimino
Pierre Weis
Damien Doligez
Leo White
Jacques Garrigue
Xavier Clerc
Luc Maranget
Wojciech Meyer
Fabrice Le Fessant