Switch ocamldebug to compiler-libs in order to avoid hidden dependencies issue.
In particular, the Env module is only fully initialized after the Typemod and Includemod modules have been linked. Calling `Env.find_type` before that initialization may result in an assert false when functor-derived types are involved.
This commit adds dynlink support for ocamldebug. As a side effect, it also:
- factorizes the various functions searching for a code fragment into one,
called [caml_find_code_fragment];
- removes the [caml_register_code_fragment], which does not seem to
be used anywhere, and which clearly should not be used by external code.
This commit makes e.g. make -s world.opt completely
silent when everything works, in order to increase the visibility of
any unexpected message occurring during the build.
This will be useful for instance during CI, in particular it should
make it easier to catch undefined build variables.
The implementation is straightforward. The OCAMLLEX_FLAGS variable is
defined in Makefile.common.in and then used consistently by all
lexing recipes.
In addition, in tools/Makefile, the two rules producing the lexers
from cvt_emit.mll and make_opcodes.mll have been replaced by a
pattern-rule and the useless .SUFFIXES target has been removed.
* Various file moves in the middle end: this is the first stage of improving separation between the middle end and backend.
* Creation of file_formats/ directory (with associated file moves) to hold the definitions of compilation artifact formats.
* Creation of lambda/ directory (with associated file moves) to hold Lambda language definition files, transformation passes and construction passes from Typedtree.
* Disable (hopefully temporarily) dynlink, debugger and ocamldoc for the dune build.
The debugger reimplements its own error-reporting logic without using
the reporter-registration mechanism of the compiler, so it needs to be
adapted after the split between `Env` and `Persistent_env` in #2228.
(Interestingly, this forced me to expose the `Error of error`
exception in the Persistent_signature, which was not the case
before. It was probably a mistake to not expose an exception value
that can be raised by (correctly-written) consumers of the module.)
I noticed the issue while inspecting a testsuite failure (#8544).
Before this patch:
```
$ cat tests/tool-debugger/find-artifacts/_ocamltest/tests/tool-debugger/find-artifacts/debuggee/ocamlc.byte/debuggee.byte.output
Loading program... done.
Breakpoint: 1
10 <|b|>print x;
Uncaught exception: Persistent_env.Error(_)
```
After:
```
$ cat tests/tool-debugger/find-artifacts/_ocamltest/tests/tool-debugger/find-artifacts/debuggee/ocamlc.byte/debuggee.byte.output
Loading program... done.
Breakpoint: 1
10 <|b|>print x;
Debugger [version 4.09.0+dev0-2019-01-18] environment error:
The files /usr/local/lib/ocaml/stdlib.cmi
and [...]_ocamltest/tests/tool-debugger/find-artifacts/debuggee/ocamlc.byte/out/blah.cmi
make inconsistent assumptions over interface Stdlib
```
This commit removes support for gprof-based profiling (the -p option to ocamlopt). It follows a discussion on the core developers' list, which indicated that removing gprof support was a reasonable thing to do. The rationale is that there are better easy-to-use profilers out there now, such as perf for Linux and Instruments on macOS; and the gprof support has always been patchy across targets. We save a whole build of the runtime and simplify some other parts of the codebase by removing it.
After consultation on the core developers' list I am proposing this patch to remove support for compiler plugins.
The main motivations for removing compiler plugins are:
- They are a potential security risk.
- They increase the complexity of the build system and make maintenance of the Dynlink libraries more difficult (although actually, this complexity could probably be reduced after #2268 is merged).
- Many applications of plugins should be able to be expressed by building custom compiler drivers that link against compilerlibs.
* Remove compiler plugins and hooks
* Add new function Dynlink.unsafe_get_global_symbol but keep it outside the documented API.
* Remove otherlibs/dynlink/nodynlink.ml
* Update Changes
Persistent_env is a new module that handles the relation between the
type-checking state and the "persistent" typing information laying in
.cmi files on the filesystem. In particular, it handles the collection
and production of CRC information for the .cmi files being read and
written to the filesystem; the using modules (in our case, only Env)
are in charge of turning the cmi files into higher-level information
(components and signatures).
Persistent_env exposes a type `'a t` of a persistent environment,
which acts as a mutable store of `'a` values. There is no global state
in the module itself: while Env (and thus the OCaml type-checker) uses
a single global persistent environment, it should be possible to
create several independent environments to represent, for example,
several independent type-checking sessions.
Note: Typos found with https://github.com/codespell-project/codespell
Here is the (semi-manual) command used to get (and correct) the typos:
$ codespell -i 3 -w --skip=".png,.gif,./ocaml/boot,./ocaml/.git,./ocaml/manual/styles,./ocaml/manual/manual/htmlman" -L minimise,instal,contructor,"o'caml",cristal,pres,clos,cmo,uint,iff,te,objext,nto,nd,mut,upto,larg,exten,leage,mthod,delte,tim,atleast,langage,hten,iwth,mke,contant,succint,methids,eles,valu,clas,modul,que,classe,missings,froms,defaut,correspondance,differents,configury,reachs,cas,approche,normale,dur,millon,amin,oje,transfert
- Add a Load_path module which caches files lookup
- Instead of falling back to the external environment, allow to
declare in the environment that a module comes from the external
world. This allows persistent structures to shadows non-persistent
ones
This change should be a refactoring no-op.
Before, a DEPFLAGS variable existed in some makefiles to contain
include directories to be passed to ocamldep invocations, but no
support for easily adding command-line flags to ocamldep was available
(invocations would systematically use -slash, which was duplicated
across callsites).
With this PR, a new DEPINCLUDES variable contains the include
directories, and DEPFLAGS is repurposed to contain other command-line
flags for the tool -- currently "slash".
- The code responsible for printing Syntaxerr errors is moved to the
Parse module (so that it can depend on the variable printer in
Pprintast).
- Pprintast becomes a dependency for a few tools that link some
compiler modules in an ad hoc way (they would better be implemented
in terms of compiler-libs).
- Ident.create now takes a scope as argument
- added Ident.create_var to use when the scope doesn't matter
- the current_time and the current_level are unrelated as of this
commit. But one has to remember to bump the level when creating new
scopes.
In order to prepare the transition to autoconf, this commit moves the
configuration Makefile out of the config directory which will disappear
and gives it the name it will have once intstalled, namely Makefile.config.
- inline Pervasives in Stdlib and re-add Pervasives as a deprecated
module that aliases all elements of Stdlib except the stdlib modules.
- remove special case for Stdlib.Pervasives in printtyp.ml
I can observe weird performance bottlenecks on my machine caused by
the use of 'cp' in the 'install' scripts of OCaml. When installing
into a directory that is already populated by an existing
installation, 'make install' can routinely take 10s on my machine¹. After this
change it reliably takes 1.5s, independently of whether the
destination is already populated or not.
¹: a brtfs filesystem on an old-ish SSD
Why I care
----------
An extra 10s delay due to 'make install' can be noticeable in tight
change-build-install-test feedback loops for a compiler change where
we change the compiler, have a fast 'make world.opt' due to
incremental builds, install the change and test it -- possibly after
installing a couple opam packages, which can be fairly quick.
Partial diagnosis
-----------------
The performance issue seems to be caused by the fact that 'cp' (at
least the GNU coreutils version), when the file already exists,
replaces it by opening it in writeonly+truncate mode and writing the
file content ('strace' shows that the delay is caused within an
'openat' call). In particular, using the --remove-destination option
(which changes 'cp' to just remove the destination file before
copying) removes the performance issue, but this option seems missing
from the BSD/OSX 'cp' so it could cause portability issue.
Change
------
The present commit rewrites the 'install' targets of all Makefiles to
use the 'install' command instead. 'install' by default gives
executable-like permission to the destination file, instead of reusing
the source file's permissions, so we specify manually the permission
modes, depending on whether the installed file is an executable (or
dynamically-linked library) or just data (including other compiled
object files).
Testing
-------
I checked manually that the permissions of the installed files are
identical to the ones of the current 'cp'-using targets, except for
some '.mli' file in middle_end which currently have +x bits enabled
for no good reason.
Remark: To test this, playing with the DESTDIR variable is very useful
(this lets you install to a new directory (or the same as before)
without having to re-run the configure script). I used the following,
fairly slow shell script to collect permissions:
for f in $(find $DESTDIR); do \
echo $(basename $f) $(ls -l $f | cut -d' ' -f1); \
done | sort
Remark: it is important to run `sync` in-between 'make install' runs
to avoid timing effects due to filesystem or disk caching
strategies. I believe that this corresponds to the natural time delay
(and unrelated disk activity) that would occur in realistic
change-install-test feedback loops.
This commit adds the following options to ocamldebug:
* -no-prompt: suppress all prompts
* -no-version: do not print version at startup
* -no-time: do not print times
* -no-breakpoint: do not print message at breakpoint setup and removal
* -topdirs-path: set path to the directory containing topdirs.cmi
Florian Angeletti
Gabriel Scherer
Jeremie Dimino
Leo White
Jacques-Henri Jourdan
Sébastien Hinderer
Mark Shinwell
Fourchaux
Daniel Bünzli
alainfrisch
Thomas Refis
Thomas Refis
Xavier Clerc
François Bobot
Nicolás Ojeda Bär
Gabriel Radanne
Damien Doligez