- The code responsible for printing Syntaxerr errors is moved to the
Parse module (so that it can depend on the variable printer in
Pprintast).
- Pprintast becomes a dependency for a few tools that link some
compiler modules in an ad hoc way (they would better be implemented
in terms of compiler-libs).
typo.long-line can still be explicitly set in .gitattributes (and
typo.long-line typo.very-long-line=may will still issue unused if no
lines are more than 80 columns)
Before this patch, check-typo is directed by a single git attribute,
ocaml-typo, which is used as a key to set a value:
ocaml-typo=long-line,missing-header
(the value here is `long-line,missing-header`, and the code splits the
comma later)
This model is very fragile because .gitattributes does not allow to
give attribute keys a collecting/aggregating semantic: each new
setting of the key removes the previous setting, instead of adding to
them. For example,
testsuite/tests/** ocaml-typo=missing-header
testsuite/tests/win-unicode/*.ml ocaml-typo=utf8
and
testsuite/tests/win-unicode/*.ml ocaml-typo=utf8
testsuite/tests/** ocaml-typo=missing-header
are not equivalent, and instead of using either one we would introduce
redundancy for robustness:
testsuite/tests/** ocaml-typo=missing-header
testsuite/tests/win-unicode/*.ml ocaml-typo=missing-header,utf8
With this patch, we switch to a model where each ocaml-typo setting is
its own attribute, of the form `typo.<<attribute>>`. The lines above
would be written, in either order:
testsuite/tests/** typo.missing-header
testsuite/tests/win-unicode/*.ml typo.utf8
Not only does this approach make our .gitattributes more robust, it
allows for a more fine-grained treatment of the "unused-prop"
marker. This was used as an attribute to say: don't make it in an
error if the given typo-rule is in fact respected (by default, opting
out of a typo-rule gives an error if the typo-rule is respected). But
because of the single-key nature of ocaml-typo, unused-prop would
range over all settings, not just one of them. For example
emacs/caml.el ocaml-typo=long-line,unused-prop,missing-header
seems to suggest that 'unused-prop' only qualifies the 'long-line'
rule, but in fact it also ranges over 'missing-header'. In contrast,
with this patch, we write the following:
emacs/caml.el typo.long-line=may typo.missing-header
the `=may` value setting is used to make an exception to a typo-rule
optional.
Interestingly, most .gitattributes lines worked without extra error
when I turned each unused-prop in a =may setting over the rule just
before, instead of all rules: our checking is now more precise than
before, better capturing the intent of the .gitattributes author.
As I had to rewrite parts of the check-typo code for this, I took the
opportunity to rename a couple variables speaking about SVN (now long
defunct) into more meaningful names:
- `$is_svn` => `$path_in_index`
- `$svnrules` => `$attr_rules`
Currently our CI machine on FreeBSD fails to build with the following
error message:
```
bytecomp/generate_runtimedef.sh runtime/caml/fail.h runtime/primitives > bytecomp/runtimedef.ml
/bin/sh: bytecomp/generate_runtimedef.sh: not found
```
This PR converts all the files with a `/bin/bash` shebang to
`/usr/bin/env bash`. Some files use `/bin/sh` instead, and they are
unchanged. It may be the case that some 'bash' files could in fact run
using 'sh', but checking this requires more work and I decided to
follow the original author's intent.
(Personally I like using bash explicitly as it gives a consistent
programming environment that I can easily test on my machine; but this
should not be at the expense of portability.)
* add a release checklist
Currently this list isn't publicly available, it sits in various
different versions on @damiendoligez's filesystems. He sent me a copy
when I took care of some of the recent releases. The present
presentation is a result of significant cleanups and changes to the
checklist -- in particular, some mistakes may have jumped in.
This is not a scripted process, it is very informal and it is likely
that there are some mistakes/omissions in the list. Yet, it sounds
better to have it somewhere in the source repository than not have it
around at all. It was certainly helpful to me, and it probably would
be to other release-help volunteers.
(The list is put in tools/ because there isn't a clearly better place
for it. This choice was suggested by Damien.)
In order to prepare the transition to autoconf, this commit moves the
configuration Makefile out of the config directory which will disappear
and gives it the name it will have once intstalled, namely Makefile.config.
Some Makefiles were using export to set OCAML_FLEXLINK "globally" while
others set a variable FLEXLINK_ENV and set the environment explicitly.
All Makefiles now use FLEXLINK_ENV and also only invoke it on linking
commands (rather than, for example, all invocations of ocamlopt).
I hit a Lexer error by mistakenly closing an environment with
\end{caml_example*}{verbatim}
instead of
\end{caml_example*}
and caml_tex would not quote the wrong input or indicate
at which line the error was, which makes debugging painful.
The goal of this change is to avoid conflicts encountered by
compiler-libs users that would also use their own MenhirLib runtime
for their own parsers.
I first tried to implement a solution to this module-name-conflict
issue using module aliases and -open, but this proven too fragile and
too difficult to get right.
Uses the new $symbolstartpos feature of Menhir
to get locations identical to the OCamlYacc ones.
REBASE POINT: at the point of this commit, using a diff program
on parser.mly and parser_menhir.mlyp should give identical results
after the header code. If you rebase the Menhir-parser patchset
against a newer ocamlyacc parser (parser.mly), those two files
will have diverged, and you need to merge the parser.mly change
back into parser_menhir.mlyp -- and then deal with them in the
rest of the patch series.
- inline Pervasives in Stdlib and re-add Pervasives as a deprecated
module that aliases all elements of Stdlib except the stdlib modules.
- remove special case for Stdlib.Pervasives in printtyp.ml
Instead of the current print to stderr. This way it's treated the same
as other warnings: it has a position, colors, can be made an error,
disabled, goes in the expected formatter, is documented.
It has been deprecated since 2000, shown a deprecation warning
since 4.02, and Sort.merge is documented to have undefined behavior
when the lists being merged are not sorted in the first place.
There seem to be occasions when rebuilding PRs where the TRAVIS_COMMIT
variable (which should be the SHA of the merge commit) is out-of-date.
When this happens, and the commit referred to does not exist, update
TRAVIS_COMMIT to be FETCH_HEAD as retrieved from GitHub instead.
Travis was incorrectly reading the files to run through check-typo from
just the tip commit of the PR branch, instead of the entire commit
range (wrong parameter number).
When checking an entire branch, check-typo was reading files from the
tip of the pull request branch, which is only correct if the branch is
based on trunk - it should read from the merge commit instead.
This affects branches based on older trunk before check-typo compliance
was enforced, since it can cause now-corrected check-typo violations to
re-appear.
TRAVIS_COMMIT_RANGE is not correct unless a PR is based on the tip of
the target branch - it will include commits being merged from the target
branch as well.
The check-typo, changes and tests builds now use
$TRAVIS_BRANCH..$TRAVIS_PULL_REQUEST_SHA which gives the precise range
of commits included in the pull request (i.e. the author's) only.
Three alterations to tools/check-typo:
1. Binary check may be on any commit where the default is HEAD
2. .gitattributes may be read from any commit, rather than just the
working tree
3. Detection of files under version control may be relative to a
specific commit, rather than relying on git ls-files
- Introduce `-dcamlprimc`, to keep the generated C file containing the primitive list
- Use `-fdebug-prefix-map` for compiling temporary C files when this option is supported
This commit renames a few C compiler related build variables so that
they are reserved for the build system. They will then be re-introduced,
but this time as user varialbes whose value can be freely customized
when compiling the package, without risking to conflict with those
command-line flags that are required by the build system itself.
Here are the variables this commit renames:
- CFLAGS -> OC_CFLAGS
- CPPFLAGS -> OC_CPPFLAGS
- LDFLAGS -> OC_LDFLAGS
Note: before this commit the compilation of scheduler.c in
otherlibs/threads was relying on make's implicit rule to compile C files.
Since this commit stops using the standard variables for flags,
it is necessary to introduce an explicit rule to compile C files
and that makes use of the newly introduced variables.
Some makefiles (lex, stdlib, otherlibs) would only offer allopt, while
others (ocamldoc, tools) only offered opt.opt. It is inconvenient to
have to remember which target name to use while going through various
repositories.
Now that the CI scripts have been moved also on the 4.07 branch and that the
Jenkins jobs have been updated to call the scripts from their new locations,
get rid of the legacy ones.
This tool is used to compile ocamlc. Given that make_opcodes is itself
recompiled as part of the bootstrap process, it needs to be run on the new
(rather than old) runtime. In other words, make_opcodes needs to be run
using byterun/ocamlrun rather than boot/ocamlrun.
This commit fixes this.
This CI script builds and tests the system using Clang and GCC
"sanitizers" to perform additional run-time checks on the C code,
e.g. out-of-bound memory accesses.
Currently, we use the Address, Undefined Behavior and Thread sanitizers.
The Memory sanitizer (reads from uninitialized memory) doesn't quite
work and is commented out in the script.
This commit stops enabling the instrumented and debug runtime at
configure time. Given that they are now enabled by default this is
no longer necessary.
In environments where the executables compiled to native code,
such as ocamlopt.opt, are always used in preference to the bytecode
versions then space can be saved by not installing the latter.
This patch provides a configure option to do such. It is relatively lightly
engineered; in particular, it won't complain if the native code executables
aren't themselves being built; but given this is an option for knowledgeable
users we think that it is reasonable.
The proposed behavior of `-config-var s` is as follows:
- if `s` is an existing configuration variable, print its value as
a string and exit with a success return value (0)
- if `s` is not an existing configuration variable, print nothing
and exit with a failure return value (non-0)
Note that we do not print a newline after the value of the
configuration variable. In particular, if the value is an empty
string, the output is undistinguishable from the output for
non-existing variables, the return value has to be considered instead.
The following alternative behaviors were considered:
- We could print a newline after the configuration value, which
would let users distinguish empty values from non-existing variables
by counting the lines of output, and would also be more pleasant for
users invoking the option from the command-line. However, the way
bash works on Windows means that $(ocamlc -config-var foo) would keep
a trailing \r in its output, and portable scripts would have to use
$(ocamlc -config-var foo | tr -d '\r') instead, which is a pain.
(This issue was pointed out by David Allsopp)
- We could print a message on the error output if the configuration
variable does not exist. This is clearer to a human user, but it is
annoying for scripts if they forget to silence the error output and
get their output mixed with our error messages. The main use of this
new feature is for scripting purposes.
bash considers sequences of integers starting with zero to be octal, so e.g. a
$minor value of "08" will be taken as an (illegal) octal number, causing an
error.
To fix this, we drop any leading zero when extracting the minor version.
Instead of printing no CRC at all, this option should rather preserve the
distinction between no CRC at all (represented by a dummy CRC
consisting of 32 '-' characters) and a real CRC represented by a
null CRC consisting of 32 '0' characters.
This commit thus renames the option from -no-crc to -null-crc.
I can observe weird performance bottlenecks on my machine caused by
the use of 'cp' in the 'install' scripts of OCaml. When installing
into a directory that is already populated by an existing
installation, 'make install' can routinely take 10s on my machine¹. After this
change it reliably takes 1.5s, independently of whether the
destination is already populated or not.
¹: a brtfs filesystem on an old-ish SSD
Why I care
----------
An extra 10s delay due to 'make install' can be noticeable in tight
change-build-install-test feedback loops for a compiler change where
we change the compiler, have a fast 'make world.opt' due to
incremental builds, install the change and test it -- possibly after
installing a couple opam packages, which can be fairly quick.
Partial diagnosis
-----------------
The performance issue seems to be caused by the fact that 'cp' (at
least the GNU coreutils version), when the file already exists,
replaces it by opening it in writeonly+truncate mode and writing the
file content ('strace' shows that the delay is caused within an
'openat' call). In particular, using the --remove-destination option
(which changes 'cp' to just remove the destination file before
copying) removes the performance issue, but this option seems missing
from the BSD/OSX 'cp' so it could cause portability issue.
Change
------
The present commit rewrites the 'install' targets of all Makefiles to
use the 'install' command instead. 'install' by default gives
executable-like permission to the destination file, instead of reusing
the source file's permissions, so we specify manually the permission
modes, depending on whether the installed file is an executable (or
dynamically-linked library) or just data (including other compiled
object files).
Testing
-------
I checked manually that the permissions of the installed files are
identical to the ones of the current 'cp'-using targets, except for
some '.mli' file in middle_end which currently have +x bits enabled
for no good reason.
Remark: To test this, playing with the DESTDIR variable is very useful
(this lets you install to a new directory (or the same as before)
without having to re-run the configure script). I used the following,
fairly slow shell script to collect permissions:
for f in $(find $DESTDIR); do \
echo $(basename $f) $(ls -l $f | cut -d' ' -f1); \
done | sort
Remark: it is important to run `sync` in-between 'make install' runs
to avoid timing effects due to filesystem or disk caching
strategies. I believe that this corresponds to the natural time delay
(and unrelated disk activity) that would occur in realistic
change-install-test feedback loops.
This commit contains two improvements to the cleanup done by ci-build on
Windows machines:
1. When a task is killed, also kill its subtasks (taskkill's /t option).
2. Add ocamltest.byte and ocamltest.opt to the list of tasks to kill.
These options allow to control whether identifiers are made unique by
appending a stamp to them when dumping intermediate representations or not.
The default is to print the stamp, as is done currently.
The "-dno-unique-ids" option is useful e.g. to simplify the comparison
between a produced intermediate reprsentation (-dlambda, say) and the
expected one, in the context of the testsuite, for instance.
Add a new warning non-ascii-utf8 displayed only if the non-ascii
attribute is specified and UTF-8 characters were ignored in the
copyright or authors lines in the header.
Inria's CI uses an "other-configs" job to test various, less
widespread configurations of the OCaml compiler.
Before this commit, the list of configurations to test was part of the
configuration of the Jenkins job itself, making it impossible to
have branch-specific configurations.
This commit introduces the tools/ci-build-other-configs script, which
can vary from branch to branch.
The job's configuration has been updated to execute the script when
it exists and to fallback to the former, built-in list of
configurations on the branches where this script is not present.
Install tools/read_cmt as ocamlcmt
* Add option -save-cmt-info to add more info in .annot files
* Provide more annotations
* Add option -I <dir> to find .cmi files
This commit moves:
- config/m.h to byterun/caml/m.h
- config/s.h to byterun/caml/s.h
Consequently, m.h and s.h now get installed alongside other
OCaml header files.
This commit also updates the .depend files, introducing updates in the
dependencies which are not consequences of this commit itself.
This commit ensures the install directory will be distinct for each build,
with the two following motivations:
1. If required, it will make it possible to have several builds running
concurrently on the same slave without conflicts between their make install
2. So far, if for any reason make install fails, then the build script
gets stopped and the install directory will not be removed.
Thus if a future build tries to install to the same directory, its
installation could in principle fail, not because it is broken but because
the previous build has left the install directory in a corrupted state.
So this commit ensures that make install will be done in an empty
directory.
The instdir environment variable is used by ci-build to define where
make install should install the compiler. Before this commit, a value was
assigned to instdir for the Windows builds that do not use configure,
but then this value was not taken into account, making the assignments useless
for those builds that do not use configure.
This commit fixes this by using instdir to override the value of PREFIX
in config/Makefile.
This commit sets the installation directory for the Windows build to the
same value as in the config/Makefile.m* files.
So, in itself this commit is a no-op. It however introduces the next one
which will ensure ci-build's instdir setting is honored even for
the builds that do not use configure.
This is done by passing the -with-debug-runtime option to the configure
script on the non-Windows systems.
On Windows, a sed invocation is used to set RUNTIMED to true in
config/Makefile.
So far, the cleanup of processes on the windows nodes of Inria's CI was
done in a .bat file called by the Windows jobs.
This commit integrates this process cleanup into ci-build.
The install-flexdll target now puts the object files for FlexDLL in a
subdirectory flexdll of the Standard Library instead of in the Standard
Library itself.
A configuration tweak means that -I +flexdll is effectively added to all
compiler invocations and also a pseudo-option -L+flexdll to ocamlmklib
calls to Config.mkdll which fixes PR#7373.
Before this commit there was BYTECCRPATH and NATIVECCRPATH, but they
were actually identical to each other.
This commit gets rid of them and uses the RPATH variable.
The code in this commit was written by Marcell Fischbach & Benedikt Meurer.
See [Mantis#5324](http://caml.inria.fr/mantis/view.php?id=5324) for some
context.
The code (which was originally written against 3.12) was ported to trunk by
doing
```bash
git clone https://github.com/bmeurer/ocaml-experimental/
cd ocaml-experimental
git diff master...linear-scan-register-allocator > t.diff
```
and then applying the diff by hand.
Run 'make lintapidiff' in the root of a git checkout to get a list of
potentially missing or wrong @since annotations.
The tool is not built by default, you have to first run 'make
world.opt', and then run 'make lintapidiff'.
lintapidiff doesn't support stop comments: add explicit list of changes to ignore.
see copyright header for license.
ocamlyacc doesn't correctly handle \r on Unix meaning that long strings
with escaped newlines cause unterminated string warnings.
Similarly, the make_opcodes script in the build system cannot cope with
a CRLF checkout of the header files.
* Check that all the required modules are provided
Also remove provided cmo from the required globals
* Remove required globals from packs when provided
* Remove provided globals after adding required ones
Pack modules can require globals they provide.
* Dumpobj tool can print relocation information
The parse_expand function does the same as parse and parse_dynamic
but allows Expand. However it does not update current to avoid
the problem that the array is modified.
This function is used for the ocamlopt, ocamlc, ocamlcp,
ocamloptp and ocamldep.
The read_aux function trims the cr character for the new-line
based method and the write functions are simplified without an
additional sep function.
The argument names are changed to match the default naming format
and the documentation is updated to match the implementation
behavior.
Also two new tests for reading and writing empty command lines as
well as a command lines containing empty arguments are added.
In essence, this commit does three things:
1. Move the content of Makefile.shared to the main Makefile
2. Modify the include directive in Makefile.nt accordingly
3. Move the Windows-specific definition of CAMLOPT from Makefile.nt
to Makefile and make sure it is evaluated only under Windows.
Adds the required_globals information to bytecode compilation units.
This patch also bootstrap ocamlc. The cmo format is changed by this
commit, there is no way around bootstraping here. Note that ocamldep and
ocamllex does not rely on the cmo format, so they are not present in
this commit.
Changes in tests:
* Update test/transprim/comparison_table.ml.reference:
The (opaque (global List!)) expression is not present anymore
* Update tests/no-alias-deps/aliases.cmo.reference
The output of objinfo changed
In order to remove some redundancy, the Pparse modules used a dirty
Obj.magic trick to manipulate either structure or signature values
(ASTs parsed from source files). This unsafe approach means that
programming mistakes may result in broken type soudness, and indeed
I myself had to track a very puzzling Segfault during the development
of my Menhir backend (due to a copy-paste error I was passing
Parse.implementation instead of Parse.interface somewhere). Wondering
why your parser generator seems to generate segfaulting parsers is
Not Fun.
This change means that the external interface of Pparse has changed
a bit. There is no way to fulfill the type of Pparse.file in
a type-safe way
val file : formatter -> tool_name:string -> string ->
(Lexing.lexbuf -> 'a) -> string -> 'a
as it promises to be able to process any ast type 'a depending on the
magic number (the last string argument). The knew type-safe interface is
val file : formatter -> tool_name:string -> string ->
(Lexing.lexbuf -> 'a) -> 'a ast_kind -> 'a
where ['a ast_kind] is a GADT type:
type 'a ast_kind =
| Structure : Parsetree.structure ast_kind
| Signature : Parsetree.signature ast_kind
cmpbyt was returning 2 to report both an error in the way it is
called (wrong number of command-line arguments) and the fact that
its two arguments are different programs. This patch fixes this and makes
cmpbyt's behaviour similar to the one of the cmp Unix command, i.e.
returning 0 to report file similarity, 1 to report file difference
and 2 for real errors.
Previously, `ocamlc`, `ocamlopt`, `ocamllex`, and `ocamldep` defaulted
to the bytecode versions of the tools. However, there is normally no
advantage to the bytecode versions if the native-code versions are
available, except in unusual circumstances. Instead, install the
native-code versions without the `.opt` suffix. They are still
installed with the `.opt` suffix for backwards compatibility. Finally,
install the bytecode versions with the `.byte` suffix, and build
native-code versions of tools that are not currently built in native
code.
Also, remove some duplication in tools/Makefile.shared.
Supersedes GPR #512.
Script intended to make the compilation instructions for the Microsoft
Visual C++ Compiler ports a little simpler.
This script allows the user to start Cygwin from the appropriate Visual
Studio Command Prompt (e.g. by issuing `C:\cygwin\bin\mintty -`) and then,
from the OCaml sources directory, issue
```bash
eval `tools/msvs-promote-path`
```
This script emits an error if the Microsoft Linker cannot be found in any
of the directories in `PATH` or if the `link` command already refers to the
Microsoft Linker. If, as will usually be the default, `link` refers to the
coreutils `link` command, `PATH` is altered to move the first entry
containing the Microsoft Linker to the start of `PATH`. This has the
effect of overriding the Unix interpretation of the `link` command, but
this will never affect OCaml's build process.
This script is taken from msvs-tools
(https://github.com/metastack/msvs-tools) and re-licensed by the copyright
holder.
This allows external tools to rely on the features provided by this module. An example
is ocamldoc itself, which could now be implemented as an external tool.
Note that LexiFi variants of OCaml has been embedding depend.ml in the compiler for a long
time in order to support a stricter dependency mode where the compiler is only allowed to load .cmi
files corresponding to dependencies as inferred by ocamldep (hereby ensuring that ocamldep is
sound by construction).
This introduces a tiny change of behavior since command line options which affect the
printing of dependencies (-modules, -all, -ml-synonym, -mlfi-synonym) are taken into
account even for files mentioned before on the command-line. This is probably not going to break
anything.
Note: this commit also reverse the ordering of the output, but we will normalize it in the next
commit anyway.
This module checks all the AST invariants. This is to ensure that all
invariants are written down in one place and are consistently checked
between the various clients of the AST (typer, pprintast, ...).
The invariants are checked in Pparsee, after applying the ppx
rewriters.
The formatters used for printing warnings have text mode translation
enabled which means that any Windows endings which creep into warning
texts (e.g. from attributes) result in \r\r\n in the output.
The effect is largely innocuous, except that it causes the
deprecated_module_use test to fail on Windows.
Extend the previous patch allowing make -f Makefile.nt flexdll
install-flexdll not to require the install-flexdll stage.
OCAML_FLEXLINK is utilised as required to allow compilation of the entire
system using an in-tree compiled flexlink. The build process simply
required the flexdll target to appear before world.
opt.opt compiles a native code version of flexlink.exe as flexlink.opt.
install always installs flexlink.exe if it was compiled along with any
required .manifest files. It also installs the appropriate .o/.obj files
to $(INSTALL_LIBDIR).
At present, the bootstrapping is not extended to the Cygwin ports.
OCAML_FLEXLINK is inspected by ocamlopt, ocamlc and ocamlmklib and allows
the flexlink command to be overriden. This is primarily intended as a
solution for bootstrapping OCaml with FlexDLL, thus allowing a bytecode
image of flexlink launched with ocamlrun to be executed, instead of
requiring flexlink to be in PATH.
I'm not familiar enough with Subversion to know how properties work
there, but the translation for detecting binary files to git using
attributes is incorrect. Checking for the binary attribute with
check-attr only detects if the file has been *specified* as binary.
Script altered to use a "well-known" plumbing command to detect whether
git regards a particular file as binary. This takes into account
.gitattributes, but also works if the file has been automatically
detected as binary by git.
Runhang Li
Leo White
Alain Frisch
Gabriel Scherer
Daniel Bünzli
Sébastien Hinderer
Armaël Guéneau
David Allsopp
Thomas Refis
Damien Doligez
Mark Shinwell
Nicolás Ojeda Bär
Florian Angeletti
Jérémie Dimino
Hugo Heuzard
Xavier Clerc
sliquister
François Bobot
Gabriel Radanne
Thomas Refis
whitequark
poechsel
Dwight Guth
Xavier Leroy
Pierre Chambart
Fabrice Le Fessant
Fabrice Le Fessant
Stephen Dolan
Török Edwin
Xavier Leroy
Stephane Glondu
aalekseyev
hendriktews
Bernhard Schommer
Andreas Hauptmann
Mark Shinwell
Pierre Chambart
Demi Obenour
Leo White
Christophe Troestler
Nicolas Ojeda Bar
Jeremie Dimino
Damien Doligez