If a test fails or is skipped, then ocamltest skips all the tests below
it. This makes the output of the testsuite a little hard to interpret.
Alter the output so n/a is displayed for the tests underneath - this has
the effect of reducing the number of reported skipped tests back to
pre-ocamltest levels.
This variable contains a command that can be used to invoke make.
Its initial value is the one of the MAKE variable in the syste
environment of ocamltest. If this value is undefined, it defaults to "make".
If you forget to commit a test file with your patch (but include
it in ocamltests), ocamltest will fail (print a message to
standard error and return a non-zero code), but until now it would
not print the standard-format "testing ... => unexpected error"
message which makes the failure visible to the summarize.awk
script.
This commit ensures that such early failures still result in an
explicit unexpected-error message being shown in the standard way,
in addition to stderr-level error reporting.
When -promote is set, running the test automatically overwrites each
reference output file with the actual output of the test. This option,
which mirrors the "make promote" target of old-style testsuite tests,
is very useful when a minor change in compiler/toplevel output affects
a lot of reference files in innocuous ways.
This is the only option of ocamltest that overwrites file in the
source directory, so it should be used with care -- under strict
version-control supervision.
The 'make promote' target is not yet supported for new-style testsuite
tests, as this feature depends on the not-yet-merged GPR#1574
https://github.com/ocaml/ocaml/pull/1574
For a previous discussions of this feature, see GPR#1519
https://github.com/ocaml/ocaml/pull/1519
This module contains types describing test results and functions to
build and use them.
Before this commit, only successful actions were returning an environemnt.
Starting from this commit, actions always return an environemnt, no matter
their result.
This will make it possible to write negations over tests.
This commit saves the current directory before processing a test file
and restores it afterwards.
This is necessary when several test files are given on the command line.
This commit contains:
- The initial version of the tool itself, in the ocamltest directory
- The required additions to the main .gitignore and .merlin files.
The integration of ocamltest in OCaml's main build system and its use
in the testsuite are not part of this commit.
Credits go to David Allsopp for
- Extending the computation of ocamlsrcdir to the Windows case
- Making ocamltest compile when flexlink
is being bootstrapped.
David Allsopp
Nicolás Ojeda Bär
Gabriel Scherer
Sébastien Hinderer
Gabriel Radanne