MPR#7672: add a Filename.quote_command function
This function takes care of quoting the command and its arguments
so that they are correctly parsed by the system shell
(/bin/sh for Unix, cmd.exe for Win32).
Redirections for std input (< file) and std output (> file) and
std error (2> file) can also be specified as optional arguments.
A 2>&1 redirection is used if stdout and stderr are redirected to the
same file.
The result is a string that can be passed directly to Sys.command or
to the Unix functions that expect shell command lines.
Two bugs were present before the patch:
- Formatting_gen would always be printed as "@{",
so for example "@[foo@]" would be reprinted as "@{foo@]"
- The Formatting_gen payload would be printed as a string literal,
escaping '%', while it is a raw string representation of a format;
so for example "@[<%d>" would be reprinted as "@[<%%d>"
(This second bug was spotted by Florian Angeletti)
- Currently the check is a no-op, because the "emit a warning if the
check fails" was never implemented. (It would actually require some
work to pass a source location there to emit a warning, so it's not
trivial.)
- The check is implemented by calling `open_box_of_string` and
catching the `Failure _` exception if that function fails. This is
just wrong: `Failure _` should be reserved to fatal program errors,
and should not be caught for control-flow.
- The current implement is buggy (it fails all the time, but we don't
notice because no warning is emitted):
CamlinternalFormat.open_box_of_string expects a string of the form
"v 3", but check_open_box would pass a string of the form "<v 3>"
(or an empty string), which is the payload of the format value. So
the check always fails.
- The idea of the check is wrong: "@[<x>foo@]" is an incorrect format
string to pass to Format (the box indication does not make sense),
but it is a perfectly fine format string to pass to Printf, where it
just prints "@[<x>foo@]" on the output. So we cannot complain to the
user at type-checking time, when we don't know how the format string
will be used, whether the boxes will be interpreted as actual boxes
or string literals.
This makes sure that:
- Callbacks are never called when another is running
- The postponed queue is purged when setting memprof parameters
We now use a FIFO implemented as a circular buffer for remembering of
postponed blocks.
The workaround used for ignoring samples in the minor heap in native
mode now makes allocation very slow (or non-terminating) when the
sampling rate is not small enough. This will be fixed when sampling in
the minor heap in native mode will be implemented.
Allocations ignored by this version
- Marshalling
- In the minor heap by natively-compiled OCaml code
Allocations potentially sampled
- In the major heap
- In the minor heap by C code and OCaml code in bytecode mode
If an allocation fails, the decrement of young_ptr should be undone
before the GC is entered. This happened correctly on bytecode but not
on native code.
This commit (squash of pull request #8619) fixes it for all the
platforms supported by ocamlopt.
amd64: add alternate entry points caml_call_gc{1,2,3} for code size
optimisation.
powerpc: introduce one GC call point per allocation size per function.
Each call point corrects the allocation pointer r31 before calling
caml_call_gc.
i386, arm, arm64, s390x: update the allocation pointer after the
conditional branch to the GC, not before.
arm64: simplify the code generator: Ialloc can assume that less than
0x1_0000 bytes are allocated, since the max allocation size for the
minor heap is less than that.
This is a partial cherry-pick of commit 8ceec on multicore.
A #! line should not exceed 128 characters (including the \0
terminator). This adds a test - both to the generation of the camlheader
files and also to the -use-runtime flag which falls back to #!/bin/sh
and uses exec to invoke the the interpreter.
fixes#7156
Two new functions exposed:
```ocaml
val to_string_default: exn -> string
(** [Printexc.to_string_default e] returns a string representation of the
exception [e], ignoring all registered exception printers.
@since 4.09
*)
val use_printers: exn -> string option
(** [Printexc.use_printers e] returns [None] if there are no registered
printers and [Some s] with else as the resulting string otherwise.
@since 4.09
*)
```
When running a script with "ocaml foo.ml", the toplevel needs to
run foo.ml with a different Sys.argv than the initial value, since
foo.ml must not see the initial "ocaml" argument.
Previously, this was done with Obj.truncate to shorten the Sys.argv
array. This patch changes it by introducing a primitive %sys_argv.
Uses of this primitive expand to a call to a new C primitive, which
returns the argv array (and can be modified by the toplevel).
CamlinternalMod contains an optimisation for the initialisation
of recursive modules containing closures, where dummy closures
are updated in-place. This optimisation was buggy on bytecode,
since the bytecode interpreter relies on the lengths of blocks
containing closures (see #4008).
This commit disables the optimisation for bytecode (where it
had much less effect than on native code, and where performance
is of less concern anyway). The optimisation is still applied
on native-code, but without the use of Obj.truncate.
Also adds a test for #4008 (which introduced the truncate).
Hashtbl.MakeSeeded.{add,replace}_seq were not using the hash function
provided by the functor (Hashtbl.MakeSeeded.of_seq uses replace_seq and
so also has to be redefined locally).
This commit removes support for gprof-based profiling (the -p option to ocamlopt). It follows a discussion on the core developers' list, which indicated that removing gprof support was a reasonable thing to do. The rationale is that there are better easy-to-use profilers out there now, such as perf for Linux and Instruments on macOS; and the gprof support has always been patchy across targets. We save a whole build of the runtime and simplify some other parts of the codebase by removing it.
Note: Typos found with https://github.com/codespell-project/codespell
Here is the (semi-manual) command used to get (and correct) the typos:
$ codespell -i 3 -w --skip=".png,.gif,./ocaml/boot,./ocaml/.git,./ocaml/manual/styles,./ocaml/manual/manual/htmlman" -L minimise,instal,contructor,"o'caml",cristal,pres,clos,cmo,uint,iff,te,objext,nto,nd,mut,upto,larg,exten,leage,mthod,delte,tim,atleast,langage,hten,iwth,mke,contant,succint,methids,eles,valu,clas,modul,que,classe,missings,froms,defaut,correspondance,differents,configury,reachs,cas,approche,normale,dur,millon,amin,oje,transfert
- Treat as an error the case where ~finally raises an exception
- Move to Fun module
- Describe the purpose in the documentation
- Remove boilerplate
https://github.com/ocaml/ocaml/pull/2118
This change should be a refactoring no-op.
Before, a DEPFLAGS variable existed in some makefiles to contain
include directories to be passed to ocamldep invocations, but no
support for easily adding command-line flags to ocamldep was available
(invocations would systematically use -slash, which was duplicated
across callsites).
With this PR, a new DEPINCLUDES variable contains the include
directories, and DEPFLAGS is repurposed to contain other command-line
flags for the tool -- currently "slash".
Without this patch, make -j often fails to build the stdlib with
a message along the lines of:
no cmx file was found in path for module CamlinternalLazy
The issue is that stdlib files that use `lazy` actually depend on
camlinternalLazy.cmi because matching.ml expands lazy pattern
matches to code that refers to CamlinternalLazy. However, since
this dependency does not appear in the source code, there is no
way for ocamldep to discover it. This means that when building
the stdlib, there is no constraint ensuring that CamlinternalLazy
is built before stdlib modules using Lazy.
This causes issues with parallel make, but the issue can be
reproduced using a sequential make invocation:
cd stdlib
make clean
make stdlib_stream.cmo
This patch adds a dependency on CamlinternalLazy into lazy.mli.
Its presence makes ocamldep see that all files that use Lazy also
depend on camlinternalLazy.cmi.
from Frédéric Bour (@let-def)
from Mark Shinwell (@mshinwell)
particularly:
- in *_copy, avoid an infinite loop by triggering a minor collection
after 8 rounds. But since truncation and tag setting will be
deprecated we could soon remove this code.
* Add an Extension_constructor submodule to Obj.
Deprecate top-level functions extension_constructor / extension_name /
extension_id.
* Add 'true' and 'false' to the definition of Bool.t
* Add aliases for the built-in 'list' and 'array' types.
* Add an alias for 'exn' to Printexc.
* Changes entry: built-in type aliases
* Add a Unit module.
* Add paths for built-in exceptions.
Adds a fused multiply-add operation to the Float module.
The following changes are made:
- configure: check math.h for the C99 fma() operation.
- fma declarations in float.ml[i] (stdlib/).
- C fma() call or emulation in runtime/floats.c.
- dedicated tests in testsuite/tests/fma.
In order to prepare the transition to autoconf, this commit moves the
configuration Makefile out of the config directory which will disappear
and gives it the name it will have once intstalled, namely Makefile.config.
- inline Pervasives in Stdlib and re-add Pervasives as a deprecated
module that aliases all elements of Stdlib except the stdlib modules.
- remove special case for Stdlib.Pervasives in printtyp.ml
* Move the description of these literals to refman/const.etex and
refman/lex.etex
* Mention their existence in library/builtins.etex
* Add examples to Int32, Int64 and NativeInt module documentation
Function `pp_skip_token` had a comment saying that the queue is never empty
when it is called. However, I came up with a unit test that falsifies that
invariant, see `pp_skip_token.ml`. This prompted to change `pp_skip_token` to
gracefully handle the case when the queue is empty. Before, the invariant was
wrong, but the code still worked correctly because the exception would have
been caught in `advance_left` which used to use `Queue.Empty` exception for
control flow.
This reduces the volume of code in Format and adds clarity by using a
familiar Stack interface. However, this introduces a dependency from
Format to Stack.
This reduces the volume of code in Format and adds clarity by using a
familiar Queue interface. However, this introduces a dependency from
Format to Queue.
Note, `state.pp_queue` was unnecessarily mutable.
Use proper abstract module instead of "%identity" trick.
Add Size.unknown and Size.is_unknown to aid readability
instead of relying on a comment saying that negative
sizes mean unkown sizes.
As a result of the poll in #1855.
The votes at the time of commiting are:
- 18 for `protect ~finally`
- 12 for `try_finally ~finally`
- 1 for `finally ~cleanup`
- 0 for `try_finally ~always`
- 0 for `try_finally ~cleanup`
Signed-off-by: Marcello Seri <marcello.seri@gmail.com>
`try_finally work ~always ~exceptionally` is designed to run code
in `work` that may fail with an exception, and has two kind of
cleanup routines:
- `always`, that must be run after **any** execution of the function
(typically, freeing system resources), and
- `exceptionally`, that should be run **only** if `work` or `always`
failed with an exception (typically, undoing user-visible state
changes that would only make sense if the function completes
correctly).
I had to locally re-define `rab_backtrace`, `get_raw_backtrace`,
`raise_with_backtrace` because I could not refer to `Printexc`.
Signed-off-by: Marcello Seri <marcello.seri@gmail.com>
When -a is defined as Unit, accepted -a=1.
When -a is defined as Tuple, accepted -a=1 b c as -a b=1 c=1.
When -a is defined as Rest, looped infinitely on -a=1.
It has been deprecated since 2000, shown a deprecation warning
since 4.02, and Sort.merge is documented to have undefined behavior
when the lists being merged are not sorted in the first place.
Reported at https://caml.inria.fr/mantis/view.php?id=7820
Array.of_seq applies a circular permutation of one cell to the right
on the sequence.
With OCaml 4.07.0 and trunk, we have
- : int array = [|3; 1; 2|]
In stdlib/array.ml, line 337 (last line of of_rev_list), we have
fill (len-1) tl
whereas it should be
fill (len-2) tl
since hd, which should be assigned to the cell (len - 1), is skipped.
This commit renames a few C compiler related build variables so that
they are reserved for the build system. They will then be re-introduced,
but this time as user varialbes whose value can be freely customized
when compiling the package, without risking to conflict with those
command-line flags that are required by the build system itself.
Here are the variables this commit renames:
- CFLAGS -> OC_CFLAGS
- CPPFLAGS -> OC_CPPFLAGS
- LDFLAGS -> OC_LDFLAGS
Note: before this commit the compilation of scheduler.c in
otherlibs/threads was relying on make's implicit rule to compile C files.
Since this commit stops using the standard variables for flags,
it is necessary to introduce an explicit rule to compile C files
and that makes use of the newly introduced variables.
Some makefiles (lex, stdlib, otherlibs) would only offer allopt, while
others (ocamldoc, tools) only offered opt.opt. It is inconvenient to
have to remember which target name to use while going through various
repositories.
Fixes: MPR#7253 (in large part), MPR#7796 (in full).
As shown in the PRs above there are several cases where do_at_exit
is called several times, causing functions registered with at_exit
to be called several times. Also, an at_exit function that raises
could prevent other at_exit functions from being run.
This commit doesn't try to prevent multiple calls to do_at_exit,
but makes sure that each function registered with at_exit is run
at most once. The idea is due to Nicolás Ojeda Bär.
* Uniformise documentation of `_opt` functions in Pervasives
The `option`-returning versions now document the behaviour of the
functions and are refered to by the raising versions using the following
pattern:
> Same as {!Pervasives.f_opt}, but raise [x] instead of returning [None].
* Document function `read_float` raising an exception
1. Add Bigarray lines to library/stdlib.etex
2. Move intro from libbigarray.etex to stdlib/bigarray.mli
3. Note that 0-dimensional arrays are supported.
4. Move C interface description to cmds/intf-c.etex
5. Change wording in libbigarray.etex to reflect legacy status.
6. Add a label to libunix.etex (needed for link from libbigarray.etex)
7. Put the changes in Changes (for 4.07).
When AFL instrumentation is enabled, the inlining of Lazy.force
is disabled, so that the GC optimisation of removing Forward_tag
blocks is no longer visible in the instrumentation output.
Calls to raise in Format can accidentally appear in backtraces,
especially when formatting an error message after catching an exception.
By using raise_notrace, the backtrace is no longer clobbered.
I can observe weird performance bottlenecks on my machine caused by
the use of 'cp' in the 'install' scripts of OCaml. When installing
into a directory that is already populated by an existing
installation, 'make install' can routinely take 10s on my machine¹. After this
change it reliably takes 1.5s, independently of whether the
destination is already populated or not.
¹: a brtfs filesystem on an old-ish SSD
Why I care
----------
An extra 10s delay due to 'make install' can be noticeable in tight
change-build-install-test feedback loops for a compiler change where
we change the compiler, have a fast 'make world.opt' due to
incremental builds, install the change and test it -- possibly after
installing a couple opam packages, which can be fairly quick.
Partial diagnosis
-----------------
The performance issue seems to be caused by the fact that 'cp' (at
least the GNU coreutils version), when the file already exists,
replaces it by opening it in writeonly+truncate mode and writing the
file content ('strace' shows that the delay is caused within an
'openat' call). In particular, using the --remove-destination option
(which changes 'cp' to just remove the destination file before
copying) removes the performance issue, but this option seems missing
from the BSD/OSX 'cp' so it could cause portability issue.
Change
------
The present commit rewrites the 'install' targets of all Makefiles to
use the 'install' command instead. 'install' by default gives
executable-like permission to the destination file, instead of reusing
the source file's permissions, so we specify manually the permission
modes, depending on whether the installed file is an executable (or
dynamically-linked library) or just data (including other compiled
object files).
Testing
-------
I checked manually that the permissions of the installed files are
identical to the ones of the current 'cp'-using targets, except for
some '.mli' file in middle_end which currently have +x bits enabled
for no good reason.
Remark: To test this, playing with the DESTDIR variable is very useful
(this lets you install to a new directory (or the same as before)
without having to re-run the configure script). I used the following,
fairly slow shell script to collect permissions:
for f in $(find $DESTDIR); do \
echo $(basename $f) $(ls -l $f | cut -d' ' -f1); \
done | sort
Remark: it is important to run `sync` in-between 'make install' runs
to avoid timing effects due to filesystem or disk caching
strategies. I believe that this corresponds to the natural time delay
(and unrelated disk activity) that would occur in realistic
change-install-test feedback loops.
* add `Seq` module, expose iterator conversions in most containers
* small typo
* typo
* change order of arguments for `{Map,Set}.add_seq`
* watch for max string length in `Bytes.of_seq`
* wip: make it build again
* Fix dependency
Sys needs to be linked before Bytes in stdlib.
* Update threads/stdlib.ml
* Update stdlib_no_prefixed/.depend
* fix inconsistencies with label modules
* update testsuite to work with seq
* update change file
* small change in `Hashtbl.to_seq`, capturing only the underlying array
* add some documentation to seq.mli
* revert to good ol' module type names for hashtables
* fix test
* change style of comments in seq.mli
* follow some demands in review of GPR #1002
* some fixes for #1002
* add Seq-related functions to Ephemeron
* add some comments on `Hashtbl.of_seq`
* add more tests for `Hashtbl.{to,of}_seq`
* fix bug in `Ephemeron.to_seq`
* Update Changes
awk is symbolic link in Cygwin, which means it can't be used in -pp for
a native Windows build. Just use gawk instead, as no other package
provides the awk command on Cygwin.
Except for the Camlinternal* modules and the new Stdlib module, all
modules in the stdlib now compile to Stdlib__<module>.
Pervasives is renamed to Stdlib and now contains a list of aliases
from the long names to the short ones, so that from inside and outside
the stdlib we can refer to the standard modules as just List or
Stdlib.List rather than Stdlib__list.
In order to avoid printing the long names in error messages and in the
toplevel, the following heuristic is added to Printtyp: given a path
Foo__bar, if Foo.Bar exists and is a direct or indirect alias to
Foo__bar, then prefer Foo.Bar.
A bootstrap step was required to replace Pervasives by Stdlib as the
module opened by default.
Greta Yorsh
Leo White
David Allsopp
Guillaume Munch-Maccagnoni
Xavier Leroy
Thibault Suzanne
Stephen Dolan
Damien Doligez
Gabriel Scherer
Jacques-Henri Jourdan
JPR
Nicolás Ojeda Bär
Stephen Dolan
Sébastien Hinderer
Andrew Litteken
alainfrisch
Stephen Dolan
Mark Shinwell
thierry-martinez
Pierre Roux
Daniel Bünzli
Richard Bonichon
Florian Angeletti
Armaël Guéneau
Armaël Guéneau
Thomas Refis
Jeremy Yallop
Vladimir Keleshev
Thomas Refis
Hugo Heuzard
Damien Doligez
Pierre-Marie Pédrot
François Bobot
Laurent Thévenoux
Christophe Troestler
Etienne Millon
Foo Chuan Wei
Drup
Jeremie Dimino
TheAspiringHacker
Marcello Seri
Valentin Gatien-Baron
whitequark
Pierre Weis
ygrek
Martin Desharnais
Perry E. Metzger
Frédéric Bour
zapashcanon
Simon Cruanes
Max Mouratov
Milo Davis
Jeremie Dimino
Marc Chevalier
Jacques Garrigue