This commit deduplicates some of the Makefile rules used to compile C files.
Rather than having one such rule per Makefile for each directory
containing C files, the relevant rules are moved to Makefile.common.in.
Since we cannot access backtrace position in cmmgen.ml anymore,
Cmm.raise_kind in removed. Instead, we use Lambda.raise_kind. When
assembly code is generated, we reset the backtrace position to 0 in the
case of regular raise. Importantly, the semantics remains the same.
MPR#7672: add a Filename.quote_command function
This function takes care of quoting the command and its arguments
so that they are correctly parsed by the system shell
(/bin/sh for Unix, cmd.exe for Win32).
Redirections for std input (< file) and std output (> file) and
std error (2> file) can also be specified as optional arguments.
A 2>&1 redirection is used if stdout and stderr are redirected to the
same file.
The result is a string that can be passed directly to Sys.command or
to the Unix functions that expect shell command lines.
Dynlink does a weird dance of copying compiler files from various
places. The 'depend' target does not perform this copying, and instead
just 'touch'es the files that should be present to run -- defined in
DEPEND_DUMMY_FILES: if you run 'depend' after a build, you get the
real content of the files, but if you run without building you get
empty files.
This is (dubious but apparently) correct for most of those files which
should not generate dependencies anyway, but not for
dynlink_platform_intf.mli which contains important dependencies (as
in: removing them breaks the parallel build) and should be properly
built before 'depend' runs.
GPR#8691 refactored the signal/async callback system, and introduced
new data races to `caml_something_to_do` and friends. These data races
morally already existed and are "benign", in the sense that they can
only cause more checks for async callbacks.
The corresponding functions are now marked with a special attribute
for whitelisting them. We do no longer use -fsanitize-blacklist, which
seemed to fail preventing warnings (???).
This make us able to get rid of to xxx_to_do variables in `final.c`
and `memprof.c`. The variable is reset to 0 when entering
`caml_check_urgent_gc`, which is now the main entry point for
asynchronous callbacks. In case a callback raises an exception, we
need to set it back to 1 to make sure no callback is missed.
The finalizers and all the other asynchronous callbacks (including
signal handlers, memprof callbacks and finalizers) are now called in a
common function, [caml_async_callbacks]. It is called in
[caml_check_urgent_gc] and wherever [caml_process_pending_signals] was
called.
This makes it possible to simplify the [caml_gc_dispatch] logic by
removing the loop it contains, since it no longer calls finalizers.
Allocations ignored by this version
- Marshalling
- In the minor heap by natively-compiled OCaml code
Allocations potentially sampled
- In the major heap
- In the minor heap by C code and OCaml code in bytecode mode
This generic Makefile is parameterized by several "make" variables,
- two that must be set by the calling Makefile
- nine that can be set by the calling Makefile but have default values
otherwise.
This commit sets explicitly the default values of the nine variables above,
using "VAR ?= default-value", even if the default value is the empty string.
This avoids spurious warnings by "make --warn-undefined-variables"
and is good documentation-in-code.
* Various file moves in the middle end: this is the first stage of improving separation between the middle end and backend.
* Creation of file_formats/ directory (with associated file moves) to hold the definitions of compilation artifact formats.
* Creation of lambda/ directory (with associated file moves) to hold Lambda language definition files, transformation passes and construction passes from Typedtree.
* Disable (hopefully temporarily) dynlink, debugger and ocamldoc for the dune build.
The Graphics library is now distributed as a separate package.
The sources are at https://github.com/ocaml/graphics .
Signed-off-by: Jeremie Dimino <jeremie@dimino.org>
The compatibility macros (that were introduced while moving bigarray file mapping functions to the Unix library) are no longer necessary, so let's get rid of them.
Signed-off-by: Jeremie Dimino <jeremie@dimino.org>
After consultation on the core developers' list I am proposing this patch to remove support for compiler plugins.
The main motivations for removing compiler plugins are:
- They are a potential security risk.
- They increase the complexity of the build system and make maintenance of the Dynlink libraries more difficult (although actually, this complexity could probably be reduced after #2268 is merged).
- Many applications of plugins should be able to be expressed by building custom compiler drivers that link against compilerlibs.
* Remove compiler plugins and hooks
* Add new function Dynlink.unsafe_get_global_symbol but keep it outside the documented API.
* Remove otherlibs/dynlink/nodynlink.ml
* Update Changes
* Delete the deprecated vmthreads library
It was deprecated in 4.08.
* Remove the byte/native argument of init_path
It is no longer necessary.
* Error out when passing --{enable,disable}-vmthreads to ./configure
Signed-off-by: Jeremie Dimino <jeremie@dimino.org>
In OCaml 4.07, Unix.sleepf and Thread.delay were changed so that they
would restart the sleep when interrupted by a signal (error EINTR).
The unintended consequence is that Thread.delay will not run
signal handlers until the full delay has expired. If the effect
of the handler is to raise an exception, as with Sys.catch_break,
the delay is not terminated early.
(This is specific to threaded programs, where asynchronous invocation
of signal handlers is turned off and handlers are only run at the next
leave-blocking-section. Using Unix.sleepf in a non-threaded program
doesn't show the issue because the handler is invoked asynchronously.)
This commit implements a more intuitive behavior, closer to that of 4.06:
signals received during Thread.delay are handled immediately, and if
the handler returns normally, the delay is restarted with the remaining time.
A test is added in testsuite/tests/lib-threads/delayintr.ml
When `Unix.waitpid` returns immediately with pid = 0, make sure the status returned is always `WEXITED 0` and not some random value coming from an uninitialized C variable.
Fix Thread.sigmaks, by checking whether a signal is masked before handling it.
We use [sigprocmask] (if available) to check whether a signal is
blocked when the systhread library is not loaded. As soon as the
[Thread] module gets loaded, we use [pthread_sigmask] instead, and
redirect all the calls to [sigprocmask] to [pthread_sigmask]. Indeed,
the latter has unspecified behavior in a multi-threaded context
anyway. In practice, this should not change the semantics of
[Unix.sigprocmask] on Linux, since on this platform, [pthread_sigmask]
is actually an alias for [sigprocmask]. On MacOSX, the semantics will
change, since [sigprocmask] changes the masks of the whole process on
this platform.
Also, include [caml_pending_signals] in signals returned by
[Unix.sigpending]. Indeed, some signals might have been handled in the
POSIX sense by the C handler in the OCaml runtime, but not been
handled in the OCaml sense (for example, because they are blocked).
This commit un-reverts 1c82c481a, which has been reverted in
79eb572e4. The issues of the original commit are corrected in this commit.
Persistent_env is a new module that handles the relation between the
type-checking state and the "persistent" typing information laying in
.cmi files on the filesystem. In particular, it handles the collection
and production of CRC information for the .cmi files being read and
written to the filesystem; the using modules (in our case, only Env)
are in charge of turning the cmi files into higher-level information
(components and signatures).
Persistent_env exposes a type `'a t` of a persistent environment,
which acts as a mutable store of `'a` values. There is no global state
in the module itself: while Env (and thus the OCaml type-checker) uses
a single global persistent environment, it should be possible to
create several independent environments to represent, for example,
several independent type-checking sessions.
Note: Typos found with https://github.com/codespell-project/codespell
Here is the (semi-manual) command used to get (and correct) the typos:
$ codespell -i 3 -w --skip=".png,.gif,./ocaml/boot,./ocaml/.git,./ocaml/manual/styles,./ocaml/manual/manual/htmlman" -L minimise,instal,contructor,"o'caml",cristal,pres,clos,cmo,uint,iff,te,objext,nto,nd,mut,upto,larg,exten,leage,mthod,delte,tim,atleast,langage,hten,iwth,mke,contant,succint,methids,eles,valu,clas,modul,que,classe,missings,froms,defaut,correspondance,differents,configury,reachs,cas,approche,normale,dur,millon,amin,oje,transfert
Thread.yield invoked a trivial blocking section, which basically woke
up a competitor and then raced with them to get the ocaml lock back, invoking
nanosleep() to help guarantee that the yielder would lose the race. However,
until the yielder woke up again and attempted to take the ocaml lock, it
wouldn't be marked as a waiter.
As a result, if two threads A and B yielded to each other in a tight loop, A's
first yield would work well, but then B would execute 10000+ iterations of the
loop before A could mark itself as a waiter and be yielded to. This works even
worse if A and B are pinned to the same CPU, in which case A can't be marked as
a waiter until the kernel preempts B, which can take tens or hundreds of
milliseconds!
So we reimplement yield; instead of dropping the lock and taking it again (with
a wait in the middle), atomically wake a competitor and mark the yielding thread
as a waiter. (We essentially inline a failed masterlock_acquire into
masterlock_release, specialized for the case where we know another waiter exists
and we want them to run instead.)
Now, threads yielding to each other very consistently succeed--in that same
tight loop, we see a change of control on every iteration (with some very rare
exceptions, most likely from other uncommon blocking region invocations.)
This also means we don't have to worry about the vagaries of kernel scheduling
and whether or not a yielding or a yielded-to thread gets to run first; we
consistently let a competing thread run whenever we yield, which is what the API
claims to do.
- Treat as an error the case where ~finally raises an exception
- Move to Fun module
- Describe the purpose in the documentation
- Remove boilerplate
https://github.com/ocaml/ocaml/pull/2118
- Add a Load_path module which caches files lookup
- Instead of falling back to the external environment, allow to
declare in the environment that a module comes from the external
world. This allows persistent structures to shadows non-persistent
ones
We use [sigprocmask] (if available) to check whether a signal is blocked when
the systhread library is not loaded. When it gets loaded, we use
[pthread_sigmask] instead.
Also, include [caml_pending_signals] in signals returned by
[Unix.sigpending]. Indeed, some signals might have been handled in the
POSIX sense by the C handler in the OCaml runtime, but not been
handled in the OCaml sense (for example, because they are blocked).
I frequently get (parallel) build failures due to wrong dependencies
causing compilation failures in some otherlibs/ subdirectory. When
that happens, my workflow is to go to that subdirectory, run 'make
clean', and then compile again. But then another subdirectory is
broken for the same reason...
This otherlibs/Makefile provides a 'clean' target that will run clean
in all subdirectories (all the ones selected to be built at
configure-time), as well as 'all', 'allopt', 'partialclean' targets
that similarly dispatch to all subdirectories.
This is exactly the logic already present in the 'otherlibraries*'
targets of the root Makefile, except more fun with Make-style generic
functions.
This file is meant to be included by the per-directory makefiles
of otherlibs/*/Makefile; its target cannot be run directly from
otherlibs/. We would like to propose targets from otherlibs/,
in a new Makefile.
* Add an Extension_constructor submodule to Obj.
Deprecate top-level functions extension_constructor / extension_name /
extension_id.
* Add 'true' and 'false' to the definition of Bool.t
* Add aliases for the built-in 'list' and 'array' types.
* Add an alias for 'exn' to Printexc.
* Changes entry: built-in type aliases
* Add a Unit module.
* Add paths for built-in exceptions.
In order to prepare the transition to autoconf, this commit moves the
configuration Makefile out of the config directory which will disappear
and gives it the name it will have once intstalled, namely Makefile.config.
Some Makefiles were using export to set OCAML_FLEXLINK "globally" while
others set a variable FLEXLINK_ENV and set the environment explicitly.
All Makefiles now use FLEXLINK_ENV and also only invoke it on linking
commands (rather than, for example, all invocations of ocamlopt).
- inline Pervasives in Stdlib and re-add Pervasives as a deprecated
module that aliases all elements of Stdlib except the stdlib modules.
- remove special case for Stdlib.Pervasives in printtyp.ml
As a result of the poll in #1855.
The votes at the time of commiting are:
- 18 for `protect ~finally`
- 12 for `try_finally ~finally`
- 1 for `finally ~cleanup`
- 0 for `try_finally ~always`
- 0 for `try_finally ~cleanup`
Signed-off-by: Marcello Seri <marcello.seri@gmail.com>
`try_finally work ~always ~exceptionally` is designed to run code
in `work` that may fail with an exception, and has two kind of
cleanup routines:
- `always`, that must be run after **any** execution of the function
(typically, freeing system resources), and
- `exceptionally`, that should be run **only** if `work` or `always`
failed with an exception (typically, undoing user-visible state
changes that would only make sense if the function completes
correctly).
I had to locally re-define `rab_backtrace`, `get_raw_backtrace`,
`raise_with_backtrace` because I could not refer to `Printexc`.
Signed-off-by: Marcello Seri <marcello.seri@gmail.com>
* added otherlibs/unix/fsync.c and otherlibs/unix/unix.ml
* add Unix.fsync
* added otherlibs/win32unix/fsync.c
* Unix.fsync for windows
* big typo
* Rewrite Unix.fsync stub for Windows
* belts and braces: fail in the case of named pipe
* Add missing include
* corrected header
* better ocamldoc for fsync
* rm fsync.c from the UNIX_FILES list
* updated Changes for Unix.fsync
* Use _commit instead of FlushFileBuffers
* Include <io.h>
It has been deprecated since 2000, shown a deprecation warning
since 4.02, and Sort.merge is documented to have undefined behavior
when the lists being merged are not sorted in the first place.
The `top_of_stack` field of the `th` descriptor for the new thread was initialized too late, causing `caml_top_of_stack` to be NULL when the thread starts running. The fix is to initialize `th->top_of_stack` earlier.
Which is needed for the case without support for linkat as reported by Gabriel Scherer.
Also set _XOPEN_SOURCE == 700 instead of testing for it.
Test for availablility of linkat by looking at AT_SYMLINK_FOLLOW, which is, according to
introduced in POSIX IEEE Std 1003.1-2008 for linkat.
This commit renames a few C compiler related build variables so that
they are reserved for the build system. They will then be re-introduced,
but this time as user varialbes whose value can be freely customized
when compiling the package, without risking to conflict with those
command-line flags that are required by the build system itself.
Here are the variables this commit renames:
- CFLAGS -> OC_CFLAGS
- CPPFLAGS -> OC_CPPFLAGS
- LDFLAGS -> OC_LDFLAGS
Note: before this commit the compilation of scheduler.c in
otherlibs/threads was relying on make's implicit rule to compile C files.
Since this commit stops using the standard variables for flags,
it is necessary to introduce an explicit rule to compile C files
and that makes use of the newly introduced variables.
This target has been broken by the move of bigarray C files to the runtime.
Before the current PR:
```
$ make depend
gcc -MM -O2 -fno-strict-aliasing -fwrapv -Wall -Werror -fno-tree-vrp -fPIC -I../unix -DIN_OCAML_BIGARRAY -D_FILE_OFFSET_BITS=64 -D_REENTRANT -DCAML_NAME_SPACE -I../../byterun *.c | sed -e 's/\.o/.$(O)/g' > .depend
gcc: error: *.c: No such file or directory
gcc: fatal error: no input files
```
Unix.{in,out}_channel_of_descr now check that descr has stream semantics.
These functions fail if the given file descriptor is not suitable for
character-oriented I/O, e.g. a block device or a datagram socket.
Fixes part of MPR#7238.
The st_rdev is not the minor number, it is the device ID according to
the manpage of stat, which is actually the major * 256 + minor. I've
replaced the old incorrect docstring with the description of st_rdev
from the man page of the stat command - it looks like the implemenation
of Unix.stat just puts the st_rdev returned from stat into the st_rdev
field.
Signed-off-by: Gabor Igloi <gabor.igloi@citrix.com>
Running Clang 6.0 and GCC 8 with full warnings on suggests a few simple improvements and clean-ups to the C code of OCaml. This commit implements them.
* Remove old-style, unprototyped function declarations
It's `int f(void)`, not `int f()`. [-Wstrict-prototypes]
* Be more explicit about conversions involving `float` and `double`
byterun/bigarray.c, byterun/ints.c:
add explicit casts to clarify the intent
renamed float field of conversion union from `d` to `f`.
byterun/compact.c, byterun/gc_ctrl.c:
some local variables were of type `float` while all FP computations
here are done in double precision;
turned these variables into `double`.
[-Wdouble-promotion -Wfloat-conversion]
*Add explicit initialization of struct field `compare_ext`
[-Wmissing-field-initializers]
* Declare more functions "noreturn"
[-Wmissing-noreturn]
* Make CAMLassert compliant with ISO C
In `e1 ? e2 : e3`, expressions `e2` and `e3` must have the same type.
`e2` of type `void` and `e3` of type `int`, as in the original code,
is a GNU extension.
* Remove or conditionalize unused macros
Some macros were defined and never used.
Some other macros were always defined but conditionally used.
[-Wunused-macros]
* Replace some uses of `int` by more appropriate types like `intnat`
On a 64-bit platform, `int` is only 32 bits and may not represent correctly
the length of a string or the size of an OCaml heap block.
This commit replaces a number of uses of `int` by other types that
are 64-bit wide on 64-bit architectures, such as `intnat` or `uintnat`
or `size_t` or `mlsize_t`.
Sometimes an `intnat` was used as an `int` and is intended as a Boolean
(0 or 1); then it was replaced by an `int`.
There are many remaining cases where we assign a 64-bit quantity to a
32-bit `int` variable. Either I believe these cases are safe
(e.g. the 64-bit quantity is the difference between two pointers
within an I/O buffer, something that always fits in 32 bits), or
the code change was not obvious and too risky.
[-Wshorten-64-to-32]
* Put `inline` before return type
`static inline void f(void)` is cleaner than `static void inline f(void)`.
[-Wold-style-declaration]
* Unused assignment to unused parameter
Looks very useless. [-Wunused-but-set-parameter]
Some makefiles (lex, stdlib, otherlibs) would only offer allopt, while
others (ocamldoc, tools) only offered opt.opt. It is inconvenient to
have to remember which target name to use while going through various
repositories.
The idea is to add a variant of the Unix.open_process{,_in,_out,_full} functions that work with a pair (program, args) directly, without going through the system shell, which introduces extra complexity (in particular, with respect to quoting).
I used the name Unix.open_process_args which is the one suggested in ocaml-batteries-team/batteries-included#858, but I am not completely sold on it.
One uses the usual "close" functions Unix.close_process{,_in,_out,_full} with the new functions as well.
The "old" functions Unix.open_process{,_in,_out,_full} are re-implemented in terms of the "new" functions, so I did not feel pressed to add new tests.
Addresses: MPR#7794.
Fixes: MPR#7253 (in large part), MPR#7796 (in full).
As shown in the PRs above there are several cases where do_at_exit
is called several times, causing functions registered with at_exit
to be called several times. Also, an at_exit function that raises
could prevent other at_exit functions from being run.
This commit doesn't try to prevent multiple calls to do_at_exit,
but makes sure that each function registered with at_exit is run
at most once. The idea is due to Nicolás Ojeda Bär.
The segfault happens when you use a wildcard '*' or '?'
in the argument of the caml program compiled on Windows
64bits.
The handle was corrupted because is cast from
intptr_t (64bits) to int (32bits).
In order to make sure that both `intptr_t` and `uintptr_t`
are available on MSVC we forced the inclusion of <stdint.h>
for versions that have it (starting from Visual Studio 2010
(MSVC version 1600)). For older versions, the typedefs may be
found in <stddef.h>; therefore we forced the inclusion of this
standard header in config.h.
The proposed behavior of `-config-var s` is as follows:
- if `s` is an existing configuration variable, print its value as
a string and exit with a success return value (0)
- if `s` is not an existing configuration variable, print nothing
and exit with a failure return value (non-0)
Note that we do not print a newline after the value of the
configuration variable. In particular, if the value is an empty
string, the output is undistinguishable from the output for
non-existing variables, the return value has to be considered instead.
The following alternative behaviors were considered:
- We could print a newline after the configuration value, which
would let users distinguish empty values from non-existing variables
by counting the lines of output, and would also be more pleasant for
users invoking the option from the command-line. However, the way
bash works on Windows means that $(ocamlc -config-var foo) would keep
a trailing \r in its output, and portable scripts would have to use
$(ocamlc -config-var foo | tr -d '\r') instead, which is a pain.
(This issue was pointed out by David Allsopp)
- We could print a message on the error output if the configuration
variable does not exist. This is clearer to a human user, but it is
annoying for scripts if they forget to silence the error output and
get their output mixed with our error messages. The main use of this
new feature is for scripting purposes.
Xavier Leroy
Jeremie Dimino
Sébastien Hinderer
Jacques-Henri Jourdan
Stephen Dolan
KC Sivaramakrishnan
Gabriel Scherer
Leo White
Greta Yorsh
JPR
Nicolás Ojeda Bär
Xavier Leroy
Thomas Refis
Mark Shinwell
Damien Doligez
Stephen Dolan
Stephen Dolan
Romain Beauxis
Jacques-Henri Jourdan
David Allsopp
Tim Cuthbertson
Damien Doligez
ahh
Guillaume Munch-Maccagnoni
Florian Angeletti
Thomas Refis
Runhang Li
Alain Frisch
Daniel Bünzli
yallop
Armaël Guéneau
Marcello Seri
Hugo Heuzard
Francois Berenger
Xavier Clerc
Gabriel Radanne
whitequark
sliquister
Christopher Zimmermann
Gabor Igloi
ygrek
Marc Lasson