- Introduce `-dcamlprimc`, to keep the generated C file containing the primitive list
- Use `-fdebug-prefix-map` for compiling temporary C files when this option is supported
Some makefiles (lex, stdlib, otherlibs) would only offer allopt, while
others (ocamldoc, tools) only offered opt.opt. It is inconvenient to
have to remember which target name to use while going through various
repositories.
The target names for the 'man' and 'html' files are
stdlib_{html,man}/Stdlib.{3o,html}
but these files are never produced by the corresponding rules,
so the rule is re-run on each "make world.opt".
This patch changes these target names to Pervasives.{3o,html}, which
is an actual file produced by the build. It is not fully clear to me
whether the authors of the original rule intended to also produced
documentation for a joint Stdlib module.
Before this patch, running "make world.opt" in an already-built directory
would take 10s on my machine. Now it takes 2s.
(no change entry needed)
On a standard OCaml install, `man Nativeint` shows the following
sentence in its first paragraph:
All arithmetic operations over nativeint are taken
modulo 2^{32 or 2^{64 depending on the word size of the architecture.
The `2^{32` is a rendering bug. This PR removes the unbalanced curvy
bracket, so it prints 2^32 and 2^64 instead.
In theory just printing ^ and _ risks creating ambiguities for
multi-words superscripts or subscripts: "2^foo bar" could be either
"2^{foo bar}" or "2^{foo} bar". In practice, the ocamldoc text in the
standard library only uses the superscript for bit sizes.
Even for arbitrary user documentation comments, "^{" or "^" have the
same property of not being explicit about where the superscript ends,
so this would not be a regression even on multi-word superscripts.
I can observe weird performance bottlenecks on my machine caused by
the use of 'cp' in the 'install' scripts of OCaml. When installing
into a directory that is already populated by an existing
installation, 'make install' can routinely take 10s on my machine¹. After this
change it reliably takes 1.5s, independently of whether the
destination is already populated or not.
¹: a brtfs filesystem on an old-ish SSD
Why I care
----------
An extra 10s delay due to 'make install' can be noticeable in tight
change-build-install-test feedback loops for a compiler change where
we change the compiler, have a fast 'make world.opt' due to
incremental builds, install the change and test it -- possibly after
installing a couple opam packages, which can be fairly quick.
Partial diagnosis
-----------------
The performance issue seems to be caused by the fact that 'cp' (at
least the GNU coreutils version), when the file already exists,
replaces it by opening it in writeonly+truncate mode and writing the
file content ('strace' shows that the delay is caused within an
'openat' call). In particular, using the --remove-destination option
(which changes 'cp' to just remove the destination file before
copying) removes the performance issue, but this option seems missing
from the BSD/OSX 'cp' so it could cause portability issue.
Change
------
The present commit rewrites the 'install' targets of all Makefiles to
use the 'install' command instead. 'install' by default gives
executable-like permission to the destination file, instead of reusing
the source file's permissions, so we specify manually the permission
modes, depending on whether the installed file is an executable (or
dynamically-linked library) or just data (including other compiled
object files).
Testing
-------
I checked manually that the permissions of the installed files are
identical to the ones of the current 'cp'-using targets, except for
some '.mli' file in middle_end which currently have +x bits enabled
for no good reason.
Remark: To test this, playing with the DESTDIR variable is very useful
(this lets you install to a new directory (or the same as before)
without having to re-run the configure script). I used the following,
fairly slow shell script to collect permissions:
for f in $(find $DESTDIR); do \
echo $(basename $f) $(ls -l $f | cut -d' ' -f1); \
done | sort
Remark: it is important to run `sync` in-between 'make install' runs
to avoid timing effects due to filesystem or disk caching
strategies. I believe that this corresponds to the natural time delay
(and unrelated disk activity) that would occur in realistic
change-install-test feedback loops.
* add `Seq` module, expose iterator conversions in most containers
* small typo
* typo
* change order of arguments for `{Map,Set}.add_seq`
* watch for max string length in `Bytes.of_seq`
* wip: make it build again
* Fix dependency
Sys needs to be linked before Bytes in stdlib.
* Update threads/stdlib.ml
* Update stdlib_no_prefixed/.depend
* fix inconsistencies with label modules
* update testsuite to work with seq
* update change file
* small change in `Hashtbl.to_seq`, capturing only the underlying array
* add some documentation to seq.mli
* revert to good ol' module type names for hashtables
* fix test
* change style of comments in seq.mli
* follow some demands in review of GPR #1002
* some fixes for #1002
* add Seq-related functions to Ephemeron
* add some comments on `Hashtbl.of_seq`
* add more tests for `Hashtbl.{to,of}_seq`
* fix bug in `Ephemeron.to_seq`
* Update Changes
These options allow to control whether identifiers are made unique by
appending a stamp to them when dumping intermediate representations or not.
The default is to print the stamp, as is done currently.
The "-dno-unique-ids" option is useful e.g. to simplify the comparison
between a produced intermediate reprsentation (-dlambda, say) and the
expected one, in the context of the testsuite, for instance.
awk is symbolic link in Cygwin, which means it can't be used in -pp for
a native Windows build. Just use gawk instead, as no other package
provides the awk command on Cygwin.
Except for the Camlinternal* modules and the new Stdlib module, all
modules in the stdlib now compile to Stdlib__<module>.
Pervasives is renamed to Stdlib and now contains a list of aliases
from the long names to the short ones, so that from inside and outside
the stdlib we can refer to the standard modules as just List or
Stdlib.List rather than Stdlib__list.
In order to avoid printing the long names in error messages and in the
toplevel, the following heuristic is added to Printtyp: given a path
Foo__bar, if Foo.Bar exists and is a direct or indirect alias to
Foo__bar, then prefer Foo.Bar.
A bootstrap step was required to replace Pervasives by Stdlib as the
module opened by default.
This causes trouble when the doc comments are actually in UTF8 and
the user is providing their own preamble with an UTF8 inputenc.
The default preamble still contains \usepackage[latin1]{inputenc},
so the Latin-1 accents will still display fine.
On OpenBSD this script removes the newline at the end of each line
that is transformed. If the next line is a `# lineno "filename"`
directive, a syntax error occurs.
This commit changes the script to use a sed 's' command instead of a
'c' command. This restores the expected behavior under OpenBSD and
seems to make no difference for other systems.
This commit makes the heading hierarchy of ocamldoc start at {0 rather
than {1. This level {0 should be reserved for global titles, freeing
the use of {1 for normal subtitles.
This commit adds a new id to classes, modules and module types.
The class id replaces the preexisting name attribute that was intended
to be an id attribute.
Continuing a general effort, this commit removes the "num" library for arbitrary-precision arithmetic from the core OCaml system. A standalone distribution of this library already exists and is hosted at https://github.com/ocaml/num
To avoid module preamble repetition, ocamldoc only use as a module
preamble documentation comments that occur before any module elements,
which should also prevent some unexpected module preamble when the first
documentation comments appears in the middle of the source file.
The troff .TH macro takes up to three extra args, according to the
groff documentation at gnu.org. ocamldoc inserts a fourth argument
"source:". Remove the extra argument. Fixes commit a87c3f20e
("Enable ocamldoc to build reproducible manpages")
Without this change the header/footer of each ocaml manpage is
broken. Example: the date should be in the middle of footer,
instead of the left side. The string "source:" should not be shown
at all. No other manpage has such string.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
This commit moves:
- config/m.h to byterun/caml/m.h
- config/s.h to byterun/caml/s.h
Consequently, m.h and s.h now get installed alongside other
OCaml header files.
This commit also updates the .depend files, introducing updates in the
dependencies which are not consequences of this commit itself.
This commit replaces most of the use of <br> tags in ocamldoc html
backend by more meaningful tags, in order to improve the themability
of the generated html code.
The fact that ocamldoc uses absolute file paths in its error messages
creates a usability problem for ocamlbuild users that see the path of
the files in _build instead of the path of the files in the source
project. See <https://github.com/ocaml/ocamlbuild/issues/79>:
cd /tmp
mkdir repro; cd repro
echo "(** hey {ol {1 bla}} *)" > a.mli
echo "A" > doc.odocl
ocamlbuild doc.docdir/index.html
raises the error message:
> File "/tmp/repro/_build/a.mli", line 0, character 11:
> Error parsing text:
> hey {ol {1 bla}}
> ^
and then people jump to the reported error site from their editor, and
then they are editing temporary _build files and it is a mess.
After the patch is applied, we get the following error message instead:
> File "a.mli", line 0, character 11:
> Error parsing text:
> hey {ol {1 bla}}
> ^
The code that is removed by the patch is also a mess, of undocumented
assumptions. It is careful to Sys.chdir to the argument's directory,
and to capture a long name there and Sys.chdir back. This would make
perfect sense if other parts of ocamldoc were also using 'chdir' and
one could therefore not trust relative paths. But these were the only
two places using 'chdir', so the present change actually establishes
the property that relative paths can be trusted.
I am not imaginative enough to have a sense of what can go wrong as
we turn those absolute paths into relative paths. I looked at the
blame information, this code comes from the very first ocamldoc
commits by Maxence in 2002. My guess would be that early prototypes
used 'chdir' more agressively (maybe to call the typechecker as an
external command?), making absolute paths useful.
It is possible to craft comments that can expand to unfiltered HTML
fragments.
For example, the following program:
```ocaml
(** {{: "><script>alert(1)</script>"} } *)
let n = 0
```
Would generate a HTML file containing:
```html
<a href=" "><script>alert(1)</script>""> </a>
```
Using this technique it is possible to leak cookies to a third party.
The fix is not perfect since it does not generate usable documentation,
but the generated HTML is harmless.
Note that input like `{{: javascript:alert(1)} }` will still run
arbitrary JS but requires human interaction.
Before, parsing the comment
(**
Sample code to iterate over all frames (inlined and non-inlined):
{|
(* note: the start-of-code-block above should be after a blank line *)
|}
*)
would show
File ".../foo.mli", line 3, character 6:
{|
^
now it shows
File ".../foo.mli", line 3, character 6:
Error parsing text:
{|
^
This fixes a problem related to the way ocamldoc's bytecode executable
is called under Windows, reported by @alainfrisch at
https://github.com/ocaml/ocaml/pull/808#issuecomment-257364340
As he explains, ``Even if the PATH is extended with explicit paths to
otherlibs/win32unix and otherlibs/str, ocamlrun will still look up
the stub dlls in the installation directory first, as can be seen by
setting OCAMLRUNPARAM=v=256. If another version was installed in the
same target directory before, the old dlls will be loaded which can lead
to failure (e.g. I just got "Fatal error: unknown C primitive 'unix_lstat'").
A fix could be to set:
CAML_LD_LIBRARY_PATH="../otherlibs/win32unix;../otherlibs/str"
instead of changing PATH''.
This commit implements the proposed fix, due to @alainfrisch.
When reading a serialized-ast file, Pparse.file sets
Location.input_name to the filename embedded in the AST, and this is
correct. But before this patch it would also set Location.input_name
to the filename if this is a regular file, and this is wrong: this
filename may be a temporary file used for preprocessing (-pp option),
with a randomly-generated name, while Location.input_name is in fact
already correctly set to the user-provided source path.
I needed to fix two lines in ocamldoc/odoc_analyze.ml that
used !Location.input_file but actually required access to the
post-processing file (ocamldoc re-opens source files and rereads them
to detect documentation comments). This is not an invasive change as
the path to the post-processing file is available at this point in the
code (as the [input_file] variable).
This ocamldoc issue was caught thanks to Debian downstream work in bug
triaging ( mlpost breaks if it is not fixed, see
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=802347 ) and package
maintenance (this bug would not have been found if Debian maintainers
had not kept the mlpost documentation generator working even after
non-trivial ocamldoc changes).
There is a typo in the generated CSS for link to modules in the index
page. CSS colors should start with `#`.
As a consequence, this also makes the default stylesheet valid according
to <https://jigsaw.w3.org/css-validator>.
This commit changes the behaviour of both the Unix and Windows build systems.
The changes are either specific to one build system or common to both.
Changes specific to the Unix build system:
- Use the -slash option when invoking ocamldep. This does nothing on
Unix and makes it possible to use the same command under Unix and Windows.
- install target:
- Directories are created unconditionnally, instead of creating them
only if they do not already exist
(also true for the installopt target)
- cp no longer uses the -f flag, to be consistent with the
other directories
- The custom directory $(INSTALL_LIBDIR)/custom is no longer created
because nothing was installed there anyway
- In the opt.opt target, consecutive calls to make have been
replaced by prerequisites, enhancing parallelisation opportunities.
Changes specific to the Windows build system:
- Whaen compiling .ml files, use the same line-number-preserving
preprocessor as under Unix, rather than the grep -v DEBUG command.
- ocamldoc generators and odoc_test are now compiled
- ocamldoc can now be run from sources to generate the documentation
of the standard library
- The test targets are now also available under Windows
- opt.opt now builds native-code versions of the generators
Changes affecting both Unix and Windows build systems:
- The targets that were depending on the empty "dummy" target
are now declared as .PHONY targets, dummy has been removed.
- In the all target, successive calls to make have been replaced
by prerequisites.
- Commands executed by make clean are now displayed instead of being
executed silently.
With this commit, ocamldoc does not escape anymore space characters
within the <pre> </pre> tags and instead escape space and newline
characters inside <code> </code> when a <pre>-like behavior is desired.
Moreover, the type_* files generated by ocamldoc are correctly assigned
a <pre>-like behavior.
This commit improves support for inline record and associated
documentation within ocamldoc. Note that the support for documentation
on inline record field could be improved in the various ocamldoc
backends.
This commit adds a very basic handling of inline record within
exceptions for signature items. Previously, a signature like
sig
exception E of {lbl:int}
end
would crash OCamldoc due to an assertion failure.
With this commit, the previous signature should be correctly
displayed by ocamldoc as
sig
exception E of {lbl:int}
end
Beware, that the same exception as structure item would still
crash ocamldoc at the time of this commit.
In order to remove some redundancy, the Pparse modules used a dirty
Obj.magic trick to manipulate either structure or signature values
(ASTs parsed from source files). This unsafe approach means that
programming mistakes may result in broken type soudness, and indeed
I myself had to track a very puzzling Segfault during the development
of my Menhir backend (due to a copy-paste error I was passing
Parse.implementation instead of Parse.interface somewhere). Wondering
why your parser generator seems to generate segfaulting parsers is
Not Fun.
This change means that the external interface of Pparse has changed
a bit. There is no way to fulfill the type of Pparse.file in
a type-safe way
val file : formatter -> tool_name:string -> string ->
(Lexing.lexbuf -> 'a) -> string -> 'a
as it promises to be able to process any ast type 'a depending on the
magic number (the last string argument). The knew type-safe interface is
val file : formatter -> tool_name:string -> string ->
(Lexing.lexbuf -> 'a) -> 'a ast_kind -> 'a
where ['a ast_kind] is a GADT type:
type 'a ast_kind =
| Structure : Parsetree.structure ast_kind
| Signature : Parsetree.signature ast_kind
This allows external tools to rely on the features provided by this module. An example
is ocamldoc itself, which could now be implemented as an external tool.
Note that LexiFi variants of OCaml has been embedding depend.ml in the compiler for a long
time in order to support a stricter dependency mode where the compiler is only allowed to load .cmi
files corresponding to dependencies as inferred by ocamldep (hereby ensuring that ocamldep is
sound by construction).
Extend the previous patch allowing make -f Makefile.nt flexdll
install-flexdll not to require the install-flexdll stage.
OCAML_FLEXLINK is utilised as required to allow compilation of the entire
system using an in-tree compiled flexlink. The build process simply
required the flexdll target to appear before world.
opt.opt compiles a native code version of flexlink.exe as flexlink.opt.
install always installs flexlink.exe if it was compiled along with any
required .manifest files. It also installs the appropriate .o/.obj files
to $(INSTALL_LIBDIR).
At present, the bootstrapping is not extended to the Cygwin ports.
Previsously
ocamldoc -g better_html.cmo -html
would just use the standard HTML generator instead of keeping the
generator extended by better_html.cmo. This patch makes sure that when
a standard option (-html, -latex, -texi, -man, -dot) is requested,
extended generators for the same format are chosen if they are
present.
Note that there is still a global "current generator setting", so in particular
-g better_html.cmo -man
will throw away the extended generator, and
-g better_html.cmo -man -html
corresponds to the old behaviour.
When a type definition exposes constructors, adding an empty
constructor comment just after the last constructor is the only way to
place a documentation comment after the type definition and still
attach this comment to the type definition.
However, this empty comment is still printed by most documentation
generators. To fix this aesthetic conundrum, this commit erases empty
constructor comments when constructing the associated
Odoc_type.variant_constructor and thus prevents any printing of these
empty comments.
This commit adds a hook in "parsing/lexer.ml{i,l}" to deactivate the generation of
docstring items. This hook is used by ocamldoc to avoid interferences between these
items and the ocamldoc documentation comments parser.
From comments in typedtree.mli:
When introduced in 2000, this [type] enabled a more efficient code
generation for optional arguments. However, today the information is
redundant as labels are passed to [transl_apply] too. Could be cleaned
up.
Xavier Clerc
Florian Angeletti
Gabriel Scherer
David Allsopp
Sébastien Hinderer
Thomas Refis
Mark Shinwell
Thomas Refis
Damien Doligez
Hugo Heuzard
Jérémie Dimino
Gabriel Radanne
Simon Cruanes
Nicolás Ojeda Bär
objmagic
Jeremie Dimino
Xavier Leroy
Xavier Leroy
Leo White
Fourchaux
Valentin Gatien-Baron
Olaf Hering
Alain Frisch
Mark Li
Pierre Boutillier
Daniel
Etienne Millon
hendriktews
Fabrice Le Fessant
Leo White
Fabrice Le Fessant
Demi Obenour
Christophe Troestler
Pierre Chambart
Armael
Jacques Garrigue
Valentin Lorentz
Sergei Lebedev