************
Other Topics
************

This section describes some details of Dune for advanced users.

META File Generation
====================

Dune uses ``META`` files from the `findlib library
manager <http://projects.camlcity.org/projects/findlib.html>`__ in order
to interoperate with the rest of the world when installing libraries. It's 
able to generate them automatically. However, for the rare cases
where you would need a specific ``META`` file, or to ease the transition
of a project to Dune, it is allowed to write/generate a specific
one.

In order to do that, write or setup a rule to generate a
``META.<package>.template`` file in the same directory as the
``<package>.opam`` file. Dune will generate a ``META.<package>``
file from the ``META.<package>.template`` file by replacing lines of
the form ``# DUNE_GEN`` with the contents of the ``META`` it would
normally generate.

For instance if you want to extend the ``META`` file generated by
Dune, you can write the following ``META.foo.template`` file:

.. code::

   # DUNE_GEN
   blah = "..."

Findlib Integration
===================

Dune uses ``META`` files to support external libraries. However, it
doesn't export the full power of Findlib to the user, and it especially
doesn't let the user specify *predicates*.

This limitation is in place because they haven't been
needed thus far, and it would significantly complicate things to add full 
support for them. In particular, complex ``META`` files are often handwritten, and
the various features they offer are only available once the package is
installed, which goes against the root ideas Dune is built on.

In practice, Dune interprets ``META`` files, assuming the following
set of predicates:

- ``mt``: refers to a library that can be used
  with or without threads. Dune will force the threaded
  version.

- ``mt_posix``: forces the use of POSIX threads rather than VM
  threads. VM threads are deprecated and will soon be obsolete.

- ``ppx_driver``: when a library acts differently depending on whether
  it's linked as part of a driver or meant to add a ``-ppx`` argument
  to the compiler, choose the former behavior.

Note that Dune does not read *installed* ``META`` files for libraries
distributed with the compiler (as these files are not installed by the compiler
itself, but installed by `ocamlfind` and aren't always 
accurate). Instead, Dune uses its own internal database for this information.

Dynamic Loading of Packages with Findlib
========================================

The preferred way for new development is to use :ref:`plugins`.

Dune supports the ``findlib.dynload`` package from `Findlib
<http://projects.camlcity.org/projects/findlib.html>`_ that enables
dynamically-loading packages and their dependencies (using the OCaml Dynlink module).
Adding the ability for an application to have plugins just requires adding
``findlib.dynload`` to the set of library dependencies:

.. code:: scheme

    (library
      (name mytool)
      (public_name mytool)
      (modules ...)
    )

    (executable
      (name main)
      (public_name mytool)
      (libraries mytool findlib.dynload)
      (modules ...)
    )


Use ``Fl_dynload.load_packages l`` in your application to load 
the list ``l`` of packages. The packages are loaded
only once, so trying to load a package statically linked does nothing.

A plugin creator just needs to link to your library:

.. code:: scheme

    (library
      (name mytool_plugin_a)
      (public_name mytool-plugin-a)
      (libraries mytool)
    )

For clarity, choose a naming convention. For example, all the plugins of
``mytool`` should start with ``mytool-plugin-``. You can automatically
load all the plugins installed for your tool by listing the existing packages:

.. code:: ocaml

    let () = Findlib.init ()
    let () =
      let pkgs = Fl_package_base.list_packages () in
      let pkgs =
        List.filter
          (fun pkg -> 14 <= String.length pkg && String.sub pkg 0 14 = "mytool-plugin-")
          pkgs
      in
      Fl_dynload.load_packages pkgs

Classical PPX
=============

*Classical PPX* refers to running PPX using the ```-ppx`` compiler option, which is
composed using Findlib. Even though this is useful to run some (usually old)
PPXs that don't support drivers, Dune doesn't support preprocessing with
PPX this way. However, a workaround exists using the `ppxfind
<https://github.com/diml/ppxfind>`_ tool.

Profiling Dune
==============

If ``--trace-file FILE`` is passed, Dune will write detailed data about internal
operations, such as the timing of commands that Dune runs.

The format is compatible with `Catapult trace-viewer`_. In particular, these
files can be loaded into Chromium's ``chrome://tracing``. Note that the exact
format is subject to change between versions.

.. _Catapult trace-viewer: https://github.com/catapult-project/catapult/blob/master/tracing/README.md

.. _package-version:

Package Version
===============

Dune determines a package's version by looking at the ``version``
field in the :ref:`package stanza <package>`. If the version field isn't 
set, it looks at the toplevel ``version`` field in the
``dune-project`` field. If neither are set, Dune assumes that we are in
development mode and reads the version from the VCS if any. The way it
obtains the version from the VCS in described in :ref:`the build-info
section <build-info>`.

When installing the files of a package on the system, Dune
automatically inserts the package version into various metadata files
such as ``META`` and ``dune-package`` files.

.. _ocaml-syntax:

OCaml Syntax
============

If a ``dune`` file starts with ``(* -*- tuareg -*- *)``, then it is
interpreted as an OCaml script that generates the ``dune`` file as described
in the rest of this section. The code in the script will have access to a
`Jbuild_plugin
<https://github.com/ocaml/dune/blob/master/plugin/jbuild_plugin.mli>`__
module containing details about the build context it's executed in.

The OCaml syntax gives you an escape hatch for when the S-expression
syntax is not enough. It isn't clear whether the OCaml syntax will be
supported in the long term, as it doesn't work well with incremental
builds. It is possible that it will be replaced by just an ``include``
stanza where one can include a generated file.

Consequently **you must not** build complex systems based on it.

.. _variables-for-artifacts:

Variables for Artifacts
-----------------------

For specific situations where one needs to refer to individual compilation
artifacts, special variables (see :ref:`variables`) are provided, so the
user doesn't need to be aware of the particular naming conventions or directory
layout implemented by Dune.

These variables can appear wherever a :ref:`deps-field` is expected and also
inside :ref:`user-actions`. When used inside :ref:`user-actions`, they
implicitly declare a dependency on the corresponding artifact.

The variables have the form ``%{<ext>:<path>}``, where ``<path>`` is
interpreted relative to the current directory:

- ``cmo:<path>``, ``cmx:<path>``, and ``cmi:<path>`` expand to the corresponding
  artifact's path for the module specified by ``<path>``. The basename of
  ``<path>`` should be the name of a module as specified in a ``(modules)``
  field.

- ``cma:<path>`` and ``cmxa:<path>`` expands to the corresponding 
  artifact's path for the library specified by ``<path>``. The basename of ``<path>``
  should be the name of the library as specified in the ``(name)`` field of a
  ``library`` stanza (*not* its public name).

In each case, the expansion of the variable is a path pointing inside the build
context (i.e., ``_build/<context>``).

Building an Ad Hoc ``.cmxs``
----------------------------

In the model exposed by Dune, a ``.cmxs`` target is created for each
library. However, the ``.cmxs`` format itself is more flexible and is
capable to containing arbitrary ``.cmxa`` and ``.cmx`` files.

For the specific cases where this extra flexibility is needed, one can use
:ref:`variables-for-artifacts` to write explicit rules to build ``.cmxs`` files
not associated to any library.

Below is an example where we build ``my.cmxs`` containing ``foo.cmxa`` and
``d.cmx``. Note how we use a :ref:`library` stanza to set up the compilation of
``d.cmx``.

.. code:: lisp

    (library
     (name foo)
     (modules a b c))

    (library
     (name dummy)
     (modules d))

    (rule
     (targets my.cmxs)
     (action (run %{ocamlopt} -shared -o %{targets} %{cmxa:foo} %{cmx:d})))
