# Contributing to Alcotest

## Setting up your working environment

To clone the project's sources and install its various dependencies, run:

```sh
git clone https://github.com/mirage/alcotest.git    # Get the repository
cd alcotest

opam switch create ./ ocaml-base-compiler.4.10.0    # OPTIONAL: install a project-local Opam switch
opam install -t --deps-only .                       # Install regular and test dependencies
```

Optionally, install `js_of_ocaml-compiler` and [nodejs](https://nodejs.org/en/download/package-manager/) to be able
run the testsuite using `js_of_ocaml`.

```sh
opam install -t js_of_ocaml-compiler
```

Now you can work on the project using the standard Dune commands:

```sh
dune build              # Build all libraries, executables and tests
dune test               # Run the test-suite
dune build @runtest-js  # Run the test-suite on nodejs (using js_of_ocaml)
dune clean              # Delete all artefacts
```

## Project structure

The high-level project structure is as follows:

```sh
├── src/
│   ├── alcotest-engine/
│   │   ├── core.ml           # The main test runner, parameterised over...
│   │   ├── monad.ml          # ... a concurrency monad
│   │   ├── platform.ml       # ... a platform implementation / OS bindings
│   │   │
│   │   ├── cli.ml            # Extends `core.ml` with a command-line API
│   │   │
│   │   ├── pp.ml             # Pretty-printers for Alcotest output
│   │   ├── test.ml           # Combinators for test assertions (with `Alcotest.check`)
│   │   └── utils.ml          # Standard-library extension
│   │
│   │                         # Specific backends:
│   ├── alcotest/             # - Unix-specific API (compatible with js_of_ocaml)
│   ├── alcotest-async/       # - Extends `alcotest/` for test-suites using Async concurrency
│   ├── alcotest-lwt/         # - Extends `alcotest/` for test-suites using Lwt concurrency
│   └── alcotest-mirage/      # - MirageOS-specific API
│
├── test/                     # Project tests (see '§ Expect tests' below)
│
├── dune-project              # Project metadata (opam files are generated from
├── alcotest-async.opam       # data in the `dune-project` file by running
├── alcotest-lwt.opam         # `dune build @install`).
├── alcotest-mirage.opam
├── alcotest.opam
│
├── alcotest-help.txt         # Manpage output (generated by `dune test --auto-promote`)
├── dune
│
├── CHANGES.md                # All user-facing changes get an entry here
└── README.md
```

## Expect tests

The repository contains a number of 'expect' tests in
https://github.com/mirage/alcotest/tree/main/test/e2e. These are short,
self-contained usages of the Alcotest API, with the corresponding output
snapshotted in an `.expected` file:

```sh
test/e2e/alcotest/passing/
├── basic.ml          # The test case
├── basic.expected    # The expected output of the test case
└── basic.opts        # OPTIONAL: command-line options to pass to the test
```

If you change the Alcotest output in some way, you will likely need to update
the expect test snapshots:

1. run `dune test` to get a list of diffs to the tests,
1. check that the diffs are as expected,
1. run `dune promote` to update the snapshots in the repository.

To add a new test, create a new `foo.{ml,expected}` pair in one of the test
directories and run `dune test --auto-promote` to automatically generate the
Dune rules necessary to run your test.

### Random output in tests

Some sections of the Alcotest output are not portable / reproducible under test,
for instance:

- the time taken to run a particular test;
- randomly-generated unique identifiers;
- the full path to the repository on disk;
- the executable file extension.

When running an expect test, these portions of the output are sanitized by
[`strip_randomness.ml`](./test/e2e/strip_randomness.ml) before being compared to
the `.expected` file, resulting in files with portable placeholders:

```
The full test results are available in `<build-context>/_build/_tests/<uuid>`.
Test Successful in <test-duration>s. 4 tests run.
```

If your change adds non-determinism to the Alcotest output, you may need to
alter `strip_randomness.ml` to get it past the CI.
