---
title: "The config_ambre database"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{The config_ambre database}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  message = FALSE,
  warning = FALSE,
  fig.width = 7,
  fig.height = 4.5,
  out.width = "100%"
)
```

```{r setup}
library(ambre)
set.seed(2024)
```

```{r attach-dplyr}
library(dplyr)
```

Every risk figure `ambre` produces is ultimately read out of one bundled object:
`config_ambre`. It is the *knowledge base* of the package -- pathogen
concentrations, dose-response models, log-reduction credits for treatment and
field barriers, exposure paths, crops and costs, all in one nested list. When
you run `create_scenario()`, a full copy of `config_ambre` is embedded in each
row of the scenario and mutated in place as the pipeline runs (see
`vignette("a-get-started", package = "ambre")`).

This vignette is the practical reference for that object: what is inside it,
how the tables join, and how you would adapt it to your own case.

## Overview: 8 slots, 16 CSVs

`config_ambre` has **eight** top-level slots. Three of them (`treatment`,
`path`, `economic`) are themselves lists of sub-tables, so the object holds
**16 tables** in total. Each table is assembled from one CSV in `data-raw/` by
the build script `data-raw/config_ambre.R`.

```{r overview-str}
str(config_ambre, max.level = 1)
```

The sub-tables live under the three list slots:

```{r overview-subtables}
lapply(config_ambre[c("treatment", "path", "economic")], names)
```

Here is the full file-to-slot mapping, read straight from
`data-raw/config_ambre.R`. This is the map you need when you want to edit the
source data:

```{r mapping-table}
mapping <- tibble::tribble(
  ~`CSV in data-raw/`,                  ~`Slot in config_ambre`,
  "exposure.csv",                       "exposure",
  "ambre_pathogene.csv",                "inflow",
  "ambre_barriere_general.csv",         "treatment$processes",
  "treatment_schemes.csv",              "treatment$schemes",
  "ambre_barriere_voie.csv",            "treatment$barriere_path",
  "ambre_barriere_specifique.csv",      "treatment$barriere_specific",
  "ambre_barriere_deperissement.csv",   "treatment$barriere_decay",
  "ambre_doseresponse.csv",             "doseresponse",
  "ambre_voie_description.csv",         "path$description",
  "ambre_voie_frequence.csv",           "path$frequency",
  "ambre_voie_volume.csv",              "path$volume",
  "ambre_culture_hauteur.csv",          "crop",
  "ambre_sante.csv",                    "health",
  "ambre_culture_eau.csv",              "economic$water_need",
  "ambre_population.csv",               "economic$population",
  "ambre_barriere_cout.csv",            "economic$cost"
)
knitr::kable(mapping)
```

In one sentence per slot: `exposure` holds the Monte-Carlo controls (how many
runs, how many events, the per-event volume); `inflow` is the raw pathogen
concentration in the treated wastewater; `treatment` carries the log-reduction
credits; `doseresponse` and `health` turn a dose into an infection, an illness
and a DALY; `path` describes the 19 exposure routes; `crop` and `economic`
support the agronomic and costing calculations.

## The entity-relation keys

The tables are relational: they share a small set of ID columns and are joined
on them during the pipeline. Knowing the keys is what lets you cross-reference
(and safely extend) the database.

| Key | Meaning | Appears in |
|-----|---------|------------|
| `PathogenID` / `PathogenGroup` | one of 38 pathogens (Bacteria / Viruses / Protozoa) | `inflow`, `doseresponse`, `health`, `treatment$processes` |
| `TreatmentID` | one barrier (treatment step or field practice) | `treatment$*`, `economic$cost` |
| `PathID` | one of 19 exposure routes | `path$description`, `path$frequency`, `path$volume` |
| `PopulationID` | one of 6 exposed populations | `path$description`, `economic$population`, `barriere_*` |
| `MatrixID` | the environmental compartment (Water / Air / Plant / Soil / Product) | `path$description`, `barriere_path` |
| `CropID` / `CropHeight` | one of 6 crops and its height above the irrigation system | `crop`, `economic$water_need` |

`path$description` is the hub that ties a route to *who* is exposed and to
*which* matrix:

```{r er-path}
config_ambre$path$description |>
  select(PathID, PopulationName, MatrixName, PathDescription) |>
  head(4)
```

And `PathogenID` is what links a pathogen's concentration to its health burden:

```{r er-pathogen}
config_ambre$health |>
  select(PathogenID, PathogenName, dalys_per_case) |>
  inner_join(
    config_ambre$inflow |> select(PathogenID, PathogenGroup),
    by = "PathogenID"
  ) |>
  head(4)
```

## The barrier families

The multi-barrier philosophy of `ambre` rests on the PhD thesis of Alice-Rose
Thomas (INSA Lyon, 2024), carried out in the Multiware research context, which
catalogues up to **49 candidate log-reduction barriers** covering both the
treatment plant and the field. `config_ambre$treatment$processes` ships a
curated subset of that catalogue. Every barrier has a `TreatmentName` prefixed
by its type:

* **`Q.`** -- *Quality*: the wastewater-treatment process (Activated Sludge,
  Maturation Pond, UV Reactor, Chlorination, ...).
* **`E.`** -- *Equipment*: on-field hardware (drip irrigation, signage, fences,
  personal protective equipment, ...).
* **`P.`** -- *Practices*: cultivation and irrigation practices (non-edible
  crop, night-time irrigation, cooking, peeling, washing, ...).

The shipped `TreatmentGroup` labels and the number of distinct barriers in each
family are:

```{r barrier-groups}
config_ambre$treatment$processes |>
  distinct(TreatmentID, TreatmentName, TreatmentGroup) |>
  count(TreatmentGroup, name = "n_barriers")
```

```{r barrier-prefix}
config_ambre$treatment$processes |>
  distinct(TreatmentID, TreatmentName) |>
  mutate(family = sub("[.].*", "", TreatmentName)) |>
  count(family, name = "n_barriers")
```

So the bundled database currently encodes 28 distinct barriers across three
families -- a subset of the PhD's 49, and an intentionally extensible one.
Credits from several barriers combine by simple **log-additivity** when a
scenario stacks a treatment scheme and a field barrier
(`vignette("b-treatment-vs-multibarrier", package = "ambre")`).

## Anatomy of a distribution

Eight of the sixteen tables (`exposure`, `inflow`, `treatment$processes`, the
three `barriere_*` tables, `path$frequency` and `path$volume`) share the same
nine-column **distribution block**:

```
type | value | min | max | mode | mean | sd | meanlog | sdlog
```

This is what makes `ambre` a Monte-Carlo engine: instead of a single number, a
barrier or a concentration is a *distribution* that gets resampled on every run.
The `type` column decides which of the other columns are actually read:

```{r dist-legend}
knitr::kable(tibble::tribble(
  ~type,           ~`columns read`,   ~draws,
  "value",         "value",           "a constant, repeated",
  "uniform",       "min, max",        "runif(min, max)",
  "triangle",      "min, max, mode",  "EnvStats::rtri(min, max, mode)",
  "norm",          "mean, sd",        "rnorm(mean, sd)",
  "lognorm",       "meanlog, sdlog",  "rlnorm(meanlog, sdlog)",
  "log10_uniform", "min, max",        "10^runif(min, max)",
  "log10_norm",    "mean, sd",        "10^rnorm(mean, sd)"
))
```

Here is a real barrier row. `Q.1 - Activated Sludge` removes a `uniform`
0.5-to-1 log of protozoa; every other distribution column is `NA` because a
uniform draw only needs `min` and `max`:

```{r dist-row-barrier}
config_ambre$treatment$processes |>
  filter(TreatmentName == "Q.1 - Activated Sludge") |>
  select(TreatmentName, PathogenGroup, type, value, min, max, mode)
```

And the matching inflow row: the raw *Campylobacter jejuni* concentration is a
`uniform` draw between 100 and 5000 organisms per litre:

```{r dist-row-inflow}
config_ambre$inflow |>
  filter(PathogenName == "Campylobacter jejuni") |>
  select(PathogenName, PathogenGroup, type, value, min, max)
```

The `exposure` slot uses the same block for the sampling controls themselves --
1000 runs, 365 events, and a triangular per-event volume:

```{r dist-exposure}
config_ambre$exposure
```

How these rows are turned into 1000 x 365 random values is the subject of
`vignette("g-monte-carlo-engine", package = "ambre")`, via
`create_random_distribution()` and `generate_random_values()`.

## Customizing the database for your case

Because the object is rebuilt from plain CSVs, adapting it is a three-step
loop: **edit the right CSV, rebuild, verify**.

1. **Edit** the CSV that maps to the slot you want to change (see the mapping
   table above). Add a *coherent* row -- a fresh `TreatmentID`, a `type` from
   the legend above, and the columns that `type` requires.
2. **Rebuild** by re-running `data-raw/config_ambre.R`, which re-reads the 16
   CSVs and writes `data/config_ambre.rda`.
3. **Verify** the new entry is reachable with the `query_*` helpers.

The chunk below writes to `data-raw/` and rebuilds the dataset, so it is shown
for reference only (`eval = FALSE`):

```{r customize, eval = FALSE}
# 1. add a new quality barrier to the treatment CSV
csv_path <- file.path("data-raw", "ambre_barriere_general.csv")
new_barrier <- data.frame(
  TreatmentID = 99, TreatmentName = "Q.8 - Ozonation",
  TreatmentGroup = "Quality", PathogenGroup = "Bacteria",
  type = "uniform", value = NA, min = 2, max = 4,
  mode = NA, mean = NA, sd = NA, meanlog = NA, sdlog = NA
)
readr::write_csv(
  rbind(readr::read_csv(csv_path), new_barrier),
  csv_path
)

# 2. rebuild data/config_ambre.rda from the 16 CSVs
source(file.path("data-raw", "config_ambre.R"))

# 3. verify the new barrier resolves to its ID
query_barrier(barrierName = "Q.8 - Ozonation")
```

The verification helpers each translate a *name* into the *ID* the engine works
with (and error early if the name is unknown), so they are the quickest way to
confirm an edit landed. They run against the live `config_ambre`:

```{r query-helpers}
query_pathogen(pathogenName = "Campylobacter jejuni")
query_barrier(barrierName = "Q.3 - UV Reactor")
query_crop(cropName = "Tomato")
```

`query_volume()` and `query_frequency()` take a `PathID` instead and return the
per-path `min`/`max` block that gets injected into a scenario's exposure (this
is what `update_volume()` / `update_frequency()` do automatically inside the
pipeline):

```{r query-path}
query_volume(pathID = 1)
query_frequency(pathID = 1)
```

## Known pitfalls

A few sharp edges to keep in mind before you rely on or extend the database.

**The tables are `spec_tbl_df`.** Because they are read with `readr`, each slot
carries a `readr` column specification and an extra class:

```{r pitfall-class}
class(config_ambre$crop)
```

This is harmless -- `dplyr` verbs work as usual -- but if you compare tables
with `identical()` or write your own I/O you may trip over the `spec`
attribute. Filtering or mutating usually drops it.

**`economic$cost` is the least polished table.** `mean` / `meanlog` are 
currently all `NA`, so a normally- or lognormally-distributed cost you add expecting
`mean`/`meanlog` will not be picked up, see
`vignette("g-monte-carlo-engine", package = "ambre")`.

**`CropHeight` is a signed height in centimetres** above the irrigation
outlet: `-1` means underground (Onion), `0` means at ground level
(Potato, Salad), and positive values mean the edible part sits above the spray
(Tomato 25, Corn seed / Apple 50). It is the intended key for height-dependent
barrier credits.

```{r pitfall-crop}
config_ambre$crop |>
  select(CropName, CropHeight)
```


## See also

* `vignette("a-get-started", package = "ambre")` -- how `config_ambre` is embedded
  in a scenario and mutated through the pipeline.
* `vignette("g-monte-carlo-engine", package = "ambre")` -- how the distribution
  block is sampled into 1000 x 365 values.
* `vignette("c-input-file", package = "ambre")` -- the Excel catalogues (crops,
  populations, paths, barriers) whose allowed values come from these tables.
* `vignette("e-economic-analysis", package = "ambre")` -- how `economic$cost`,
  `water_need` and `population` drive the costing.
