Organisms

Compositional-rule organism at escalating difficulty (section 2.10.2, generator 1).

The rule is a boolean combinator over predicates (_LEVELS); at level the chosen response satisfies the combinator and the rejected does not. At strength = 1 every pair obeys the rule exactly (the foundry test asserts this on both train and OOD); below 1 a (1 - strength) fraction of pairs are label-flipped so an instrument’s ROC has separable error to measure.

Returns the training DataView and the AnswerKey whose rule is this combinator.

Dose-controlled spurious-correlation organism (section 2.10.2, generator 2).

The true rule is a single predicate on true_feature: the chosen side has it and the rejected does not. A spurious_feature is correlated with the label at strength rho: exactly one side carries the spurious marker, and it lands on the chosen (label-aligned) side with probability rho. So the empirical agreement rate between “spurious is on the chosen side” and the label is rho: at rho = 0.5 the spurious feature is independent of the label, at rho = 1.0 it is a perfect confound. This is the dose axis the scorecard must be monotone in (section 2.10.3).

rho is the dose; the standard sweep is {0.5, 0.6, 0.7, 0.8, 0.9, 1.0}.

Planted-hidden-objective organism (section 2.10.2, generator 3).

The advertised rule is a single visible predicate (visible_feature), but the label is decided by a utility w_v * visible + w_h * hidden: the model also secretly rewards hidden_feature. A meaningful fraction of pairs are tied on the visible feature and broken by the hidden one, so the hidden objective is present and recoverable, which is the object the auditing game hunts for (section 2.10.4). A blind auditor who only checks the visible rule misses the hidden term.

Synthetic multi-objective gate organism (section 2.10.2, generator 4).

The rule is a hard safety gate followed by a quality ordering: a response is preferable only if it is safe (carries no [unsafe] marker) and factual. A response with the unsafe marker is vetoed regardless of quality, so some pairs pit a factual-but-unsafe rejected side against a safe-factual chosen side (testing the veto) and others pit two safe responses differing only in quality (testing the ordering). The combinator "safe AND factual" is the gate.

Planted-intransitivity organism: cyclic preferences as tournaments (section 2.10.2, gen 5).

Each item is a Tournament over three responses A, B, C with a planted 3-cycle A > B > C > A (each beats the next by wins to 0). This is the curl ground truth for the topology science (S6): a preference operator whose intransitive mass is nonzero by construction. The foundry test asserts every tournament contains a cycle (has_cycle).

Planted annotator-mixture organism with known H(V) (section 2.10.2, generator 6).

The population of annotators is a categorical distribution mixing over named annotators, each of whom prefers a different feature (_ANNOTATOR_FEATURE). Every pair is labelled by an annotator drawn from the mixture, and the chosen side is the one carrying that annotator’s favoured feature. The mixture entropy H(V) (in bits) is known exactly from the chosen weights and is the channel-capacity ground truth; the foundry test checks the recorded entropy against a recompute and the empirical assignment entropy against the target.

Planted-rubric organism at controlled (K, d, correlation) (section 2.10.2, generator 7).

K criterion directions are planted in R^d with an exact pairwise cosine correlation (make_rubric_directions). Each response carries a latent x in R^d; its aggregate reward is sum_k v_k . x and the chosen side is the higher-reward one. The ground truth here is the set of criterion directions (stored in true_directions), which the coherence/Welch science reads; the text rendering is a coarse encoding of the dominant criterion and is deliberately secondary.

Planted-hack-direction organism (section 2.10.2, generator 8; A12’s predicted hack mode).

The label rewards hack_feature (the chosen side carries it, the rejected does not), so an RM trained on this data learns to love the hack, giving it positive susceptibility chi > 0. But the hack is arranged to be anti-correlated with the gold objective gold_feature: with probability anti_correlation the gold feature sits on the rejected (non-hacky) side. So optimizing the learned reward (more hack) reduces gold, which is exactly the chi_i > 0 with Cov(f_i, gold) <= 0 signature the forecast science hunts for. The ground truth is the hack and gold directions in feature space (true_directions).

Planted-epistemic-error organism: mislabeled receipts at rate epsilon (section 2.10.2).

The rule prefers grounded (factual) responses, but a fraction epsilon of the “factual” markers are fabricated receipts: the response looks grounded to the grader while gold says it is false. The grader’s error here is epistemic (a false belief about truth), which the epistemic-axiological factorization (L2) separates from value error below. meta['fabricated'] records which pairs carry the mislabeled receipt.

Planted-value-error organism: preference inversion at fixed truth (section 2.10.2).

Truth is never in doubt (the factual side is always identifiable), but at rate delta the rule prefers the less truthful, more agreeable side anyway. The grader’s error here is axiological (a misaligned value, not a false belief), the complement of epistemic error above. meta['inverted'] records which pairs carry the inversion.

Planted curl vs harmonic mass with a known Hodge decomposition (section 2.10.2, generator 11).

Generates a collection of tournaments containing both pure curl (3-cycles) and pure harmonic (chordless rings of length 4 to 7) preferences. This serves as the ground truth for preference topology (S6).

The exact ground truth an instrument is graded against (section 2.10.1).

rule and channels describe the planted structure. true_directions maps a name to the direction that structure lives along where one is known by construction (a rubric’s criterion vectors, a hack direction in feature space); it is None when the direction is only recoverable from a trained trunk. governs_behavior_oob records whether the rule provably governs behaviour out of distribution; it starts False and is set only by verify.py on a held-out OOD split, never assumed (section 2.10.3). family names the organism family for scorecard and calibration references, and notes carries any human-readable caveats.

This type is intentionally mutable so verify.py can stamp governs_behavior_oob after it has the trained signal; every other spec type is frozen.

Grades an instrument against an organism’s answer key (section 2.10.3).

An instance names the instrument being graded (observable). evaluate consumes the instrument’s per-item scores, either on a single regime or across a dose sweep, and produces a ScorecardEntry. The scorecard is agnostic to what produced the scores: the pure test grades a synthetic detector, the M3 integration grades DLA and patching, and the code path is identical.

Train, verify, and recover a planted rule on the tiny CPU trunk (section 2.10.3, R6).

Generates a planted single-rule (prefer factual) organism with a dose-spurious confound (cites correlated with the label at rho), trains the tiny trunk, verifies the rule governs behaviour on a held-out OOD split, and runs the built-in mean-difference detector to recover the rule OOD. The returned recovered flag is the calibration go/no-go a CI test asserts (R6).