API reference

Core and evidence

What does a measurement carry besides its value? In reward_lens.core the answer is: an uncertainty with an honest sample size, a gauge status, a calibration reference, a provenance record, and a computed trust level. This subsystem is the epistemics layer, and it pulls only numpy, so from reward_lens.core import ... never loads torch.

The design and the plain-words tour of these objects live in the anatomy of evidence and a measurement you can trust. This page is the exact surface.

The evidence object

Evidence is a frozen, generic dataclass. You never build one field by field; make_evidence computes the trust level for you from the calibration and registration facts, so the value and its credentials cannot drift apart.

# class

Evidence

Source

reward_lens.core.evidence.Evidence

@dataclass(frozen=True)

Bases: Generic[T]

The universal typed measurement return value (section 2.1.2).

Construct via make_evidence, which computes the content-derived id and the gate-computed trust level. The trust level is not a constructor argument on purpose: it is a function of the calibration reference and the registration status, and allowing a caller to set it would defeat the gates.

Attributes

Name	Type	Default
`id`	`EvidenceID`
`observable`	`str`
`observable_version`	`str`
`subject`	`SubjectRef`
`value`	`T`
`uncertainty`	`Uncertainty`
`gauge`	`GaugeStatus`
`calibration`	`CalibrationRef \| None`
`trust`	`TrustLevel`
`provenance`	`Provenance`
`created_at`	`str`

Properties

is_calibrated : bool

Methods

envelope(sidecar_dir: Any = None) -> dict[str, Any]

src

The JSON-serializable store envelope for this Evidence.

# func

make_evidence

Source

reward_lens.core.evidence.make_evidence

make_evidence(
    *,
    observable: str,
    observable_version: str,
    subject: SubjectRef,
    value: T,
    uncertainty: Uncertainty | None = None,
    gauge: GaugeStatus = GaugeStatus.INVARIANT,
    calibration: CalibrationRef | None = None,
    provenance: Provenance | None = None,
    registered: bool = False,
    adjudicated: bool = False,
    created_at: str | None = None
) -> Evidence[T]

Build an Evidence, computing its content id and gate-derived trust level.

The id hashes the observable, subject, value, gauge, calibration, and provenance (excluding the wall-clock timestamp) so identical measurements from identical inputs share an id, which is what makes the store a deduplicating DAG. Trust is computed by compute_trust; passing registered=True (the study runner does this) yields REGISTERED, a calibration reference yields at least CALIBRATED, and the two together with adjudicated=True yield ADJUDICATED.

The uncertainty is where the honesty lives. n_effective can fall far below n when rows are clones of each other, and method records how the interval was built, down to the bootstrap-CLONE-INFLATED stamp when someone opts into resampling correlated rows.

# class

Uncertainty

Source

reward_lens.core.evidence.Uncertainty

@dataclass(frozen=True)

The uncertainty of a measurement (section 2.1.2).

n is the nominal row count; n_effective is the lineage-aware effective sample size (section 2.10.1) which, on a dataset of clones, is far smaller than n and is the structural death of v1’s fake-n failure class. seed_spread is the cross-seed standard deviation where a quantity is measured over multiple seeds. method names how the interval was produced (“bootstrap-bca”, “analytic”, “conformal”, and crucially “bootstrap-CLONE-INFLATED” when a caller opted into resampling across clones).

Attributes

Name	Type	Default
`ci_low`	`float \| None`	`None`
`ci_high`	`float \| None`	`None`
`ci_level`	`float \| None`	`None`
`n`	`int \| None`	`None`
`n_effective`	`float \| None`	`None`
`seed_spread`	`float \| None`	`None`
`method`	`str`	`'none'`

Trust and gauge

Trust is an ordered ladder, computed by the gates and never set by the caller. REGISTERED outranks CALIBRATED outranks EXPLORATORY; ADJUDICATED is the top rung.

# class

TrustLevel

Source

reward_lens.core.types.TrustLevel

Bases: enum.IntEnum

The trust ladder the three gates compute (section 1.3).

Ordered so that comparisons and max express “the highest applicable rung”. The level is never set by a caller; it is computed from whether the Evidence carries a calibration reference, whether it was produced under a frozen study, and whether it survived its kill criteria and review. See reward_lens.core.gates.compute_trust.

Members

Name	Value
`EXPLORATORY`	`0`
`CALIBRATED`	`1`
`REGISTERED`	`2`
`ADJUDICATED`	`3`

Gauge status says whether a number is a fact or a coordinate. INVARIANT survives a change of basis. COVARIANT is the “needs a frame” rung: it means nothing across models until a shared frame is supplied, and the comparison gate raises without one. RAW_ONLY is a bare coordinate.

# class

GaugeStatus

Source

reward_lens.core.types.GaugeStatus

Bases: enum.Enum

How an Observable’s value transforms under the reward gauge group (I3, gate 2).

INVARIANT quantities are safe to compare across signals directly. COVARIANT quantities (directions, angles, subspace overlaps) require a shared Frame to compare and the comparison APIs take a frame argument with no default. RAW_ONLY quantities are computable and scientifically interesting (E19 proved this) but are typed as raw coordinates and rendered as such, never mistaken for invariant.

Members

Name	Value
`INVARIANT`	`'invariant'`
`COVARIANT`	`'covariant'`
`RAW_ONLY`	`'raw_only'`

Provenance and identity

A calibration reference is the token that lifts a measurement off the exploratory floor: it names the scorecard and the regime the instrument was validated on.

# class

CalibrationRef

Source

reward_lens.core.gates.CalibrationRef

@dataclass(frozen=True)

A reference to the scorecard entry that calibrates a measurement (gate 1).

scorecard_entry is the Evidence id of the answer-key ROC that certifies this observable on this signal family and data regime. regime_match is a free-text note on how closely the calibrated regime matches the production regime, so a mismatch (calibrated on 0.5B organisms, applied to an 8B model) is visible rather than assumed away. organism_family names the planted-structure family the scorecard was built from.

Attributes

Name	Type	Default
`scorecard_entry`	`str`
`organism_family`	`str`
`regime_match`	`str`	`'exact'`
`operating_point`	`dict[str, float] \| None`	`None`

SubjectRef and ModelFP say what was measured; Provenance and Cost say how, recording the git sha, the seeds, the parent evidence, and the metered compute.

# class

SubjectRef

Source

reward_lens.core.types.SubjectRef

@dataclass(frozen=True)

The subject of a measurement (section 2.1.2).

Names the signal(s) by fingerprint, the dataset, the readout, the frame (for covariant quantities), and any interventions applied, by fingerprint. Recording intervention fingerprints here is what makes an erased-model card structurally unable to masquerade as a base-model card: the interventions are part of the subject’s identity.

Attributes

Name	Type	Default
`signals`	`tuple[ModelFP, ...]`	`()`
`dataset`	`DatasetID \| None`	`None`
`readout`	`str \| None`	`None`
`frame`	`FrameID \| None`	`None`
`interventions`	`tuple[str, ...]`	`()`
`extra`	`dict[str, Any]`	`field(default_factory=dict)`

# attr

ModelFP

Source

reward_lens.core.types.ModelFP

ModelFP = NewType('ModelFP', str)

No docstring. See the source for detail.

# class

Provenance

Source

reward_lens.core.provenance.Provenance

@dataclass(frozen=True)

The full provenance envelope carried by every Evidence.

config_hash is a content hash of whatever configuration object produced the run; seeds records every RNG seed used; oracle_calls lists OracleCall ids for any LLM-derived input (R10); parents lists the Evidence ids this quantity was derived from (I5). study is the frozen StudyID when the run was registered (gate 3), else None.

Attributes

Name	Type	Default
`git_sha`	`str`	`'unknown'`
`config_hash`	`str \| None`	`None`
`seeds`	`tuple[int, ...]`	`()`
`cost`	`Cost`	`field(default_factory=Cost)`
`oracle_calls`	`tuple[str, ...]`	`()`
`parents`	`tuple[EvidenceID, ...]`	`()`
`study`	`str \| None`	`None`
`extra`	`dict[str, Any]`	`field(default_factory=dict)`

# class

Cost

Source

reward_lens.core.provenance.Cost

@dataclass(frozen=True)

The compute a measurement consumed (R13).

Metered on every Evidence so studies and cards can report their total cost, and so the Atlas at population scale is budgetable. gpu_seconds is wall-clock GPU time; tokens counts forward-pass tokens; wall_seconds is total wall time including CPU work.

Attributes

Name	Type	Default
`gpu_seconds`	`float`	`0.0`
`tokens`	`int`	`0`
`wall_seconds`	`float`	`0.0`

The gates

Two of the three gates live here as plain functions. compute_trust is gate 1: it reads the calibration and registration facts and returns the trust level, downgrading to EXPLORATORY when there is no scorecard rather than raising.

# func

compute_trust

Source

reward_lens.core.gates.compute_trust

compute_trust(
    *,
    calibration: CalibrationRef | None,
    registered: bool,
    adjudicated: bool = False
) -> TrustLevel

Compute the trust level of an Evidence from the gate inputs (section 1.3).

The trust level is never set by a caller. It is derived from three facts the gates establish: whether a calibration reference is present (gate 1), whether the run happened under a frozen study (gate 3), and whether the study runner has determined the result survived its own kill criteria and review.

The ladder is EXPLORATORY < CALIBRATED < REGISTERED < ADJUDICATED. REGISTERED ranks above CALIBRATED deliberately: a preregistered prediction is a stronger epistemic claim than a calibrated but exploratory measurement. A REGISTERED result that lacks calibration still carries calibration: None on the Evidence and renders as unvalidated on cards, so the two axes stay independently visible even though the headline trust level is a single rung.

ADJUDICATED requires the full set: registered, calibrated, and adjudicated. A caller that passes adjudicated=True without the other two gets the highest rung the facts actually support, never ADJUDICATED on the strength of the flag alone.

require_frame_for_comparison is gate 2: it raises GaugeError when a COVARIANT quantity is asked to compare across subjects with no frame. This is the check that stops a raw cross-model cosine from being reported as if it meant something. The ladder as a whole is documented on the trust ladder.

# func

require_frame_for_comparison

Source

reward_lens.core.gates.require_frame_for_comparison

require_frame_for_comparison(gauge_status: GaugeStatus, frame: FrameID | None) -> None

Enforce gate 2 at a cross-signal comparison site.

A COVARIANT quantity (a direction, an angle, a subspace overlap) is only comparable across signals in a shared frame. If the comparison is attempted without one, this raises GaugeError rather than returning a number that conflates a coordinate change with a functional change, which is exactly the E19 cos = 0.005 failure. INVARIANT quantities need no frame; RAW_ONLY quantities are allowed through but the caller is responsible for typing the resulting Evidence as raw.

The store

Every measurement lands in an append-only store: a JSONL file you can append to, read, and search, but never edit or delete. It refuses derived evidence whose parents it cannot resolve, so the provenance graph stays whole. The design is on the evidence store.

# class

EvidenceStore

Source

reward_lens.core.store.EvidenceStore

EvidenceStore(path: str | Path | None = None)

A directory-backed, append-only store of Evidence.

Not thread-safe across processes (files are the interface; use one writer), but guarded by a lock within a process. The in-memory index maps id to envelope for O(1) lookup; it is built once on construction by streaming the JSONL.

Methods

append(evidence: Evidence[Any]) -> EvidenceID

src

Append an Evidence, returning its id. Idempotent on content-derived ids.

Enforces the DAG invariant: if the Evidence declares parents, every parent must already be in the store, else ProvenanceError. Re-appending an id that is already present is a no-op (the content is identical by construction), so replaying a study is safe.

get(ev_id: str) -> Evidence[Any]

src

Load an Evidence by id, reconstructing its typed payload.

find(
    observable: str | None = None,
    signal: str | None = None,
    dataset: str | None = None,
    readout: str | None = None,
    study: str | None = None,
    min_trust: TrustLevel | None = None,
    latest: bool = False
) -> list[Evidence[Any]]

src

Query the store with simple structural filters.

signal matches an Evidence whose subject names that model fingerprint. latest collapses to the most recently created Evidence per (observable, subject) key, which is the common “give me the current value” query. All filtering is over the in-memory index; for ad hoc analysis use frame to get a pandas DataFrame of envelopes.

frame() -> Any

src

Return a pandas DataFrame of flattened envelopes for ad hoc queries.

parents(evidence: Evidence[Any]) -> list[Evidence[Any]]

src

Return the immediate parent Evidence, raising if any is unresolvable.

ancestors(evidence: Evidence[Any]) -> list[Evidence[Any]]

src

Return the transitive closure of parents (the full derivation DAG of a quantity).

# func

default_store

Source

reward_lens.core.store.default_store

default_store() -> EvidenceStore

Return the process-wide default store (under the configured store path).