API reference

Concepts

Does a named idea actually move the reward, and by how much per unit of it? reward_lens.concepts builds a direction for a concept from labelled examples, measures how aligned that direction is with the reward, and measures the causal slope of reward against a dose of it. The narrated instrument is concept dose-response.

Directions and alignment

concept_direction is the difference of means between positive and negative activations. reward_alignment is its cosine with the reward direction $w_r$ , reported RAW_ONLY because a bare cosine is a coordinate, not a fact. dose_response_slope is the causal complement: the OLS slope of reward against how far you push the activation along the concept.

# func

concept_direction

Source

reward_lens.concepts.vectors.concept_direction

concept_direction(pos: torch.Tensor, neg: torch.Tensor) -> np.ndarray

Unit-normalized mean-difference concept direction from paired activations.

pos and neg are (n, d_model) activations at a site for the positive and negative side of n concept pairs. The direction is mean(pos - neg) over the pairs, normalized to unit length; a degenerate (near-zero) mean returns the raw mean rather than dividing by a vanishing norm. Returns an fp32 numpy vector (d_model,).

# func

reward_alignment

Source

reward_lens.concepts.vectors.reward_alignment

reward_alignment(direction: np.ndarray, w_r: torch.Tensor) -> float

Cosine between a concept direction and the reward direction (raw coordinates).

Both vectors are normalized before the dot product, so the result is a cosine in [-1, 1]. A positive value means the concept pushes reward up; a negative value means it pushes reward down. This is a RAW_ONLY quantity (it depends on the residual-stream basis) and is only meaningful within a single signal.

# func

dose_response_slope

Source

reward_lens.concepts.vectors.dose_response_slope

dose_response_slope(doses: np.ndarray, rewards: np.ndarray) -> float

Ordinary-least-squares slope of reward against concept dose.

doses are the intervention strengths applied along a concept direction and rewards the resulting scalar rewards. The slope is d reward / d dose from a least-squares line, which is the dose-response summary the E08 study reports. A constant dose vector (no variation) returns 0.0 rather than dividing by a zero variance.

Calibrated probes

fit_probe trains a linear probe with seed-grouped cross-validation and binds a scorecard automatically. A probe that was never calibrated returns an uncalibrated direction, and anything built on it inherits EXPLORATORY trust, so an unvalidated concept cannot launder itself into a confident number downstream.

# func

fit_probe

Source

reward_lens.concepts.probes.fit_probe

fit_probe(
    signal: SiteCaptures | RewardSignal,
    view: Any = None,
    target: Callable[[Any, str], int | None] | None = None,
    sites: tuple[Site, ...] | None = None,
    *,
    cv: int = 5,
    name: str | None = None,
    method: str = 'probe_lr',
    l2: float = 1.0,
    class_balance: bool = True,
    solver: str = 'numpy',
    answer_key: AnswerKey | None = None,
    store: EvidenceStore | None = None,
    seed: int = 0,
    target_tpr: float = 0.9,
    target_fpr: float = 0.05
) -> ProbeFit

Train a linear concept probe with grouped CV and automatic scorecard binding (section 2.5.2).

signal is either a SiteCaptures (the substrate-free path the proofs run on: activations, labels, and seed groups already in hand) or a live RewardSignal, in which case view (a DataView of pairs) and target (a (item, side) -> label callable) are captured into a SiteCaptures at sites first. The probe is a logistic readout fit per site with seed-grouped cross-validation, so no clone is ever scored by a model that trained on it; the site with the highest out-of-fold AUC becomes the returned Direction.

Scorecard binding is automatic and honest. If an answer_key is supplied (or carried on the captures), the out-of-fold scores are graded against it and the resulting CalibrationRef is attached (the direction becomes CALIBRATED). If none is available the direction carries calibration: None and stays EXPLORATORY: the gap is visible, never papered over.

Parameters

Name	Type	Default	Description
`signal`	`SiteCaptures \| RewardSignal`	required	A `SiteCaptures`, or a `RewardSignal` to capture from (with `view` and `target`).
`view`	`Any`	`None`	The DataView of pairs, when capturing from a signal.
`target`	`Callable[[Any, str], int \| None] \| None`	`None`	A `(item, side) -> int \| None` labeler, when capturing from a signal.
`sites`	`tuple[Site, ...] \| None`	`None`	Sites to sweep; defaults to every site in the captures.
`cv`	`int`	`5`	Number of seed-grouped CV folds.
`name`	`str \| None`	`None`	The concept name; defaults to the target’s or captures’ name.
`method`	`str`	`'probe_lr'`	Stored on the direction (`"probe_lr"` by default).
`l2`	`float`	`1.0`	Ridge strength on the logistic weight.
`class_balance`	`bool`	`True`	Weight the minority class up by inverse frequency.
`solver`	`str`	`'numpy'`	`"numpy"` (deterministic IRLS) or `"auto"` (sklearn, numpy fallback).
`answer_key`	`AnswerKey \| None`	`None`	The organism `AnswerKey` to grade against for calibration (gate 1).
`store`	`EvidenceStore \| None`	`None`	If given, the direction Evidence (and its scorecard parent) are appended.
`seed`	`int`	`0`	Seed for the CV shuffle.
`target_tpr`	`float`	`0.9`	The true-positive rate of the operating point the scorecard states its detection at.
`target_fpr`	`float`	`0.05`	The false-positive rate of that same operating point.

Returns

Type	Description
`ProbeFit`	A `ProbeFit` whose `direction` is the persisted section 2.5.1 `Direction`.

Difference dictionaries

train_diff_dict learns a dictionary of difference vectors that decomposes a reward margin into named, human-readable parts.

# func

train_diff_dict

Source

reward_lens.concepts.diff_dict.train_diff_dict

train_diff_dict(
    delta_h_train: np.ndarray,
    w_r: np.ndarray,
    n_atoms: int,
    *,
    method: str = 'svd',
    train_data: DatasetID | None = None,
    meta: dict[str, Any] | None = None
) -> DiffDictionary

Fit a preference-difference dictionary of n_atoms atoms from training delta_h (section 2.5.3).

The atoms are the top n_atoms right singular vectors of delta_h_train (an orthonormal basis for its dominant preference-difference subspace), so the reconstruction of any delta_h is its orthogonal projection onto that subspace and is exact whenever delta_h lies in it. Each atom’s margin contribution w_r . d_i is computed once and stored, which is what makes the margin decomposition a lookup rather than a refit. Returns a persisted-ready DiffDictionary.

Choosing n_atoms at least the rank of the training delta_h makes the identity exact on data drawn from the same subspace; choosing fewer truncates the dictionary and the trainer will report the honest residual at verification time.

The 1.0 concept tool

ConceptExtractor is the original 1.0 interface, kept for the workflows built on it. It extracts concepts, reports their reward alignment, and intervenes on them through the older API.

# class

ConceptExtractor

Source

reward_lens.concepts.legacy.ConceptExtractor

ConceptExtractor(model: RewardModel)

Extract and analyze concept vectors from reward model activations.

This enables understanding which abstract concepts (confidence, formality, agreement, etc.) influence reward scores, and identifying potential reward hacking vulnerabilities.

Parameters

Name	Type	Default	Description
`model`	`RewardModel`	required	A RewardModel instance.

Methods

extract_concepts(
    concept_pairs: dict[str, list[tuple[str, str, str]]],
    layer: Optional[int] = None,
    max_length: int = 2048,
    show_progress: bool = True
) -> dict[str, torch.Tensor]

src

Extract concept direction vectors from contrastive pairs.

For each concept, the direction is computed as the average difference between positive and negative example activations.

analyze_reward_alignment(
    concept_vectors: dict[str, torch.Tensor],
    alignment_threshold: float = 0.3,
    hacking_concepts: Optional[list[str]] = None
) -> ConceptAlignmentReport

src

Analyze how concepts align with the reward direction.

Concepts that strongly align with reward but represent surface properties (like confidence or verbosity) may indicate hackable biases.

intervene_on_concept(
    prompt: str,
    response: str,
    concept_vector: torch.Tensor,
    strength: float = 1.0,
    layer: Optional[int] = None,
    max_length: int = 2048
) -> float

src

Intervene on a concept and measure reward change.

Adds or subtracts the concept direction from activations to see how it affects reward.