Dynamics and loops

An ordered, fingerprint-chained sequence of checkpoints over training time (DESIGN 2.12).

Build one with CheckpointSequence.build from (step, ModelFP, loader) triples; the constructor computes the hash chain so the sequence is self-verifying from the moment it exists. Iterate it to sweep an Observable or index across training time (sweep.sweep_over_checkpoints); the sequence’s signature is what keys a resumable sweep to this exact chain.

Verification comes in two depths. verify_chain recomputes the links and is pure and instant: it catches an edited fingerprint, a reordered or dropped checkpoint, or a broken link. verify_fingerprints additionally loads each model and recomputes runtime.fingerprint, catching weights swapped under an unchanged record. verify runs the chain check and, when deep=True, the fingerprint check, raising ProvenanceError on the first failure so a study cannot sweep a chain it has not trusted.

The per-probe effect-size-versus-training-step curve, the bias-entry order (DESIGN 2.12).

Sweeps the reward score across the checkpoint sequence (cached and resumable through sweep), then for each probe computes the signed effect size of the bias on the reward at every checkpoint. A bias that is uncorrelated with the reward early and strongly loaded late traces a rising curve; the step at which it first crosses entry_threshold is where it entered. On the planted synthetic sequence, whose reward loading onto the probe grows by construction, the curve is monotone rising, which is the calibration that the estimator faithfully tracks a known developmental signal before it is trusted on a real run (DESIGN 2.10).

The E04 attribution-vs-patching rho as a function of training step (DESIGN 2.12, 4.4 M9).

score_pairs is one (step, attribution_scores, patching_scores) triple per checkpoint, where the two arrays are the per-component attribution and patching effects the battery produces. For each checkpoint this computes the Spearman rho with a bootstrap CI (stats.spearman_with_ci) and returns the trajectory. Whether the anti-correlation is developmental is read from the trajectory as a whole: the reward is developmental when the late portion of the run is anti-correlated and meaningfully more anti-correlated than the early portion (an anti-correlation that emerges from near zero, or one that is present early and deepens). The comparison uses the mean rho of the first and last thirds rather than the two endpoint checkpoints, so a single noisy checkpoint cannot set the flag and a constant anti-correlation present from the first checkpoint is not called developmental. The attribution and patching arrays come from the battery on each checkpoint; passing them in keeps this CPU-provable and decoupled from the concurrently-built battery (DESIGN 2.9, 2.12).

The exact KL divergence of the best-of-n policy from the base policy, in nats.

KL(bo_n || base) = log(n) - (n - 1) / n (Beirami et al. 2401.01879). Exact in the continuous-reward, no-ties limit, and a function of n alone. bon_kl(1) == 0 because best-of-1 is the base policy. This is the identity the acceptance test reproduces.

Accepts a scalar or an array of n and returns a float array so a whole ladder is one call.

Compute the best-of-n ladder and the exact KL frontier from scored base-policy samples.

scores_per_prompt is a per-prompt bank of reward-model scores: either a 2-D array (n_prompts, m) or a ragged list of per-prompt arrays. For each n in ns the expected best-of-n reward is computed per prompt with the plug-in estimator and averaged across prompts, and the exact KL(bo_n || base) is attached. The result previews the optimization frontier with no RL (DESIGN 2.13).

Returns Evidence[BoNLadder]. Gauge is INVARIANT: the KL is a real divergence in nats and the reward is in the model’s own score units, both gauge-free (a raw reward-model score is INVARIANT in this kernel, DESIGN 2.3.3). subject names the signal/dataset when the caller has them; parents links the score Evidence this consumed so the store stays a DAG.

The susceptibility spectrum chi_i = Cov_0(f_i, r) from base-policy samples (Appendix A12).

scores is the base-policy reward per sample (shape (N,)); features is the per-sample scalar activation of each feature (shape (N, k)). Returns chi_i = Cov_0(f_i, r), the predicted initial drift of feature i under optimization, computed as a population covariance mean(f_i r) - mean(f_i) mean(r) (the covariance is an expectation under pi_0, so the population form, ddof = 0, is the estimator of the theory object). The f = r diagonal Var_0(r) is reported as teacher_variance (Appendix A3).

Gauge is INVARIANT: a covariance of two scalar readouts does not depend on the activation basis (DESIGN 2.17 step 6). Trust is EXPLORATORY until a planted-chi organism scorecard calibrates it (gate 1); that calibration is a study, not this function’s job.

Names of the predicted hack modes: chi_i > 0 while Cov_0(f_i, gold) <= 0 (A12).

gold_covariance[i] = Cov_0(f_i, gold) for a reference/gold signal. A feature the proxy loves (positive susceptibility) but the gold objective does not is exactly the direction optimization exploits, which is what the forecasting science (S12) pre-registers.

Estimate the critical pressure lambda_c from the reward’s right-tail scale (A4/A5).

Fits an exponential right tail by peaks-over-threshold: the scale tau is the mean excess of the scores above their tail_quantile quantile, and lambda_c = 1 / tau (Appendix A5, an exponential tail with scale tau has critical pressure 1 / tau). This is the cheap scalar the tilt guard needs; the full Hill / peaks-over-threshold tail index lives in the tail index observable (Appendix A4), which this does not replace. Returns inf when the excess is degenerate (a bounded tail imposes no ceiling on lambda).

Emulate the tilted family across a lambda grid by SNIS, with the validity guards.

For each lambda in lambdas the tilted feature means E_lambda[f_i] and reward mean E_lambda[r] are estimated by reweighting the base-policy bank, with no gradient updates. The susceptibility chi_i = Cov_0(f_i, r) (the lambda -> 0 slope) is attached so the initial drift the sweep traces can be checked against the closed-form prediction.

Refuses, via ESSGuardError, when any requested |lambda| exceeds lambda_c / 2 (past that the tilt no longer emulates practical optimization, DESIGN 2.13; past lambda_c there is no tilted optimum for a heavy tail, Appendix A4), or when the SNIS effective sample size at some valid lambda falls below min_ess_frac of the bank. Returning a confident number from a handful of dominating samples is the failure this guard exists to prevent.

Gauge is INVARIANT (tilted means of scalar readouts and their base covariance are gauge-free).

Monitor a rollout in reward-feature space, step by step (DESIGN 2.13, science S13).

Construct with the feature bank whose doses to track, the reward direction w_r the policy is paid to excite, a batch of baseline (step-0) activations that fixes the reference mean and covariance, and optionally an explicit reward-defining subspace (default: the span of w_r). Call observe once per rollout step with that step’s activation batch and rewards, then report to get the DriftReport, or evidence to get it wrapped as Evidence.

Everything is CPU-cheap and pure-numpy. The recorder holds no model; it consumes activations a caller extracts, which is what lets it run in shadow mode on production serving with no behavior change (DESIGN 2.13) and what makes the synthetic organism a faithful stand-in for the GPU rollout.

A changepoint-based onset: which signal moved, when, and how significantly (DESIGN 2.13).

Detect a single mean-shift changepoint by CUSUM with a permutation p-value (Taylor’s method).

Builds the cumulative sum of deviations from the mean, S_j = sum_{t<=j}(x_t - xbar) with S_0 = S_T = 0; the changepoint is the index where |S| is largest, and the magnitude max(S) - min(S) is the test statistic. Significance is a permutation null: shuffle the series n_perm times, recompute the magnitude, and report the fraction at least as large (with the (count + 1) / (n_perm + 1) correction). This is the dependency-light stand-in for the BOCPD detector DESIGN 2.11 puts in stats/changepoint; the return contract is the same.

A flat or trendless series returns a non-significant changepoint (large p-value). The reported index is the split point in [0, T]: samples before it are one regime, samples after another.

Generate a CPU rollout that drifts along a planted hack direction (DESIGN 2.13, science S13).

The policy is paid to excite the reward direction w_r, which here is the hack feature’s direction (feature 0). From dose_onset on, the activation mean drifts along that direction at drift_rate per step, so the hack feature’s dose, the proxy reward, and the crystallization along w_r all climb together. The gold reward is flat until the accumulated hack dose crosses gold_tolerance, then falls at gold_slope: the anti-correlation with gold only bites once enough hack has accumulated (the chi_i > 0, Cov(f_i, gold) <= 0 signature of Appendix A12). That construction is what makes the feature-space onset precede the gold divergence, which is the lead time the recorder must recover. The remaining features are distractors with no systematic drift.

Deterministic given seed. This is the synthetic organism the recorder’s acceptance test runs on; the real RL rollout is GPU-gated and enters through the same RolloutRecorder.observe API.

Wrap a RewardSignal as the reward function a training loop calls (DESIGN 2.13).

Returns reward_fn(prompts, responses) -> list[float]: it pairs each prompt with its response, scores them through signal.score under readout, and returns plain floats, which is the contract PPO/GRPO reward hooks expect. This is the real, framework-agnostic reward path; the framework files add only the callback/worker plumbing around it.

Log the RM’s geometry every k steps on a fixed probe (DESIGN 2.13).

The framework-agnostic core the three integration callbacks wrap: hold a probe, and on each training step call maybe_log(signal, step), which returns a ProbeLog on the logging steps (every probe.k) and None otherwise, appending each log to logs. This is the piece the tiny-model test exercises directly, so the geometry logging is proven without any training framework installed; the callbacks add only the framework’s step hook around it.