Signals

The substrate abstraction: any reward signal with white-box access (section 2.3.2).

Eight adapters implement this (classifier, judge, process, implicit, rubric, trajectory, dense, ensemble). An Observable written against this protocol runs on all of them; a new grader paradigm the field invents becomes a new adapter passing the conformance suite, and the whole battery, every index, and every science become available to it for free (the extensibility contract, section 5.3).

caps declares capabilities (R3); readouts lists the readouts the signal exposes; score and score_prefixes return Evidence; capture returns a handle to activations (possibly cached); with_interventions returns a wrapped signal any Observable accepts unchanged; tokenize owns span carry-through.

A first-class readout: what to read, where, and how (section 2.3.1).

name is the human key (“reward”, “verdict”, “criterion:coherence”, “quantile:0.9”). For linear and logit_diff readouts, vector is the direction w (fp32) the scalar is projected onto; for simplex (Likert over score tokens) and token_value (per-token value) the meaning is carried in meta. site is where the readout reads, usually the final residual stream; position says at which token(s). This object is the pivot of R4: it is architecturally resolved by adapters at load time (w_r is read off the checkpoint, not probe-extracted), so there is no “is this feature real” regress at the readout (I1, property 2 in section 1.1).

A resolvable specification of which token positions a readout reads (section 2.3.1).

Nothing in the kernel hardcodes “final token” (R4). A classifier head reads at final; a process reward model reads at step_ends; a generative judge reads at judgment (the verdict token, whose location the signal detects and validates); a dense reward reads at all. detail carries the kind-specific configuration (the span kind for span_ends, the explicit indices for explicit, the delimiter config or detected index for judgment).

Build a ClassifierRM from an already-loaded model + tokenizer (no download).

Resolves the adapter (v1 dispatch) if not given, builds the SiteMap and numerics policy, fingerprints the model, reads the reward head into readouts, and runs a fast conformance quick-check (determinism plus readout-matches-head on two trivial inputs) unless disabled. This is the constructor the tests and the tiny-model path use, and the one the hub loader ends at once it has a model in hand.

Construct the tiny synthetic ClassifierRM the tests run on, entirely offline.

A real LlamaForSequenceClassification (hidden 32, 2 layers, 4 heads, num_labels=1) so the adapter, hooks, readout, grad, and hvp see the same module tree they will see on an 8B Skywork model; only the magnitudes differ. The tokenizer defaults to gpt2 (cached, fast, offset-capable); if gpt2 cannot be loaded offline, a minimal byte-level tokenizer is used so the tests still run with no network (section: hardware reality).

Load a signal from an HF id, a local path, or a SignalSpec (section 2.3.4).

Sniffs the loading convention from the config architecture and head names, chooses the adapter and numerics policy, then loads the weights and delegates to wrap_hf_model. The weight load for the 8B/27B campaign models is GPU/download-gated: unless allow_download=True is set (and the machine can hold the model), this raises a clear, marked error rather than attempting a multi-gigabyte download on hardware that cannot run it. Ambiguous conventions raise with the candidate list, never a silent guess.

A sequence-classification reward model as a RewardSignal (section 2.3.3).

Build it through signals.loaders.wrap_hf_model / from_tiny (which resolve the adapter, site map, head, policy, and fingerprint); direct construction is for the loader and tests. The signal holds a HFRuntime, the numerics policy, the readouts read off the head, and the tokenizer. caps declares its capabilities (R3); meta carries the fingerprint, lineage, template, and numerics policy.

An LLM-as-judge as a RewardSignal (section 2.3.3, adapter 2).

Build it through from_tiny (the offline test vehicle) or from_causal_lm (an already-loaded instruct model). The verdict readouts are logit_diff directions read off the LM head; the Likert readout is a simplex over score tokens. score dispatches on the readout kind: a projection for the logit-diff verdicts, an expected value under the score-token softmax for the Likert.

A step-level reward model as a RewardSignal (section 2.3.3, adapter 3).

Build it through from_tiny or from_sequence_classifier. score returns the outcome scalar (the final-token reward of the whole solution, identical to a classifier); step_scores returns the per-step reward vector read at the detected step boundaries. The step delimiter is configurable; the default is a newline.

The DPO implicit reward r-hat = beta * sum log-ratio as a RewardSignal (adapter 4).

Constructed from a policy model and a reference model (from_models / from_tiny), both CausalLM with a shared tokenizer. score returns r-hat; score_prefixes returns the cumulative per-token reward whose last entry equals r-hat; per_token_rewards returns the raw per-token decomposition (the increments). self.runtime is the policy runtime; the reference runtime is self.reference_runtime and capture(..., namespace="ref") routes to it.

A rubric grader as a RewardSignal (section 2.3.3, adapter 5).

Build it through from_tiny (a tiny multi-label classifier) or from_sequence_classifier (a real multi-objective head). readouts exposes one criterion:<name> per criterion plus a weighted reward aggregate; score projects the final hidden state onto whichever direction the readout names. The rubric spec is on self.spec.

A trajectory-level reward model as a RewardSignal (section 2.3.3, adapter 6).

Build it through from_tiny or from_sequence_classifier. tokenize accepts a Trajectory (rendering its steps with receipt/narrative/action span typing) or a plain (prompt, response) item (the classifier rendering, so the conformance suite’s generic checks apply). score reads the reward at the trajectory scoring position (the final token).

Per-token reward maps from any outcome signal, shipped GATED (section 2.3.3, adapter 7).

Wraps an outcome RewardSignal and adds dense_rewards: the first difference of the wrapped signal’s prefix-score curve, a per-token attribution that sums to the outcome score. Every other protocol method delegates to the wrapped signal, so the same battery reaches the dense extractor unchanged. dense_rewards Evidence is always EXPLORATORY (no calibration is ever attached), which is how the design enforces “certify before you trust” for dense credit assignment.

A composite of several member signals (section 2.3.3, adapter 8).

score scores every member under the requested readout, stacks the results, and reduces them by mode (mean, min, max, or a quantile at q). The Evidence subject names every member by fingerprint (member provenance), and the ensemble’s own fingerprint is derived from the members’ so two ensembles over the same members are the same subject. caps is the intersection of the members’ capabilities, plus DISTRIBUTIONAL for a quantile composite.

A distributional wrapper exposing a multi-row head’s rows as quantile readouts (adapter 8).

Wraps a signal whose head rows are quantile levels (a QRM) and presents them as quantile:tau readouts, declaring DISTRIBUTIONAL. score(view, "quantile:0.9") returns the 0.9-quantile row; median, quantile, and mean are convenience reductions over the levels. The wrapped signal does the forward work; this class only relabels and reduces.

Run the full conformance suite against a signal (section 2.3.6).

tol defaults to the signal’s numerics-policy tolerance (1e-4). Each check is wrapped so an exception becomes a failed (or, for the dtype matrix, a skipped) check rather than aborting the suite, so the report always enumerates every invariant. Returns a ConformanceReport whose passed property is the M1 acceptance gate.

Run the applicable conformance checks against one adapter (section 2.3.6).

items is the stimulus set (varied in length so batching forces left-padding). readout names the readout to score under (the adapter’s default when None). check_head runs the readout-vs-head check for adapters with a linear/logit_diff readout; pass False for composites. probe is (item, needle, kind) for the template round-trip: the item must tokenize with a span of kind covering needle in the rendered text. probe_inject handles adapters that carry explicit item spans rather than auto-typing them (a rubric grader): the check renders the item, locates needle, injects an explicit char span over the rendered text, and re-tokenizes. Each check is wrapped so a raised exception becomes a recorded failure rather than aborting the suite.