Install

What has to be on the machine before any of this runs? Less than you would guess. The measurement half of the library is pure numpy and opens on any laptop. The model half wants weights and, for the real 8B graders, a GPU. You can learn the whole thing on the CPU half first.

pip install reward-lens

Python 3.10 or newer. Two extras, both optional:

  • pip install "reward-lens[sae]" adds the sparse-autoencoder tooling for feature-reward alignment.
  • pip install "reward-lens[dev]" adds the test suite and tooling for a source checkout.

Half the library never imports torch

The epistemics layer, the part that computes evidence, uncertainty, effective sample size, and trust, is written in numpy and imports nothing from torch. You can confirm that in three lines:

import sys
import reward_lens, reward_lens.core, reward_lens.stats

reward_lens.__version__     # '2.0.1'
"torch" in sys.modules      # False, nothing model-touching was pulled in
"numpy" in sys.modules      # True

Anything that actually reads a model imports from its own subsystem, from reward_lens.signals import ..., and only then does torch load. So a continuous-integration box, a notebook on a plane, a reviewer checking your arithmetic: none of them need a GPU to run the discipline.

What actually bites, before you load a real model

  • You need the weights. reward-lens hooks a live model in memory and reads the reward head directly, keeping it in fp32 so the readout matches the head exactly. Anything you can only reach through an API, with no weights to hold, is out of scope. If transformers can load it, an adapter can open it up.
  • The good graders are gated. Skywork and ArmoRM sit behind a license click on the Hugging Face Hub. Accept the terms on the model page, then huggingface-cli login with a token, or the load returns a 401 and nothing else.
  • An 8B trace wants a GPU. An 8B reward model in fp32 does not fit an 8 GB card, and the campaign numbers on this site were measured on larger hardware and committed as artifacts. That is a hardware fact, not a soft limit: the 8B code path names the call it would make and refuses rather than pretend. Everything conceptual runs first on the CPU through from reward_lens.signals import from_tiny, which builds a real (small) reward model with no download.

Where to start

Install done, the fastest next step is one of the two quickstarts on the getting-started page: one opens a real grader end to end, the other runs the whole arc on the CPU with from_tiny. If your first question is instead whether your particular grader is even supported, a classifier head or a DPO checkpoint or an LLM judge, that is answered on models and signals. Adding a model family the adapters do not yet cover is one small class.