reward-lens

Start here

From nothing to a real reading.

pip install reward-lens copy copied select to copy

installs 3.0.0

bash 
python -c "import reward_lens; print(reward_lens.__version__)"

Output

3.0.0

Start with your run.

The capability report reads what your record and your access can actually support. It names which questions are answerable, what each one costs, and which information was never recorded, before you spend anything.

bash 
reward-lens capabilities --substrate NEURAL_GEN

Output

ACCESS RESOLVED
  TASK      NONE                         (no task refs in the record and no environment source
                                         supplied)
  GRADER    NONE                         (no grader endpoint supplied; --grader would add QUERY once
                                         probed)
  POLICY    NONE                         (no weights supplied; --policy would add FORWARD)
  RECORD    NONE                         (no record supplied; --record points at a run directory)
  SUBSTRATE NEURAL_GEN                   (declared by the caller)
  PHASE     PRE_RUN                      (no record supplied, so nothing has been optimised that
                                         this could be after)

REGIME MEASURED
  not measured. Every estimator with a non-empty envelope is reported below with
  its envelope unchecked rather than satisfied; measure the regime to settle them.

The first reading

This runs as pasted. It reads one quantity off a scoring program and returns either a value with its trust level or a refusal naming the input it needed. No GPU, no model download, no network.

grader.py 
def grade(answer: str, gold: str) -> float:
    """One point for an answer that matches the gold answer once trimmed."""
    if answer.strip() == gold.strip():
        return 1.0
    if answer.strip().lower() == gold.strip().lower():
        return 0.5
    return 0.0
plan.py 
from reward_lens.measure.card import CardInputs, card_plan

plan = card_plan(CardInputs(grader_name="grader.py"))
print(plan.render())

Output

CARD PLAN  grader.py
  0 of 13 fields would read; 13 would refuse.
  cost  at least 0 grader calls, no GPU and no model
  not checked  access
  not checked  substrate
  not checked  phase
  not checked  envelope (regime not measured)
  not checked  limit of detection

It will refuse, and that is the system working

More than half the fields on a real card refuse. A refusal is a value rather than an error: it carries the reason, the input that was missing, and a remedy written as an instruction. The first one is startling and then it is the most useful thing the library gives you, because it tells you what you would need in order to make the measurement real.

verifier.decision_coverage

ACCESS_INSUFFICIENT declined

No estimator for verifier.decision_coverage exists on this install, so nothing was computed.

remedy

install the optional extra: pip install 'reward-lens[verifier]'. Every other field on this card reads without it.

Nothing here is broken when this happens. An instrument that returned a number anyway would be the thing to worry about.

bash 
pip install 'reward-lens[verifier]'
read.py 
from reward_lens.measure.card import CardInputs, grader_card, render_card
from reward_lens.verifier import ListCorpus, Rollout, VerifierUnderTest

corpus = ListCorpus.of([
    Rollout("r1", {"answer": "42", "gold": "42"}, score=1.0),
    Rollout("r2", {"answer": "41", "gold": "42"}, score=0.0),
    Rollout("r3", {"answer": " Paris ", "gold": "paris"}, score=0.5),
])

card = grader_card(
    CardInputs(
        verifier=VerifierUnderTest("grader.py", entrypoint="grade"),
        corpus=corpus,
        grader_name="grader.py",
        mutation_limit=8,
    )
)
print(render_card(card))

Output

  2 of 13 fields read and 11 refused. The lowest trust among them is exploratory.

  coverage                   Across 3 rollouts grade has never taken 0% of its branches (0 of 4). Those are behaviours it cannot distinguish.  [exploratory]
                             a random sample of the same size reaches 0.777 against the corpus's 1.000
  surviving mutants          Of 8 mutants of grade, 0 survive: 0 ways it could be wrong that no rollout in your corpus would reveal.  [exploratory]
                             mutation score 1.000 over 8 competent
                             reproducers withheld. this payload enumerates concrete ways to make the grader wrong. It is written to the store with a sensitive flag, excluded from rendered artifacts unless explicitly requested, and not published without a recorded decision.

Where to go next