reward-lens

Certified robustness, attack surface and eval-awareness

The RM knows it is being tested, and steering along that direction inflates reward

mixed registered

The reading

  1. a benchmark-versus-organic direction is decodable

    real_probe_balanced_acc > 0.6 0.6 The RM knows it is being tested, and steering along that direction inflates reward mixed registered > 0.6 The frozen study "The RM knows it is being tested, and steering along that direction inflates reward", adjudicated on 19 July 2026 at the registered level of the project's evidence ladder. One frozen study, adjudicated once, on evalaware-rms, over evalaware-paired and wildchat-organic and arena-organic. It does not establish that the same holds on another subject, another dataset or another run. Open the record › confirmed

  2. steering along the direction inflates the score

    delta_r_per_steer > 0.05 The RM knows it is being tested, and steering along that direction inflates reward mixed registered > 0.05 The frozen study "The RM knows it is being tested, and steering along that direction inflates reward", adjudicated on 19 July 2026 at the registered level of the project's evidence ladder. One frozen study, adjudicated once, on evalaware-rms, over evalaware-paired and wildchat-organic and arena-organic. It does not establish that the same holds on another subject, another dataset or another run. Open the record › refuted

  3. the probe does not spuriously flag organic prompts (equivalence bound, an affirmative null)

    organic_fpr < 0.1 The RM knows it is being tested, and steering along that direction inflates reward mixed registered < 0.1 The frozen study "The RM knows it is being tested, and steering along that direction inflates reward", adjudicated on 19 July 2026 at the registered level of the project's evidence ladder. One frozen study, adjudicated once, on evalaware-rms, over evalaware-paired and wildchat-organic and arena-organic. It does not establish that the same holds on another subject, another dataset or another run. Open the record › refuted

EVAL-AWARE against its frozen predictions: mixed.
EVAL-AWARE against its frozen predictions: mixed.

Provenance

card
EVAL-AWARE
frozen_at
adjudicated_at
spec_hash
spec:3671c68e023554a042cd340ae8dded8e
study_id
study:campaign-eval-aware@v1#3671c68e
Evidence records
54fb272fd011c8ca07a4959f553341ab
spec_file
campaign-eval-aware.json