ERR-RB2 S16 · Certified robustness, attack surface and eval-awareness

Error anticipation on RewardBench 2

Refuted K-err fired

frozen 2026-07-18 · adjudicated 2026-07-19 · ev:6bf316c059230ff4ff99b118b1664631 · spec:c8ac36e72adcc

The frozen predictions

H-err-qwen internal features beat the exposed seen-pair margin at predicting held-out RB2 completion errors on Qwen3-8B

err_auroc_delta_qwen8b > 0.03 registered p 0.6 Refuted
H-err-llama the same holds on Llama-8B (cross-family)

err_auroc_delta_llama8b > 0.03 Refuted

K-err fired on err_auroc_delta_min_family <= 0 , observed -0.003471 : internals add nothing over the seen-pair margin the RM already exposes when predicting held-out errors, on at least one family. A fired kill criterion is a registered negative result, not a failure of the machinery.

Adjudication figure for ERR-RB2 — ERR-RB2 fired its kill criterion (K-err); the registered negative result.

Registered companions

Subjects and artifacts

signals: skywork-v2-qwen3-8b , skywork-v2-llama31-8b
datasets: rb2-full
frozen spec: campaign-err-rb2.json
figure files: light · dark
study id: study:campaign-err-rb2@v1#c8ac36e7