Susceptibility chi predicts best-of-n feature drift on a real bank

The reading

1. chi predicts realized best-of-n feature-drift ordering on a real bank

   chi_bon_spearman > 0.3 0.6 0.9643 Susceptibility chi predicts best-of-n feature drift on a real bank confirmed registered confidence interval 0.7500000000000002 to 1 > 0.3 The frozen study "Susceptibility chi predicts best-of-n feature drift on a real bank", adjudicated on 19 July 2026 at the registered level of the project's evidence ladder. One frozen study, adjudicated once, on skywork-v2-qwen3-8b and skywork-v2-llama31-8b, over ultrafeedback-bank. It does not establish that the same holds on another subject, another dataset or another run. split-bank estimator: chi and drift read disjoint prompt halves, so the null re-centers at zero, but the correlation's exchangeable unit is k = 6 features and the null still passes the frozen 0.3 threshold 0.28 of the time; the registered CI exclusion carries size control and only orderings near the synthetic calibration (0.958) clear the 0.91 MDE. Open the record › Copy technical provenance confirmed

2. chi beats a base-variance-only baseline

   chi_minus_variance_baseline_spearman > 0 0.5714 Susceptibility chi predicts best-of-n feature drift on a real bank confirmed registered confidence interval 0.07142857142857151 to 0.5714285714285716 > 0 The frozen study "Susceptibility chi predicts best-of-n feature drift on a real bank", adjudicated on 19 July 2026 at the registered level of the project's evidence ladder. One frozen study, adjudicated once, on skywork-v2-qwen3-8b and skywork-v2-llama31-8b, over ultrafeedback-bank. It does not establish that the same holds on another subject, another dataset or another run. split-bank estimator: chi and drift read disjoint prompt halves, so the null re-centers at zero, but the correlation's exchangeable unit is k = 6 features and the null still passes the frozen 0.3 threshold 0.28 of the time; the registered CI exclusion carries size control and only orderings near the synthetic calibration (0.958) clear the 0.91 MDE. Open the record › Copy technical provenance confirmed

Measurements

chi_bon_spearman_ci_low

0.75 Susceptibility chi predicts best-of-n feature drift on a real bank confirmed registered confidence interval 0.7500000000000002 to 1 > 0.3 The frozen study "Susceptibility chi predicts best-of-n feature drift on a real bank", adjudicated on 19 July 2026 at the registered level of the project's evidence ladder. One frozen study, adjudicated once, on skywork-v2-qwen3-8b and skywork-v2-llama31-8b, over ultrafeedback-bank. It does not establish that the same holds on another subject, another dataset or another run. split-bank estimator: chi and drift read disjoint prompt halves, so the null re-centers at zero, but the correlation's exchangeable unit is k = 6 features and the null still passes the frozen 0.3 threshold 0.28 of the time; the registered CI exclusion carries size control and only orderings near the synthetic calibration (0.958) clear the 0.91 MDE. Open the record › Copy technical provenance

chi_bon_spearman_ci_high

1 Susceptibility chi predicts best-of-n feature drift on a real bank confirmed registered confidence interval 0.7500000000000002 to 1 > 0.3 The frozen study "Susceptibility chi predicts best-of-n feature drift on a real bank", adjudicated on 19 July 2026 at the registered level of the project's evidence ladder. One frozen study, adjudicated once, on skywork-v2-qwen3-8b and skywork-v2-llama31-8b, over ultrafeedback-bank. It does not establish that the same holds on another subject, another dataset or another run. split-bank estimator: chi and drift read disjoint prompt halves, so the null re-centers at zero, but the correlation's exchangeable unit is k = 6 features and the null still passes the frozen 0.3 threshold 0.28 of the time; the registered CI exclusion carries size control and only orderings near the synthetic calibration (0.958) clear the 0.91 MDE. Open the record › Copy technical provenance

chi_bon_spearman_second_model

1 Susceptibility chi predicts best-of-n feature drift on a real bank confirmed registered none registered The frozen study "Susceptibility chi predicts best-of-n feature drift on a real bank", adjudicated on 19 July 2026 at the registered level of the project's evidence ladder. One frozen study, adjudicated once, on skywork-v2-qwen3-8b and skywork-v2-llama31-8b, over ultrafeedback-bank. It does not establish that the same holds on another subject, another dataset or another run. Nothing was registered against this value in advance, so it is a supporting measurement rather than a tested one. split-bank estimator: chi and drift read disjoint prompt halves, so the null re-centers at zero, but the correlation's exchangeable unit is k = 6 features and the null still passes the frozen 0.3 threshold 0.28 of the time; the registered CI exclusion carries size control and only orderings near the synthetic calibration (0.958) clear the 0.91 MDE. Open the record › Copy technical provenance

chi_minus_variance_baseline_spearman_ci_low

0.07143 Susceptibility chi predicts best-of-n feature drift on a real bank confirmed registered confidence interval 0.07142857142857151 to 0.5714285714285716 > 0 The frozen study "Susceptibility chi predicts best-of-n feature drift on a real bank", adjudicated on 19 July 2026 at the registered level of the project's evidence ladder. One frozen study, adjudicated once, on skywork-v2-qwen3-8b and skywork-v2-llama31-8b, over ultrafeedback-bank. It does not establish that the same holds on another subject, another dataset or another run. split-bank estimator: chi and drift read disjoint prompt halves, so the null re-centers at zero, but the correlation's exchangeable unit is k = 6 features and the null still passes the frozen 0.3 threshold 0.28 of the time; the registered CI exclusion carries size control and only orderings near the synthetic calibration (0.958) clear the 0.91 MDE. Open the record › Copy technical provenance

chi_minus_variance_baseline_spearman_ci_high

0.5714 Susceptibility chi predicts best-of-n feature drift on a real bank confirmed registered confidence interval 0.07142857142857151 to 0.5714285714285716 > 0 The frozen study "Susceptibility chi predicts best-of-n feature drift on a real bank", adjudicated on 19 July 2026 at the registered level of the project's evidence ladder. One frozen study, adjudicated once, on skywork-v2-qwen3-8b and skywork-v2-llama31-8b, over ultrafeedback-bank. It does not establish that the same holds on another subject, another dataset or another run. split-bank estimator: chi and drift read disjoint prompt halves, so the null re-centers at zero, but the correlation's exchangeable unit is k = 6 features and the null still passes the frozen 0.3 threshold 0.28 of the time; the registered CI exclusion carries size control and only orderings near the synthetic calibration (0.958) clear the 0.91 MDE. Open the record › Copy technical provenance

Provenance

card

CHI-DRIFT

frozen_at

2026-07-18

adjudicated_at

2026-07-19

spec_hash

spec:1c12042d2eeb93b23ff9cf27b6433125

study_id

study:campaign-chi-drift@v1#1c12042d

Evidence records

c978e3958769ecf93dcf079771a3aedb

power_acceptance

split-bank estimator: chi and drift read disjoint prompt halves, so the null re-centers at zero, but the correlation's exchangeable unit is k = 6 features and the null still passes the frozen 0.3 threshold 0.28 of the time; the registered CI exclusion carries size control and only orderings near the synthetic calibration (0.958) clear the 0.91 MDE

spec_file

campaign-chi-drift.json