Stats

Lineage-aware effective sample size from seed multiplicities (Kish).

Group the rows by their seed label and let w_s be the number of rows carrying seed s. The effective sample size is the Kish formula

ESS = (sum_s w_s)**2 / sum_s w_s**2

which is the inverse-Simpson count of “how many independent seeds is this dataset worth.” Three anchor cases pin the intuition:

• N rows that are all clones of ONE seed: one weight w = N, so ESS = N2 / N2 = 1.0. Cloning buys nothing.
• N rows from N distinct seeds: ESS = N**2 / N = N. Every row is its own lineage, so the effective n is the nominal n.
• A balanced design, e.g. 30 seeds each expanded to 5 mutations: weights are all 5, so ESS = 150**2 / (30 * 25) = 30. The effective n is the number of seeds, not the number of expanded rows.

Empty input returns 0.0.

Detect exact-duplicate content and collapse it to weighted items.

Exact-duplicate content hashes are byte-identical stimuli that must not count as independent observations. This is the ingest-time check R7 calls for: it reports how many rows are duplicates and the per-hash multiplicity so the data plane can collapse them to weights with a warning.

Bootstrap CI that resamples at the seed (cluster) level by default.

The default draws whole seeds with replacement (a cluster/block bootstrap): on each resample it picks K seeds from the K observed seeds with replacement and takes all rows belonging to each drawn seed, then computes the statistic on the pooled rows. This is the honest interval, because the unit of independence is the seed, not the expanded row. On a dataset of clones of a single seed there is only one cluster to draw, so a CI cannot be formed and the bounds are nan (method "bootstrap-cluster"): the routine refuses to manufacture precision from clones.

Setting allow_clone_resampling=True reverts to resampling individual rows with replacement, which is exactly the v1 behaviour that inflated n. It is kept because there are legitimate uses (a dataset that genuinely has one independent row per seed), but it stamps the result with method="bootstrap-CLONE-INFLATED" so any Evidence built from it is marked as potentially inflated and no downstream reader is misled.

Paired permutation test that flips signs at the seed (cluster) level.

The row-level paired_permutation_test flips the sign of each row’s difference independently, which over-counts evidence when many rows share a seed: fifty clones of one stimulus give fifty independent sign flips that the lineage says should move together. Here the sign is drawn once per seed and applied to every row of that seed, so the permutation null respects the unit of independence. With K seeds the null explores 2**K sign assignments, not 2**n.

Returns nan (not inf) when n < 2 or the relevant std is zero. The v1 hacking detector returned inf in those cases — that’s the bug this function exists to fix.

Bias-corrected and accelerated (BCa) bootstrap CI.

The percentile bootstrap is biased when the sampling distribution of the statistic is skewed or when the statistic is a biased estimator (Cohen’s d and correlations both qualify). BCa corrects for that with two quantities:

• z0 (bias correction): the normal quantile of the fraction of bootstrap replicates falling below the point estimate. If the replicates straddle the point symmetrically z0 is 0 and no bias correction is applied.
• a (acceleration): a jackknife estimate of how fast the standard error changes with the true value, capturing skew. With m_i the leave-one-out estimates and mbar their mean, a = sum((mbar - m_i)**3) / (6 * (sum((mbar - m_i)**2))**1.5).

These map the requested central interval onto adjusted percentiles of the replicate distribution. When the acceleration denominator is zero (a constant jackknife, e.g. degenerate input) a falls back to 0, and when the bias correction is undefined (all replicates on one side of the point) the interval falls back to the plain percentile bounds. Both keep the result finite instead of propagating nan or inf.

Returns NaN bounds for n < 2, mirroring bootstrap_ci.

Spearman rank correlation with bootstrap CI.

Pair-bootstrap (resample (x_i, y_i) jointly, preserving pairing). The point estimate uses scipy if available, else a numpy implementation.

Compute ROC and PR curves from scores and binary labels.

Reliability-diagram data and the expected calibration error.

Bins the predicted probabilities into n_bins equal-width bins over [0, 1] and, per bin, reports the empirical accuracy (mean label) against the mean confidence (mean probability). A perfectly calibrated model has accuracy equal to confidence in every bin.

KSG estimator 1 of the mutual information between continuous x and y (Kraskov 2004).

For each point the distance eps to its k-th nearest neighbour is taken in the joint space under the maximum norm; then n_x and n_y count the points falling strictly within eps in each marginal. The estimate is I = psi(k) + psi(N) - <psi(n_x + 1) + psi(n_y + 1)> in nats, where psi is the digamma function and the marginal counts are read self-inclusive (query_ball_point includes the point itself, so the returned count is already n_x + 1). A tiny tie-breaking jitter is added first. Negative estimates (which the estimator can produce near zero MI) are clamped to zero, and the result is returned in bits by default.