← all findings

PoC findings: BeFOND sparse autoencoders

Correction (2026-10-08): the original results on this page used a flawed setup and are superseded. In the original code the data generator and each model's initial dictionary were drawn from the same random seed, so every model started with the true dictionary (mean |cosine| with the true atoms = 1.0, checked at d = 64, 32 and 24). The MCC values (0.975–1.000), the "highest MCC not reproduced" verdict, the tuned learning rates (which sat on the grid edge because they moved the dictionary least), and the d = 24 collapse all came from that. The F1 comparison mixed inference quality with a perfect starting dictionary. Everything was re-run with an independent initialisation stream and fresh hyperparameter grids (below). The old table is kept further down, marked superseded, so the change is visible. Found while setting up a follow-up: a TopK run at learning rate 1e-5 returned MCC 1.000 after 2,000 steps, which Adam cannot produce by learning from scratch.
Why this matters (plain-language summary; technical detail is below)

What the paper researches. A new way to train sparse autoencoders, the tools used to find human-readable 'features' inside a language model. The method works out which features are active by letting them compete, and claims better feature recovery when many features overlap.

What we found. After correcting a bug of mine (every model had secretly started from the true answer), the new method clearly beat standard TopK/BatchTopK at the moderately hard setting: feature detection F1 0.89 vs 0.58 / 0.44, dictionary recovery 0.94 vs 0.84 / 0.72. It held with different amplitudes, detection rules and longer training. At harder settings every method failed, so no ranking is supported. Synthetic data only.

Why it matters. Interpretability underpins auditing and safety claims; a method that stays reliable when features overlap would give more trustworthy feature lists.

How industry can use it.

What this does not show. Everything here is on synthetic data from the method's own favourable data model and at toy size; the paper's results on real language-model activations were not tested.

Corrected results (random initialisation, independent of the data)

Same data family as below (256 features, mean 8 active, Zipf firing, noise 0.05), 2,000 steps × 128 samples, 3 seeds (0–2), each model tuned per d on a grid (TopK/BatchTopK lr 3e-4 to 3e-2; BeFOND ηΦ 0.005 to 0.08). Tuning used the same seeds as the evaluation, so numbers are optimistic for every model alike. Best values were interior to the grid except where BeFOND diverged (ηΦ ≥ 0.04 gave NaN at d = 64 in 3 of 3 seeds). Random-dictionary MCC floor: 0.35 (d = 64), 0.49 (d = 32), 0.56 (d = 24).

dModel (best hyperparameter)MCCF1Rare F1Dead latents
64BeFOND (η = 0.02)0.9400.8910.7950
64TopK (lr 3e-3)0.8360.5760.32017
64BatchTopK (lr 3e-3)0.7240.4420.11669
32BeFOND (η = 0.02)0.6230.1160.007168
32TopK (lr 1e-2)0.6110.1530.05677
32BatchTopK (lr 1e-2)0.5270.0680.018179
24all threeindistinguishable from a random dictionary (best MCC: BeFOND 0.605, TopK 0.589, BatchTopK 0.579; best F1 ≤ 0.081; BeFOND ~211 dead at every η)

What now holds. At d = 64 BeFOND detects features much better than TopK/BatchTopK (F1 0.89 vs 0.58 and 0.44) and has the highest MCC (0.94 vs 0.84 and 0.72), so the paper's MCC claim, listed below as "not reproduced", does reproduce at this scale once the leak is removed. A 10,000-step check (2 seeds) keeps the d = 64 picture: BeFOND F1 0.89/0.92 and MCC 0.94/0.95, TopK F1 0.64/0.65 and MCC 0.885. What is not supported. At d = 32 and d = 24 every model fails to learn the dictionary within the budget (more than 150 dead latents for BeFOND at d = 32), longer training (10,000 steps) does not help any of them, and the ranking there is within noise: no feature-detection advantage is established at high superposition. The d = 24 collapse is therefore not specific to my BeFOND, and the earlier "may be a tuning issue" reading was wrong in the other direction: it is non-learning of all models at this scale. At d = 64, where the models learn, the hyperparameter chosen by MCC and by F1 coincide for all three models; at d ≤ 32, where nothing learns, they differ and the choice is noise.

Robustness checks on the corrected code (d = 64, 3 seeds)

Limits of the d = 64 comparison

Not tested: the paper's language-model results; other seeds beyond 0–2; training length beyond 10,000 steps for BeFOND at d ≤ 24; the paper's rolling-reference variant. Code for the corrected runs: pocs/2610.07389/fixed_init/.

Superseded: original results (leaky initialisation)

Paper: arXiv 2610.07389, "Inference and learning in sparse autoencoders as natural gradient flow". Experiment run 2026-10-07. Pure numpy, CPU only (2 cores), about 2 hours total including tuning and diagnostics.

Verdict: partly reproduced, with two paper claims that did not hold in this toy setup.
  • Held: a simplified BeFOND detects which features are active far better than TopK and BatchTopK SAEs as superposition rises (macro F1 0.955 vs 0.668 at d=64; 0.685 vs 0.425 at d=32). The edge survives random feature amplitudes and survives giving the baselines generous, in-sample-tuned detection thresholds.
  • Did not hold: "BeFOND has the highest dictionary recovery (MCC)". Here TopK/BatchTopK were slightly higher at every d ≤ 64 (about 0.998 vs 0.975–0.994), though all are near the ceiling, so MCC barely discriminates at this scale.
  • Did not hold: "no dead latents". At the highest superposition tested (d=24) my BeFOND collapsed: 174 of 256 latents dead and about 2.5 active per sample. This may be a tuning or stability limit of my simplified version rather than of the paper's method. I did not investigate it.
Everything here is a toy-scale test of a simplified re-implementation on data from BeFOND's own generative family. It says little about the paper's language-model results, which were not tested.

What was run

Data (SynthSAEBench-style): K*=256 ground-truth features, random unit-norm atoms in d dimensions, Zipf-like firing probabilities (pi ∝ i-0.7, capped at 0.5, mean 8 active features), Gaussian noise 0.05. Superposition is raised by shrinking d at fixed K* and measured as ρmm (mean max absolute cosine between atoms; paper eq. 18): d = 96, 64, 48, 32, 24 gives ρmm = 0.30, 0.37, 0.43, 0.52, 0.58. The main sweep uses binary amplitudes (which matches BeFOND's Bernoulli prior). A fairness variant draws amplitudes uniformly from [0.5, 1.5] (d = 64, 32).

Models (all given the true mean L0 = 8, trained for the same 2,000 steps × 128 samples): a toy BeFOND (paper eqs. 13–17: fixed-prior mean-field inference with a feedforward drive, recurrent explaining-away term, variance penalty and homeostatic leak; decoder, noise and prior learned with the Fisher-preconditioned rules), a TopK SAE and a BatchTopK SAE (both Adam, unit-norm decoder columns).

Metrics: MCC (Hungarian-matched |cosine| between learned and true atoms), macro F1 of detecting each matched feature on 10,000 held-out samples (BeFOND: posterior > 0.5; SAEs: activation > 0), Rare F1 (rarest 25% of features), dead latents. 3 seeds per cell (seeds change atoms and initialization). Code: pocs/2610.07389/.

Results vs the paper

ClaimPaperHere (mean over 3 seeds; BeFOND vs best baseline)
Highest dictionary recovery (MCC)yes, across the superposition sweepd=96: 1.000 vs 0.999. d=64: 0.994 vs 0.998. d=48: 0.991 vs 0.998. d=32: 0.987 vs 0.997. d=24: 0.975 vs 0.998. All near the ceiling (random dictionary: 0.29–0.56)not reproduced
Better feature detection (macro F1) under superpositionyesd=96: 0.995 vs 0.819. d=64: 0.955 vs 0.668. d=48: 0.880 vs 0.559. d=32: 0.685 vs 0.425reproduced (d ≥ 32)
Better rare-feature detectionyesRare F1, d=96/64/48/32: 0.992/0.937/0.846/0.632 vs 0.769/0.596/0.483/0.347reproduced (d ≥ 32)
Edge is not an artifact of binary amplitudesnot claimeduniform amplitudes: F1 0.918 vs 0.602 (d=64), 0.632 vs 0.409 (d=32)holds
Edge is not an artifact of the baselines' "activation > 0" detection rulenot claimedBaselines with in-sample per-latent oracle thresholds: TopK F1 0.715 (d=64), 0.454 (d=32), still well below BeFOND's 0.955 and 0.685holds
Zero dead latentsyes (K=4,096)0 dead for d ≥ 32. At d=24: 174 ± 3 dead, F1 0.071, Rare F1 0.002not reproduced at d=24
Natural-gradient learning beats Euclidean for rare features (theory); lateral interactions drive the gain; results on Gemma activationsyesnot testeduntested

Sweep figures

Main sweep, binary amplitudes

Main sweep. Left to right: MCC, macro F1, rare-feature F1, dead latents. Error bars are sd over 3 seeds and are tiny because seeds mostly change atoms and initialization.

Fairness variant, uniform amplitudes

Fairness variant: amplitudes uniform in [0.5, 1.5].

Caveats

About this analysis

Done by Claude (an AI model by Anthropic), directed by vishwasaidev-dev. Claude can make mistakes, including implementation errors, misreading the paper, or over-reading results from 3 seeds on synthetic data. If you think something here is wrong, please open an issue. Issues are picked up in the next working session (nothing watches GitHub automatically), and corrections are made in the open with the change noted below.

Corrections log