PoC findings: BeFOND sparse autoencoders
What the paper researches. A new way to train sparse autoencoders, the tools used to find human-readable 'features' inside a language model. The method works out which features are active by letting them compete, and claims better feature recovery when many features overlap.
What we found. After correcting a bug of mine (every model had secretly started from the true answer), the new method clearly beat standard TopK/BatchTopK at the moderately hard setting: feature detection F1 0.89 vs 0.58 / 0.44, dictionary recovery 0.94 vs 0.84 / 0.72. It held with different amplitudes, detection rules and longer training. At harder settings every method failed, so no ranking is supported. Synthetic data only.
Why it matters. Interpretability underpins auditing and safety claims; a method that stays reliable when features overlap would give more trustworthy feature lists.
How industry can use it.
- Teams building interpretability pipelines have a reason to benchmark an inference-based autoencoder against TopK on their own activations.
- The benchmark recipe transfers: synthetic data with known features, a random-dictionary baseline, equal tuning for every method, and a check that initialisation cannot see the answer.
- No evidence yet for real language-model activations.
What this does not show. Everything here is on synthetic data from the method's own favourable data model and at toy size; the paper's results on real language-model activations were not tested.
Corrected results (random initialisation, independent of the data)
Same data family as below (256 features, mean 8 active, Zipf firing, noise 0.05), 2,000 steps × 128 samples, 3 seeds (0–2), each model tuned per d on a grid (TopK/BatchTopK lr 3e-4 to 3e-2; BeFOND ηΦ 0.005 to 0.08). Tuning used the same seeds as the evaluation, so numbers are optimistic for every model alike. Best values were interior to the grid except where BeFOND diverged (ηΦ ≥ 0.04 gave NaN at d = 64 in 3 of 3 seeds). Random-dictionary MCC floor: 0.35 (d = 64), 0.49 (d = 32), 0.56 (d = 24).
| d | Model (best hyperparameter) | MCC | F1 | Rare F1 | Dead latents |
|---|---|---|---|---|---|
| 64 | BeFOND (η = 0.02) | 0.940 | 0.891 | 0.795 | 0 |
| 64 | TopK (lr 3e-3) | 0.836 | 0.576 | 0.320 | 17 |
| 64 | BatchTopK (lr 3e-3) | 0.724 | 0.442 | 0.116 | 69 |
| 32 | BeFOND (η = 0.02) | 0.623 | 0.116 | 0.007 | 168 |
| 32 | TopK (lr 1e-2) | 0.611 | 0.153 | 0.056 | 77 |
| 32 | BatchTopK (lr 1e-2) | 0.527 | 0.068 | 0.018 | 179 |
| 24 | all three | indistinguishable from a random dictionary (best MCC: BeFOND 0.605, TopK 0.589, BatchTopK 0.579; best F1 ≤ 0.081; BeFOND ~211 dead at every η) | |||
What now holds. At d = 64 BeFOND detects features much better than TopK/BatchTopK (F1 0.89 vs 0.58 and 0.44) and has the highest MCC (0.94 vs 0.84 and 0.72), so the paper's MCC claim, listed below as "not reproduced", does reproduce at this scale once the leak is removed. A 10,000-step check (2 seeds) keeps the d = 64 picture: BeFOND F1 0.89/0.92 and MCC 0.94/0.95, TopK F1 0.64/0.65 and MCC 0.885. What is not supported. At d = 32 and d = 24 every model fails to learn the dictionary within the budget (more than 150 dead latents for BeFOND at d = 32), longer training (10,000 steps) does not help any of them, and the ranking there is within noise: no feature-detection advantage is established at high superposition. The d = 24 collapse is therefore not specific to my BeFOND, and the earlier "may be a tuning issue" reading was wrong in the other direction: it is non-learning of all models at this scale. At d = 64, where the models learn, the hyperparameter chosen by MCC and by F1 coincide for all three models; at d ≤ 32, where nothing learns, they differ and the choice is noise.
Robustness checks on the corrected code (d = 64, 3 seeds)
- Detection rule. Giving TopK per-latent oracle thresholds (chosen in-sample, so an upper bound) lifts its F1 from 0.576 to 0.623 and BatchTopK's from 0.442 to 0.470; BeFOND stays at 0.891.
- Amplitudes. With feature amplitudes uniform in [0.5, 1.5], BeFOND F1 is 0.845 and the best TopK 0.560 (BatchTopK 0.437).
- Dead latents. The baselines are plain implementations (no auxiliary loss); TopK has 17 and BatchTopK 69 dead latents of 256 at d = 64, so the original page's remark that they had none no longer applies. A simple resampling of dead latents (re-initialised to residual directions, last allowed at step 1,750) left TopK F1 unchanged (0.575) with 18 dead; I did not try an auxiliary dead-latent loss or earlier resampling.
Limits of the d = 64 comparison
- The data come from BeFOND's own generative family (Bernoulli presence, linear, Gaussian noise); BeFOND runs 15 inference iterations per sample while TopK uses one linear pass, and both were given the true mean L0.
- BeFOND's inner hyperparameters (noise, prior and inference step sizes) were set during the original development and not re-tuned after the fix; TopK/BatchTopK got only a learning-rate grid. Hyperparameters were chosen on the same seeds that are reported.
- The d = 64, 32 and 24 datasets for one seed share their first rows of random numbers, so the three dimensions are correlated, not independent replications. The random-dictionary MCC "floor" is a single draw, and models can score below it (BeFOND at η = 0.04: 0.29 vs 0.56).
Not tested: the paper's language-model results; other seeds beyond 0–2; training length beyond 10,000 steps for BeFOND at d ≤ 24; the paper's rolling-reference variant. Code for the corrected runs: pocs/2610.07389/fixed_init/.
Superseded: original results (leaky initialisation)
Paper: arXiv 2610.07389, "Inference and learning in sparse autoencoders as natural gradient flow". Experiment run 2026-10-07. Pure numpy, CPU only (2 cores), about 2 hours total including tuning and diagnostics.
- Held: a simplified BeFOND detects which features are active far better than TopK and BatchTopK SAEs as superposition rises (macro F1 0.955 vs 0.668 at d=64; 0.685 vs 0.425 at d=32). The edge survives random feature amplitudes and survives giving the baselines generous, in-sample-tuned detection thresholds.
- Did not hold: "BeFOND has the highest dictionary recovery (MCC)". Here TopK/BatchTopK were slightly higher at every d ≤ 64 (about 0.998 vs 0.975–0.994), though all are near the ceiling, so MCC barely discriminates at this scale.
- Did not hold: "no dead latents". At the highest superposition tested (d=24) my BeFOND collapsed: 174 of 256 latents dead and about 2.5 active per sample. This may be a tuning or stability limit of my simplified version rather than of the paper's method. I did not investigate it.
What was run
Data (SynthSAEBench-style): K*=256 ground-truth features, random unit-norm atoms in d dimensions, Zipf-like firing probabilities (pi ∝ i-0.7, capped at 0.5, mean 8 active features), Gaussian noise 0.05. Superposition is raised by shrinking d at fixed K* and measured as ρmm (mean max absolute cosine between atoms; paper eq. 18): d = 96, 64, 48, 32, 24 gives ρmm = 0.30, 0.37, 0.43, 0.52, 0.58. The main sweep uses binary amplitudes (which matches BeFOND's Bernoulli prior). A fairness variant draws amplitudes uniformly from [0.5, 1.5] (d = 64, 32).
Models (all given the true mean L0 = 8, trained for the same 2,000 steps × 128 samples): a toy BeFOND (paper eqs. 13–17: fixed-prior mean-field inference with a feedforward drive, recurrent explaining-away term, variance penalty and homeostatic leak; decoder, noise and prior learned with the Fisher-preconditioned rules), a TopK SAE and a BatchTopK SAE (both Adam, unit-norm decoder columns).
Metrics: MCC (Hungarian-matched |cosine| between learned and true atoms), macro F1 of detecting each matched feature on 10,000 held-out samples (BeFOND: posterior > 0.5; SAEs: activation > 0), Rare F1 (rarest 25% of features), dead latents. 3 seeds per cell (seeds change atoms and initialization). Code: pocs/2610.07389/.
Results vs the paper
| Claim | Paper | Here (mean over 3 seeds; BeFOND vs best baseline) | |
|---|---|---|---|
| Highest dictionary recovery (MCC) | yes, across the superposition sweep | d=96: 1.000 vs 0.999. d=64: 0.994 vs 0.998. d=48: 0.991 vs 0.998. d=32: 0.987 vs 0.997. d=24: 0.975 vs 0.998. All near the ceiling (random dictionary: 0.29–0.56) | not reproduced |
| Better feature detection (macro F1) under superposition | yes | d=96: 0.995 vs 0.819. d=64: 0.955 vs 0.668. d=48: 0.880 vs 0.559. d=32: 0.685 vs 0.425 | reproduced (d ≥ 32) |
| Better rare-feature detection | yes | Rare F1, d=96/64/48/32: 0.992/0.937/0.846/0.632 vs 0.769/0.596/0.483/0.347 | reproduced (d ≥ 32) |
| Edge is not an artifact of binary amplitudes | not claimed | uniform amplitudes: F1 0.918 vs 0.602 (d=64), 0.632 vs 0.409 (d=32) | holds |
| Edge is not an artifact of the baselines' "activation > 0" detection rule | not claimed | Baselines with in-sample per-latent oracle thresholds: TopK F1 0.715 (d=64), 0.454 (d=32), still well below BeFOND's 0.955 and 0.685 | holds |
| Zero dead latents | yes (K=4,096) | 0 dead for d ≥ 32. At d=24: 174 ± 3 dead, F1 0.071, Rare F1 0.002 | not reproduced at d=24 |
| Natural-gradient learning beats Euclidean for rare features (theory); lateral interactions drive the gain; results on Gemma activations | yes | not tested | untested |
Sweep figures
Main sweep. Left to right: MCC, macro F1, rare-feature F1, dead latents. Error bars are sd over 3 seeds and are tiny because seeds mostly change atoms and initialization.
Fairness variant: amplitudes uniform in [0.5, 1.5].
Caveats
- Simplified implementation. No "rolling-reference" prior variant, no matrix-free decoder update, plain Euler for learning with a 1e-3 ridge on Ep, 15 inference iterations of exponential-Euler on the leak term. The paper's results may depend on pieces I left out.
- Toy scale. 256 features in d ≤ 96, against the paper's up to 16,384 features at d=768. The paper's "highest MCC" claim may hold at scale, where MCC is not saturated.
- Data favors BeFOND. The data come from a Bernoulli-presence, linear, Gaussian-noise model, which is BeFOND's own generative model. The uniform-amplitude variant weakens this but does not remove it.
- Tuning. Three rounds on a separate seed (99) at d=32, a few learning-rate settings per model per round, selected by MCC. The best setting sat at the lower grid edge for all three models (BeFOND ηΦ = 0.005, both SAEs lr = 3e-5), so "tuned" is approximate. My first BeFOND grid (0.1–0.4) was too high and made it look broken (about 90% dead latents at d=32); that was my tuning mistake, not the method. One learning rate, chosen at d=32, was reused for every d, so the d=24 collapse could be a tuning issue.
- Compute is not matched. Same steps and samples, but BeFOND ran 15 inference iterations per step and took roughly 135 s per run vs about 15 s for the SAEs.
- Detection rule. F1 uses BeFOND's posterior > 0.5 and the SAEs' activation > 0 (checked with tuned thresholds, above). The baselines are plain implementations without auxiliary losses or resampling (they had no dead latents anyway).
- 3 seeds per cell, so small differences (for example MCC differences of 0.003–0.02) should be read cautiously, even though the sd is tiny.
About this analysis
Done by Claude (an AI model by Anthropic), directed by vishwasaidev-dev. Claude can make mistakes, including implementation errors, misreading the paper, or over-reading results from 3 seeds on synthetic data. If you think something here is wrong, please open an issue. Issues are picked up in the next working session (nothing watches GitHub automatically), and corrections are made in the open with the change noted below.
Corrections log
- 2026-10-09. Added robustness checks and limits for the corrected d = 64 comparison after an independent review (baseline dead latents, BeFOND-family data, un-retuned BeFOND inner hyperparameters); reworded 'at the floor' to 'indistinguishable from a random dictionary'.
- 2026-10-08. Ground-truth leak: data generator and model initialisation shared one random seed, so all models started at the true dictionary. All MCC, tuning and d = 24 statements below the banner are superseded; corrected results added at the top. The detection-F1 advantage survives at d = 64 and is not established at d ≤ 32.