Neural networks are legible without sparse autoencoders

A new perspective on mechanistic interpretability challenges the prevailing view that neural networks inherently obscure their internal logic through superposition. The author argues that with modern architectures and improved tools, most neurons in current language and vision models perform single, understandable tasks.

What happened

For years, the field of mechanistic interpretability operated under the assumption that neural networks were fundamentally tangled. Researchers believed that when models ran out of capacity, they employed a strategy called superposition, folding unrelated concepts into the same neurons to save space. This created polysemantic neurons, which appeared to handle multiple unrelated jobs, such as detecting both cat faces and car fronts. To study these networks, scientists felt compelled to first untangle them using secondary models like sparse autoencoders.

The author now contends that this canonical evidence was misread. The apparent chaos in early models like InceptionV1 was not due to a complex learning principle but rather architectural limitations and weak analysis tools. When modern architectural improvements are applied to similar models, nearly all neurons in the final layers become easily interpretable. The network does not arbitrarily mix concepts; instead, it uses coarser-grained concepts when space is limited and differentiates into specific features when capacity increases.

This shift suggests that the field can return to studying networks directly. Instead of building decomposition models to approximate an unfolded version of the network, researchers can examine neurons and circuits as they exist. The author describes this as a return to rapid normal science, where one observes data flow and tests explanations at scale without the intermediate step of untangling perceived knots.

How it works

The core mechanism identified is recursive generalization rather than superposition. In this view, a neuron does not randomly group unrelated items. Instead, it learns a broad concept that naturally encompasses related sub-concepts. For example, a neuron might activate for various types of cars. If the model has limited capacity, it retains this broad car detector. If capacity increases, that single neuron may split into multiple specialized neurons for specific car models. This is a standard learning progression, not a compression artifact.

Previous interpretations of polysemanticity often failed to account for weak activations and context. Tools that only looked at strong activations or ignored forward-looking context missed the logical connections between seemingly unrelated triggers. A neuron appearing to detect both the Titanic and steam engines is likely recognizing a broader theme of historical voyages or era-specific technology. When these weak signals and contextual relationships are included, the neuron’s function becomes coherent and singular.

Key details

• In updated versions of InceptionV1, 99-100% of neurons in the last layer are easily interpretable, compared to almost none in the original study.
• Preliminary audits of Qwen3-32B estimate that roughly 90% of neuron poles have one main job, with another 7% handling two related tasks.
• Only 2-4% of neurons in modern language models remain unexplained, a sharp contrast to earlier findings of widespread polysemanticity.
• The famous circuit showing a car feature spreading into dog neurons was re-evaluated as ordinary generalization, not a hiding mechanism.
• Architectural improvements designed for deep layer learning inadvertently made models more legible without changing size or training data.
• Weak activations often reveal related offshoots of a main concept, such as MAGA hats activating for a neuron primarily focused on Trump.

Why it matters

This perspective significantly lowers the barrier to entry for mechanistic interpretability. If networks are already clean, engineers do not need to train and maintain complex sparse autoencoders to understand model internals. This reduces computational costs and simplifies the workflow for debugging and auditing AI systems. It allows teams to focus on direct observation and experimentation rather than developing increasingly sophisticated reconstruction objectives.

For product builders, this means greater transparency and trust in model behavior. Understanding that neurons represent coherent concepts rather than arbitrary mixtures enables more reliable intervention and control. It suggests that failures in interpretability were often due to tooling limitations or outdated architectures, not inherent flaws in deep learning. This opens the door for more robust safety checks and feature engineering based on direct neuron manipulation.

What you can do

• Re-evaluate older interpretability findings with modern tools that account for weak activations and bidirectional context.
• Focus on direct neuron analysis in current models before investing in sparse autoencoder training pipelines.
• Examine neuron activations across different percentiles to identify broad conceptual themes rather than just strong triggers.
• Test explanations by observing data flow through circuits at agentic scale to confirm functional roles.
• Consider architectural upgrades as a primary method for improving model legibility alongside training data quality.
• Document neuron functions by tracing their related offshoots to build a comprehensive map of model concepts.