Anthropic's model interpretability team's researchers recorded a podcast Introducing the relationship between model interpretability and model safety, and why they are important The research process of interpretability and some explanations of common model concepts are particularly interesting

The core goal of the interpretable toolchain is to create a complete "flowchart" from "input prompt A" to "output text B." The research process consists of five main steps: Data sampling: Feeding the model with diverse prompts (conversations, code, poetry, etc.) and recording activations at each layer. Feature decomposition: Using clustering and sparse coding, compressing hundreds of millions of activations into human-interpretable "concept vectors." Concept labeling: Using statistical "light-up" methods, labeling vectors with labels such as "coffee," "Golden Gate Bridge," and "flattering praise." Causal manipulation: Artificially increasing or decreasing activation strength and observing how the output changes to verify causality rather than mere correlation. Process visualization: Connecting multiple layers of concepts in chronological order to form a human-readable step-by-step diagram, similar to a traceable code call stack. The team likens the system to a "microscope," but admits that it still has limitations: currently, it can only explain approximately 20% of decision paths, and the scale of large models (Claude 4 level) further strains the tool.

The video lists several internal concepts that will make you smile: "Sycophantic praise": Whenever the context contains excessive flattery, a certain cluster of neurons lights up, driving the output of flowery words like "brilliant" and "genius." Golden Gate Bridge representation: This vector fires regardless of whether the input is text describing driving across a bridge, a captioned image of the bridge, or even the mere suggestion of "Golden Gate," demonstrating that the model has formed a cross-modal, abstract, and robust concept of "landmark." "6 + 9" addition circuit: Whenever a number ending in 6 is added to a number ending in 9, whether in an equation, a reference year (1959 + 6), or a house number in a story, the same computational pathway is used, demonstrating that the model uses "universal operators" rather than rote memorization. Bug Tracker: When reading code, specific neural clusters are responsible for flagging potential errors and citing them in subsequent responses, demonstrating its "delayed response" capability. These examples collectively refute the view that models are merely memorizers of training data: relying solely on memory, a model cannot reuse the same logical pathways in unseen, cross-domain scenarios. Researchers further discovered that when processing character relationships in long narratives, the model assigns a "numbered concept" to the first character appearing, binding all subsequent actions and emotions to that number to maintain narrative consistency. This strategy is highly similar to human "variable binding," yet emerges spontaneously. Importantly, the "surprise concept" reveals a "gradient of abstraction": larger models have more concentrated internal semantic layers shared across languages and tasks, ultimately forming a "common semantic space," which explains Claude's consistent performance across multiple languages.