Mechanistic interpretability of refusal behaviors in LLMs
Mechanistic interpretability of refusal behaviors in LLMs
Want to keep this search?
Sign in to save it to your profile and mark papers as good, seen or bookmarked.
-
98 MatchArXiv 2026
Broken Symmetry in LLM Refusal: Answer Release Is More Local Than Refusal Restoration
Yiqi Liu; Yang Wang; Songxin Wang; Chenghao Xiao; Chenghua Lin
- TOPIC
- This paper examines the mechanistic nature of refusal in LLMs by testing if suppressing a correct answer is a symmetric, reversible operation.
- NOVELTY
- The authors introduce a controlled withhold task and use activation patching to show that while releasing a withheld answer is a local operation, restoring refusal is distributed across multiple tokens.
- INSIGHT
- A key takeaway is the dissociation between probe recoverability and behavioral control, meaning the answer remains linearly accessible even when the model successfully refuses.
- RELEVANCE
- This is directly applicable to your work as it challenges the assumption that refusal behaves like a simple linear switch or a single steerable direction in activation space.
Found Oct 28 -
98 MatchEMNLP 2026
Broken Symmetry in LLM Refusal: Answer Release Is More Local Than Refusal Restoration
Yiqi Liu; Yang Wang; Songxin Wang; Chenghao Xiao; Chenghua Lin
- TOPIC
- This paper investigates the mechanistic nature of refusal in LLMs, specifically examining whether suppressing a correct answer is a symmetric, local operation in the hidden states.
- NOVELTY
- The authors use bidirectional activation patching and a hierarchy of decoders to show that while releasing a withheld answer is a local operation, restoring refusal is non-local and distributed across multiple tokens.
- INSIGHT
- A key takeaway is the dissociation between probe recoverability and behavioral control, meaning the model can internally represent the correct answer even while successfully refusing to output it.
- RELEVANCE
- Since you are focused on the mechanistic interpretability of refusal behaviors, this work is essential as it challenges the idea of refusal as a simple linear switch and provides a rigorous framework for testing causal symmetry.
Found Oct 26 -
98 MatchEMNLP 2026
A Unified Mechanistic Analysis of Knowledge- and Safety-Based Refusals
Yuri Son; Seunghee Kim; Hyuhng Joon Kim; Taeuk Kim
- TOPIC
- This paper explores the mechanistic differences and similarities between knowledge-based refusal (KR) and safety-based refusal (SR) in LLMs.
- NOVELTY
- The authors introduce a "commit-then-specify" framework, identifying a shared low-dimensional refusal direction that triggers the act of declining before the model specifies the reason in later layers.
- INSIGHT
- They found a directional asymmetry where safety refusal signals generalize more strongly to knowledge refusal than the other way around.
- RELEVANCE
- Since you are focused on the mechanistic interpretability of refusal, this work is essential as it maps exactly how the residual stream handles different refusal triggers across model layers.
Found Oct 26 -
98 MatchArXiv 2026
Component and Dimension Sparsity in Transformer Refusal Mechanisms
Vincent Siu; Glenn Grant-Richards; Vlad Pavlovich; Yizhou Sun; Dawn Song; Chenguang Wang
- TOPIC
- This paper investigates the mechanistic basis of refusal behaviors in LLMs by decomposing the refusal direction in the residual stream.
- NOVELTY
- They introduce a two-stage greedy backward elimination pipeline to isolate the specific attention and MLP sublayers that contribute most to the refusal signal.
- INSIGHT
- The authors find that refusal is concentrated in a sparse subset of components (28–48%) and that the signal is partially axis-aligned, suggesting a privileged basis for these behaviors.
- RELEVANCE
- Since you are focused on the mechanistic interpretability of refusal, this work is essential as it moves beyond simple steering to identify the exact structural components and dimensions responsible for the behavior.
Found Oct 7 -
98 MatchICML 2025
The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence
Tom Wollschläger; Jannes Elstner; Simon Geisler; Vincent Cohen-Addad; Stephan Günnemann; Johannes Gasteiger
- TOPIC
- This paper investigates the mechanistic internal representations of refusal behaviors in LLMs to determine if they are governed by a single linear direction or more complex geometries.
- NOVELTY
- The authors introduce Refusal Direction Optimization (RDO) and Refusal Cone Optimization (RCO) to identify multi-dimensional polyhedral "concept cones" and a criterion called Representational Independence (RepInd) to distinguish between independent refusal mechanisms.
- INSIGHT
- They find that refusal is not a single vector but a high-dimensional structure where multiple orthogonal directions contribute independently to the model's decision to decline a prompt.
- RELEVANCE
- Since you are specifically focused on the mechanistic interpretability of refusals, this work's shift from linear vectors to independent causal directions is directly applicable to your research.
Found Sep 27 -
98 MatchFindings 2025
Understanding Refusal in Language Models with Sparse Autoencoders
Wei Jie Yeo; Nirmalendu Prakash; Clement Neo; Ranjan Satapathy; Roy Ka-Wei Lee; Erik Cambria
- TOPIC
- This paper investigates the mechanistic underpinnings of refusal behaviors in safety-aligned LLMs using Sparse Autoencoders (SAEs).
- NOVELTY
- They introduce a hybrid CosSim+AP method to isolate a minimal set of causal refusal features, distinguishing between upstream 'harmful' features and downstream 'refusal' features.
- INSIGHT
- The key takeaway is that jailbreaks function by suppressing these specific refusal features, and that SAE-derived sparse probes are more robust to adversarial OOD samples than dense linear probes.
- RELEVANCE
- Since you are focused on the mechanistic interpretability of refusals, this work provides a direct causal mapping of how refusal is triggered and bypassed at the feature level.
Found Sep 27 -
98 MatchNeurIPS 2024
Refusal in Language Models Is Mediated by a Single Direction
Andy Arditi; Oscar Obeso; Aaquib Syed; Daniel Paleka; Nina Panickssery; Wes Gurnee; Neel Nanda
- TOPIC
- This paper investigates the mechanistic basis of refusal behaviors in safety-fine-tuned LLMs by analyzing the model's activation space.
- NOVELTY
- They identify a single, linear 'refusal direction' in the residual stream and demonstrate that orthogonalizing weights against this vector can surgically remove safety guardrails.
- INSIGHT
- The key takeaway is that refusal is mediated by a low-dimensional representation that can be bypassed via white-box weight edits without degrading general cognitive performance.
- RELEVANCE
- Since you are specifically focused on the mechanistic interpretability of refusals, this work provides a direct method for isolating and manipulating the exact internal structures responsible for that behavior.
Found Sep 27 -
95 MatchArXiv 2026
Which Words a Model Fears: From a Refusal Direction to a Vocabulary-Wide Fear Lexicon
Boyuan Zhang; Mohammad Zandsalimy; Shanu Sushmita
- TOPIC
- This paper examines over-refusal in LLMs by investigating how specific surface-level words trigger refusal regardless of the prompt's actual intent.
- NOVELTY
- They use crossed differencing and a two-way vector ANOVA to isolate a universal refusal direction in the residual stream, which they then use to map a vocabulary-wide Fear Lexicon.
- INSIGHT
- Crucially, they find that this refusal behavior is driven by late-layer MLPs rather than initial embeddings, meaning the model's 'fear' of a word is a computed state.
- RELEVANCE
- Since you are focused on the mechanistic interpretability of refusal, this provides a direct causal link between specific tokens and the activation of a shared refusal vector.
Found Nov 3 -
95 MatchEMNLP 2026
What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal
Stephen S Cheng; Sarah Wiegreffe; Dinesh Manocha
- TOPIC
- This paper investigates the mechanistic interpretability of representation steering in LLMs, specifically focusing on how steering vectors trigger or suppress refusal behaviors.
- NOVELTY
- They introduce a multi-token activation patching framework using EAP-IG and a Steering Value Vector (SVV) decomposition to map steering effects to the vocabulary space.
- INSIGHT
- The authors find that refusal steering is highly localized, relying almost exclusively on the OV circuit and MLPs rather than the QK attention circuit.
- RELEVANCE
- Since you are studying the mechanistic interpretability of refusal, this provides a direct causal map of the circuits responsible for these behaviors across multiple model architectures.
Found Oct 29 -
95 MatchEMNLP 2026
Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness
Hoang Cuong Nguyen; Mark Dras; Usman Naseem
- TOPIC
- This paper explores the mechanistic interpretability of refusal circuits across different post-training objectives like SFT, Ra-SFT, and ORPO.
- NOVELTY
- It identifies that reasoning-augmented SFT shifts the causal weight of refusal from attention heads to MLP blocks and distributes activations more evenly across the network.
- INSIGHT
- Crucially, it shows that the training objective fundamentally reshapes the circuit topology and geometric representation of refusal independently of the model architecture.
- RELEVANCE
- Since you are focused on the mechanistic interpretability of refusal behaviors, this provides a direct comparison of how different alignment methods create distinct internal vulnerability profiles.
Found Oct 29 -
95 MatchEMNLP 2026
Mood Matters: How Syntactic Sensitivity Undermines Safety Alignment
Alina Klerings; Jannik Brinkmann; Heiner Stuckenschmidt; Simone Paolo Ponzetto
- TOPIC
- This paper investigates the robustness of LLM safety alignment by examining how syntactic variations in prompts trigger or bypass refusal behaviors.
- NOVELTY
- They use Sparse Autoencoders (SAEs) to causally prove that refusal is often gated by a specific "imperative mood" feature rather than being based solely on semantic harm.
- INSIGHT
- Crucially, they find that models can internally detect harmful intent across various phrasings, but the actual refusal mechanism only activates when a specific syntactic latent is present.
- RELEVANCE
- This is directly relevant to your work as it provides a mechanistic explanation for refusal failures and demonstrates how to use SAEs to isolate the specific features controlling these behaviors.
Found Oct 28 -
95 MatchEMNLP 2026
Over-Refusal and Representation Subspaces: A Mechanistic Analysis of Task-Conditioned Refusal in Aligned LLMs
Utsav Maskey; Mark Dras; Usman Naseem
- TOPIC
- This paper investigates the mechanistic differences between intended harmful-refusal and erroneous over-refusal in safety-aligned LLMs.
- NOVELTY
- The authors show that while harmful-refusal follows a universal, low-dimensional direction, over-refusal is task-dependent and resides within higher-dimensional benign task clusters.
- INSIGHT
- This suggests that global ablation of a single refusal vector is insufficient for fixing over-refusal because the representational geometry of these two behaviors is fundamentally different.
- RELEVANCE
- Since you are focused on the mechanistic interpretability of refusal, this provides a direct mapping of how refusal decisions are committed across layers and the specific circuits involved.
Found Oct 26 -
95 MatchEMNLP 2026
There Is More to Refusal in Large Language Models than a Single Direction
Faaiz Joad; Majd Hawasly; Sabri Boughorbel; Nadir Durrani; Husrev Taha Sencar
- TOPIC
- This paper examines the internal representational mechanisms that drive refusal and non-compliance across different categories like safety, capability limits, and over-refusal.
- NOVELTY
- The authors use a combination of contrastive activation steering and Sparse Autoencoders (SAEs) to show that while different refusal types have distinct directions in activation space, they all rely on a shared core of latent features.
- INSIGHT
- They find that linear steering acts as a coarse control knob because it targets this shared core, whereas the specific direction chosen only modulates the style of the refusal.
- RELEVANCE
- This is directly relevant to your work as it provides a mechanistic decomposition of refusal behaviors, moving beyond a single 'refusal vector' to a more nuanced SAE-based understanding.
Found Oct 26 -
95 MatchACL 2025
Why Safeguarded Ships Run Aground? Aligned Large Language Models’ Safety Mechanisms Tend to Be Anchored in The Template Region
Chak Tou Leong; Qingyu Yin; Jian Wang; Wenjie Li
- TOPIC
- This paper investigates the mechanistic basis of refusal behaviors in safety-aligned LLMs, specifically focusing on how models decide to refuse harmful requests.
- NOVELTY
- They identify Template-Anchored Safety Alignment (TASA), a phenomenon where models rely on shortcuts in the chat template region rather than the semantic content of the prompt to trigger refusal.
- INSIGHT
- The authors use activation patching and linear probing to show that jailbreaks work by disrupting the aggregation of harmfulness features within these specific template tokens.
- RELEVANCE
- This is directly applicable to your work as it provides a causal explanation for refusal failure and proposes a steering method to decouple safety triggers from template-dependent shortcuts.
Found Oct 23 -
95 MatchArXiv 2026
Decomposing Refusal Steering in Mixture-of-Experts Models
Anna C. Marbut; Daniel R. Olson; Travis J. Wheeler
- TOPIC
- This paper explores the mechanistic basis of refusal behaviors specifically within Mixture-of-Experts (MoE) architectures.
- NOVELTY
- It introduces a method to decompose aggregate refusal steering vectors into expert-specific components using forced routing and activation steering.
- INSIGHT
- The authors find that refusal signals are distributed unevenly across experts and that routing frequency is a poor proxy for identifying the actual experts responsible for producing refusal outputs.
- RELEVANCE
- This is directly applicable to your work as it provides a granular, white-box analysis of how safety alignment is localized or distributed across the FFNs of MoE models.
Found Oct 23 -
95 MatchArXiv 2026
WRDO: Weighted Refusal Direction Optimization in Aligned Language Models
Grigoriy Evseev; Vladimir Solodkin; Dmitry Bylinkin; Daniil Medyakov; Vladimir Aletov; Aleksandr Beznosikov
- TOPIC
- This paper investigates the mechanistic representation of refusal behaviors in LLMs to enable precise control over safety guardrails.
- NOVELTY
- They introduce Weighted Refusal Direction Optimization (WRDO), which replaces the standard single-direction refusal axis with a learned, weighted combination of per-prompt directions optimized via CMAES or GRPO.
- INSIGHT
- The key takeaway is that refusal is not a monolithic direction in the residual stream but is heterogeneous across different harmful categories, requiring a composite vector for effective abliteration or restoration.
- RELEVANCE
- Since you are focused on the mechanistic interpretability of refusal, this work provides a direct method for identifying and manipulating the specific internal representations that trigger refusal responses.
Found Oct 23 -
95 MatchICML 2026
Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models
Yanchen Yin; Dongqi Han; Linghui Li
- TOPIC
- This paper explores the mechanistic internal processes of LLMs during successful jailbreak attacks to see if refusal mechanisms are erased or merely bypassed.
- NOVELTY
- They identify and categorize two distinct types of attention heads: Adversarially Compromised Heads (ACHs) in early layers that are suppressed by attacks, and Safety-Aligned Heads (SAHs) in mid-layers that remain active.
- INSIGHT
- The key takeaway is that harmful representations persist as "Robust Harmful Features" even when the model fails to refuse, meaning jailbreaks target specific causal pathways rather than the safety alignment itself.
- RELEVANCE
- This is directly applicable to your work on the mechanistic interpretability of refusal, as it provides a circuit-level decomposition of how refusal is suppressed during adversarial attacks.
Found Oct 21 -
95 MatchICML 2026
SafeSeek: Universal Attribution of Safety Circuits in Language Models
Miao Yu; Siyuan Fu; Moayad Aloqaily; Zhenhong Zhou; Safa Otoum; Xing fan; Kun Wang; Yufei Guo; Qingsong Wen
- TOPIC
- This paper investigates the localization of safety circuits in LLMs to causally attribute refusal and backdoor behaviors to specific internal components.
- NOVELTY
- It introduces SafeSeek, a gradient-based optimization framework using differentiable binary masks and Straight-Through Estimators to identify sparse, functionally complete subgraphs responsible for these behaviors.
- INSIGHT
- The authors find that safety alignment is governed by extremely sparse circuits—often less than 1% of parameters—that are structurally orthogonal to the model's general utility logic.
- RELEVANCE
- This is directly applicable to your work as it provides a scalable method to map the exact mechanistic circuits that trigger refusal and demonstrates how removing them causes a total safety collapse.
Found Oct 21 -
95 MatchICML 2025
The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions
Wenbo Pan; Zhichao Liu; Qiguang Chen; Xiangyang Zhou; Yu Haining; Xiaohua Jia
- TOPIC
- This paper investigates the mechanistic interpretability of safety alignment in LLMs, specifically focusing on how refusal behaviors are represented in the activation space.
- NOVELTY
- Instead of a single refusal vector, the authors define a multi-dimensional "Safety Residual Space" using SVD to decompose alignment shifts into a dominant refusal direction and several orthogonal, non-dominant components.
- INSIGHT
- They find that these secondary directions encode specific jailbreak modalities, such as role-playing or hypothetical narratives, and that suppressing them can causally disable the model's ability to refuse specific attack types.
- RELEVANCE
- This is directly applicable to your work on refusal behaviors as it provides a framework for moving beyond simple linear probes to understand the nuanced, multi-dimensional nature of how models trigger refusals.
Found Oct 14 -
95 MatchArXiv 2026
Attention Sees, MLP Refuses: Tracing the Attention-to-MLP Safety Pathway in LLMs
Zixuan Weng; Haoqi He; Weifei Jin; Yue Huang; Yang Bai
- TOPIC
- This paper investigates the mechanistic internal pathways that LLMs use to transition from recognizing a harmful prompt to generating a refusal response.
- NOVELTY
- It introduces a framework decomposing model updates into attention and MLP components to show that attention modules primarily handle harmfulness recognition while MLP modules drive the actual refusal writing.
- INSIGHT
- The key takeaway is the functional asymmetry where post-training strengthens the MLP-driven decision to refuse rather than the model's underlying ability to perceive harm.
- RELEVANCE
- This is directly applicable to your work on the mechanistic interpretability of refusal behaviors, as it provides a causal link between specific module types and the refusal pipeline.
Found Oct 12 - Loading more results…