PaperManta Sign in
Active · from 2025 onwards (ongoing) · Shared by Marek

Mechanistic interpretability of refusal behaviors in LLMs

Mechanistic interpretability of refusal behaviors in LLMs

Want to keep this search?

Sign in to save it to your profile and mark papers as good, seen or bookmarked.

Sign in to save
149 results
  • ArXiv 2026

    Broken Symmetry in LLM Refusal: Answer Release Is More Local Than Refusal Restoration

    Yiqi Liu; Yang Wang; Songxin Wang; Chenghao Xiao; Chenghua Lin

    98 Match
    TOPIC
    This paper examines the mechanistic nature of refusal in LLMs by testing if suppressing a correct answer is a symmetric, reversible operation.
    NOVELTY
    The authors introduce a controlled withhold task and use activation patching to show that while releasing a withheld answer is a local operation, restoring refusal is distributed across multiple tokens.
    INSIGHT
    A key takeaway is the dissociation between probe recoverability and behavioral control, meaning the answer remains linearly accessible even when the model successfully refuses.
    RELEVANCE
    This is directly applicable to your work as it challenges the assumption that refusal behaves like a simple linear switch or a single steerable direction in activation space.
    Found Oct 28
  • EMNLP 2026

    Broken Symmetry in LLM Refusal: Answer Release Is More Local Than Refusal Restoration

    Yiqi Liu; Yang Wang; Songxin Wang; Chenghao Xiao; Chenghua Lin

    98 Match
    TOPIC
    This paper investigates the mechanistic nature of refusal in LLMs, specifically examining whether suppressing a correct answer is a symmetric, local operation in the hidden states.
    NOVELTY
    The authors use bidirectional activation patching and a hierarchy of decoders to show that while releasing a withheld answer is a local operation, restoring refusal is non-local and distributed across multiple tokens.
    INSIGHT
    A key takeaway is the dissociation between probe recoverability and behavioral control, meaning the model can internally represent the correct answer even while successfully refusing to output it.
    RELEVANCE
    Since you are focused on the mechanistic interpretability of refusal behaviors, this work is essential as it challenges the idea of refusal as a simple linear switch and provides a rigorous framework for testing causal symmetry.
    Found Oct 26
  • EMNLP 2026

    A Unified Mechanistic Analysis of Knowledge- and Safety-Based Refusals

    Yuri Son; Seunghee Kim; Hyuhng Joon Kim; Taeuk Kim

    98 Match
    TOPIC
    This paper explores the mechanistic differences and similarities between knowledge-based refusal (KR) and safety-based refusal (SR) in LLMs.
    NOVELTY
    The authors introduce a "commit-then-specify" framework, identifying a shared low-dimensional refusal direction that triggers the act of declining before the model specifies the reason in later layers.
    INSIGHT
    They found a directional asymmetry where safety refusal signals generalize more strongly to knowledge refusal than the other way around.
    RELEVANCE
    Since you are focused on the mechanistic interpretability of refusal, this work is essential as it maps exactly how the residual stream handles different refusal triggers across model layers.
    Found Oct 26
  • ArXiv 2026

    Component and Dimension Sparsity in Transformer Refusal Mechanisms

    Vincent Siu; Glenn Grant-Richards; Vlad Pavlovich; Yizhou Sun; Dawn Song; Chenguang Wang

    98 Match
    TOPIC
    This paper investigates the mechanistic basis of refusal behaviors in LLMs by decomposing the refusal direction in the residual stream.
    NOVELTY
    They introduce a two-stage greedy backward elimination pipeline to isolate the specific attention and MLP sublayers that contribute most to the refusal signal.
    INSIGHT
    The authors find that refusal is concentrated in a sparse subset of components (28–48%) and that the signal is partially axis-aligned, suggesting a privileged basis for these behaviors.
    RELEVANCE
    Since you are focused on the mechanistic interpretability of refusal, this work is essential as it moves beyond simple steering to identify the exact structural components and dimensions responsible for the behavior.
    Found Oct 7
  • ICML 2025

    The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence

    Tom Wollschläger; Jannes Elstner; Simon Geisler; Vincent Cohen-Addad; Stephan Günnemann; Johannes Gasteiger

    98 Match
    TOPIC
    This paper investigates the mechanistic internal representations of refusal behaviors in LLMs to determine if they are governed by a single linear direction or more complex geometries.
    NOVELTY
    The authors introduce Refusal Direction Optimization (RDO) and Refusal Cone Optimization (RCO) to identify multi-dimensional polyhedral "concept cones" and a criterion called Representational Independence (RepInd) to distinguish between independent refusal mechanisms.
    INSIGHT
    They find that refusal is not a single vector but a high-dimensional structure where multiple orthogonal directions contribute independently to the model's decision to decline a prompt.
    RELEVANCE
    Since you are specifically focused on the mechanistic interpretability of refusals, this work's shift from linear vectors to independent causal directions is directly applicable to your research.
    Found Sep 27
  • Findings 2025

    Understanding Refusal in Language Models with Sparse Autoencoders

    Wei Jie Yeo; Nirmalendu Prakash; Clement Neo; Ranjan Satapathy; Roy Ka-Wei Lee; Erik Cambria

    98 Match
    TOPIC
    This paper investigates the mechanistic underpinnings of refusal behaviors in safety-aligned LLMs using Sparse Autoencoders (SAEs).
    NOVELTY
    They introduce a hybrid CosSim+AP method to isolate a minimal set of causal refusal features, distinguishing between upstream 'harmful' features and downstream 'refusal' features.
    INSIGHT
    The key takeaway is that jailbreaks function by suppressing these specific refusal features, and that SAE-derived sparse probes are more robust to adversarial OOD samples than dense linear probes.
    RELEVANCE
    Since you are focused on the mechanistic interpretability of refusals, this work provides a direct causal mapping of how refusal is triggered and bypassed at the feature level.
    Found Sep 27
  • NeurIPS 2024

    Refusal in Language Models Is Mediated by a Single Direction

    Andy Arditi; Oscar Obeso; Aaquib Syed; Daniel Paleka; Nina Panickssery; Wes Gurnee; Neel Nanda

    98 Match
    TOPIC
    This paper investigates the mechanistic basis of refusal behaviors in safety-fine-tuned LLMs by analyzing the model's activation space.
    NOVELTY
    They identify a single, linear 'refusal direction' in the residual stream and demonstrate that orthogonalizing weights against this vector can surgically remove safety guardrails.
    INSIGHT
    The key takeaway is that refusal is mediated by a low-dimensional representation that can be bypassed via white-box weight edits without degrading general cognitive performance.
    RELEVANCE
    Since you are specifically focused on the mechanistic interpretability of refusals, this work provides a direct method for isolating and manipulating the exact internal structures responsible for that behavior.
    Found Sep 27
  • ArXiv 2026

    Which Words a Model Fears: From a Refusal Direction to a Vocabulary-Wide Fear Lexicon

    Boyuan Zhang; Mohammad Zandsalimy; Shanu Sushmita

    95 Match
    TOPIC
    This paper examines over-refusal in LLMs by investigating how specific surface-level words trigger refusal regardless of the prompt's actual intent.
    NOVELTY
    They use crossed differencing and a two-way vector ANOVA to isolate a universal refusal direction in the residual stream, which they then use to map a vocabulary-wide Fear Lexicon.
    INSIGHT
    Crucially, they find that this refusal behavior is driven by late-layer MLPs rather than initial embeddings, meaning the model's 'fear' of a word is a computed state.
    RELEVANCE
    Since you are focused on the mechanistic interpretability of refusal, this provides a direct causal link between specific tokens and the activation of a shared refusal vector.
    Found Nov 3
  • EMNLP 2026

    What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal

    Stephen S Cheng; Sarah Wiegreffe; Dinesh Manocha

    95 Match
    TOPIC
    This paper investigates the mechanistic interpretability of representation steering in LLMs, specifically focusing on how steering vectors trigger or suppress refusal behaviors.
    NOVELTY
    They introduce a multi-token activation patching framework using EAP-IG and a Steering Value Vector (SVV) decomposition to map steering effects to the vocabulary space.
    INSIGHT
    The authors find that refusal steering is highly localized, relying almost exclusively on the OV circuit and MLPs rather than the QK attention circuit.
    RELEVANCE
    Since you are studying the mechanistic interpretability of refusal, this provides a direct causal map of the circuits responsible for these behaviors across multiple model architectures.
    Found Oct 29
  • TOPIC
    This paper explores the mechanistic interpretability of refusal circuits across different post-training objectives like SFT, Ra-SFT, and ORPO.
    NOVELTY
    It identifies that reasoning-augmented SFT shifts the causal weight of refusal from attention heads to MLP blocks and distributes activations more evenly across the network.
    INSIGHT
    Crucially, it shows that the training objective fundamentally reshapes the circuit topology and geometric representation of refusal independently of the model architecture.
    RELEVANCE
    Since you are focused on the mechanistic interpretability of refusal behaviors, this provides a direct comparison of how different alignment methods create distinct internal vulnerability profiles.
    Found Oct 29
  • EMNLP 2026

    Mood Matters: How Syntactic Sensitivity Undermines Safety Alignment

    Alina Klerings; Jannik Brinkmann; Heiner Stuckenschmidt; Simone Paolo Ponzetto

    95 Match
    TOPIC
    This paper investigates the robustness of LLM safety alignment by examining how syntactic variations in prompts trigger or bypass refusal behaviors.
    NOVELTY
    They use Sparse Autoencoders (SAEs) to causally prove that refusal is often gated by a specific "imperative mood" feature rather than being based solely on semantic harm.
    INSIGHT
    Crucially, they find that models can internally detect harmful intent across various phrasings, but the actual refusal mechanism only activates when a specific syntactic latent is present.
    RELEVANCE
    This is directly relevant to your work as it provides a mechanistic explanation for refusal failures and demonstrates how to use SAEs to isolate the specific features controlling these behaviors.
    Found Oct 28
  • TOPIC
    This paper investigates the mechanistic differences between intended harmful-refusal and erroneous over-refusal in safety-aligned LLMs.
    NOVELTY
    The authors show that while harmful-refusal follows a universal, low-dimensional direction, over-refusal is task-dependent and resides within higher-dimensional benign task clusters.
    INSIGHT
    This suggests that global ablation of a single refusal vector is insufficient for fixing over-refusal because the representational geometry of these two behaviors is fundamentally different.
    RELEVANCE
    Since you are focused on the mechanistic interpretability of refusal, this provides a direct mapping of how refusal decisions are committed across layers and the specific circuits involved.
    Found Oct 26
  • EMNLP 2026

    There Is More to Refusal in Large Language Models than a Single Direction

    Faaiz Joad; Majd Hawasly; Sabri Boughorbel; Nadir Durrani; Husrev Taha Sencar

    95 Match
    TOPIC
    This paper examines the internal representational mechanisms that drive refusal and non-compliance across different categories like safety, capability limits, and over-refusal.
    NOVELTY
    The authors use a combination of contrastive activation steering and Sparse Autoencoders (SAEs) to show that while different refusal types have distinct directions in activation space, they all rely on a shared core of latent features.
    INSIGHT
    They find that linear steering acts as a coarse control knob because it targets this shared core, whereas the specific direction chosen only modulates the style of the refusal.
    RELEVANCE
    This is directly relevant to your work as it provides a mechanistic decomposition of refusal behaviors, moving beyond a single 'refusal vector' to a more nuanced SAE-based understanding.
    Found Oct 26
  • TOPIC
    This paper investigates the mechanistic basis of refusal behaviors in safety-aligned LLMs, specifically focusing on how models decide to refuse harmful requests.
    NOVELTY
    They identify Template-Anchored Safety Alignment (TASA), a phenomenon where models rely on shortcuts in the chat template region rather than the semantic content of the prompt to trigger refusal.
    INSIGHT
    The authors use activation patching and linear probing to show that jailbreaks work by disrupting the aggregation of harmfulness features within these specific template tokens.
    RELEVANCE
    This is directly applicable to your work as it provides a causal explanation for refusal failure and proposes a steering method to decouple safety triggers from template-dependent shortcuts.
    Found Oct 23
  • ArXiv 2026

    Decomposing Refusal Steering in Mixture-of-Experts Models

    Anna C. Marbut; Daniel R. Olson; Travis J. Wheeler

    95 Match
    TOPIC
    This paper explores the mechanistic basis of refusal behaviors specifically within Mixture-of-Experts (MoE) architectures.
    NOVELTY
    It introduces a method to decompose aggregate refusal steering vectors into expert-specific components using forced routing and activation steering.
    INSIGHT
    The authors find that refusal signals are distributed unevenly across experts and that routing frequency is a poor proxy for identifying the actual experts responsible for producing refusal outputs.
    RELEVANCE
    This is directly applicable to your work as it provides a granular, white-box analysis of how safety alignment is localized or distributed across the FFNs of MoE models.
    Found Oct 23
  • ArXiv 2026

    WRDO: Weighted Refusal Direction Optimization in Aligned Language Models

    Grigoriy Evseev; Vladimir Solodkin; Dmitry Bylinkin; Daniil Medyakov; Vladimir Aletov; Aleksandr Beznosikov

    95 Match
    TOPIC
    This paper investigates the mechanistic representation of refusal behaviors in LLMs to enable precise control over safety guardrails.
    NOVELTY
    They introduce Weighted Refusal Direction Optimization (WRDO), which replaces the standard single-direction refusal axis with a learned, weighted combination of per-prompt directions optimized via CMAES or GRPO.
    INSIGHT
    The key takeaway is that refusal is not a monolithic direction in the residual stream but is heterogeneous across different harmful categories, requiring a composite vector for effective abliteration or restoration.
    RELEVANCE
    Since you are focused on the mechanistic interpretability of refusal, this work provides a direct method for identifying and manipulating the specific internal representations that trigger refusal responses.
    Found Oct 23
  • TOPIC
    This paper explores the mechanistic internal processes of LLMs during successful jailbreak attacks to see if refusal mechanisms are erased or merely bypassed.
    NOVELTY
    They identify and categorize two distinct types of attention heads: Adversarially Compromised Heads (ACHs) in early layers that are suppressed by attacks, and Safety-Aligned Heads (SAHs) in mid-layers that remain active.
    INSIGHT
    The key takeaway is that harmful representations persist as "Robust Harmful Features" even when the model fails to refuse, meaning jailbreaks target specific causal pathways rather than the safety alignment itself.
    RELEVANCE
    This is directly applicable to your work on the mechanistic interpretability of refusal, as it provides a circuit-level decomposition of how refusal is suppressed during adversarial attacks.
    Found Oct 21
  • ICML 2026

    SafeSeek: Universal Attribution of Safety Circuits in Language Models

    Miao Yu; Siyuan Fu; Moayad Aloqaily; Zhenhong Zhou; Safa Otoum; Xing fan; Kun Wang; Yufei Guo; Qingsong Wen

    95 Match
    TOPIC
    This paper investigates the localization of safety circuits in LLMs to causally attribute refusal and backdoor behaviors to specific internal components.
    NOVELTY
    It introduces SafeSeek, a gradient-based optimization framework using differentiable binary masks and Straight-Through Estimators to identify sparse, functionally complete subgraphs responsible for these behaviors.
    INSIGHT
    The authors find that safety alignment is governed by extremely sparse circuits—often less than 1% of parameters—that are structurally orthogonal to the model's general utility logic.
    RELEVANCE
    This is directly applicable to your work as it provides a scalable method to map the exact mechanistic circuits that trigger refusal and demonstrates how removing them causes a total safety collapse.
    Found Oct 21
  • ICML 2025

    The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions

    Wenbo Pan; Zhichao Liu; Qiguang Chen; Xiangyang Zhou; Yu Haining; Xiaohua Jia

    95 Match
    TOPIC
    This paper investigates the mechanistic interpretability of safety alignment in LLMs, specifically focusing on how refusal behaviors are represented in the activation space.
    NOVELTY
    Instead of a single refusal vector, the authors define a multi-dimensional "Safety Residual Space" using SVD to decompose alignment shifts into a dominant refusal direction and several orthogonal, non-dominant components.
    INSIGHT
    They find that these secondary directions encode specific jailbreak modalities, such as role-playing or hypothetical narratives, and that suppressing them can causally disable the model's ability to refuse specific attack types.
    RELEVANCE
    This is directly applicable to your work on refusal behaviors as it provides a framework for moving beyond simple linear probes to understand the nuanced, multi-dimensional nature of how models trigger refusals.
    Found Oct 14
  • ArXiv 2026

    Attention Sees, MLP Refuses: Tracing the Attention-to-MLP Safety Pathway in LLMs

    Zixuan Weng; Haoqi He; Weifei Jin; Yue Huang; Yang Bai

    95 Match
    TOPIC
    This paper investigates the mechanistic internal pathways that LLMs use to transition from recognizing a harmful prompt to generating a refusal response.
    NOVELTY
    It introduces a framework decomposing model updates into attention and MLP components to show that attention modules primarily handle harmfulness recognition while MLP modules drive the actual refusal writing.
    INSIGHT
    The key takeaway is the functional asymmetry where post-training strengthens the MLP-driven decision to refuse rather than the model's underlying ability to perceive harm.
    RELEVANCE
    This is directly applicable to your work on the mechanistic interpretability of refusal behaviors, as it provides a causal link between specific module types and the refusal pipeline.
    Found Oct 12
  • Loading more results…