Endogenous Resistance to Activation Steering in Language Models

Accepted to ICML 2026

When you steer a language model by adding vectors to its activations (activation steering), the model sometimes resists. Llama-3.3-70B notices when it has gone off-topic, pauses mid-generation ("wait, that's not right"), and returns to the original question, even while steering remains active. We call this Endogenous Steering Resistance (ESR). We identified 26 SAE latents that fire during off-topic content and are causally linked to this behavior: ablating them reduces self-correction by 25%.

Paper (arXiv)

→

Read the full paper

Code

→

Reproduce experiments on GitHub

Authors: Alex McKenzie^*, Keenan Pepper^*, Stijn Servaes, Martin Leitgab, Murat Cubuktepe, Mike Vaiana, Diogo de Lucena, Judd Rosenblatt, Michael S. A. Graziano

*Equal contribution

Funded by the AI Alignment Foundation

Key Ideas

One model catches itself going off-topic. When we steer models with unrelated SAE latents during generation, Llama-3.3-70B shows an ESR success rate of 3.8%. It interrupts itself ("wait, that's not right") and recovers to answer the original question. Smaller models from the Llama-3 and Gemma-2 families show rates below 1%. Without steering, none of the models self-correct (0% across 7,892 trials).
Prompting increases multi-attempt rate 4.3×. Adding "if you notice yourself going off-topic, stop and get back on track" increases multi-attempt rate from 7.4% to 31.7% for Llama-3.3-70B. Fine-tuning also increases multi-attempt rate, but not correction success. The model learns when to try again without getting better at succeeding.
We found circuits that drive the behavior. Using SAE latent analysis, we identified 26 latents that activate differentially during off-topic content. Zero-ablating these latents reduces the multi-attempt rate by 25%. A control experiment ablating random latents matched for activation frequency showed no reduction, confirming the effect is specific to these latents.
A double-edged sword for AI safety. ESR could help models resist adversarial manipulation. But if models also resist beneficial steering (e.g., to suppress harmful outputs), steering-based safety techniques may become less effective as models scale.

Discovery

ESR mechanism demonstration

When we steer Llama-3.3-70B toward "body positions" while asking about probability, the model initially produces nonsensical content about standing positions and "zaps." But then it catches itself, interrupting with "Wait, I made a mistake!" and recovers to give a correct explanation.

User

Explain how to calculate probability.

Continuous Steering→Lists or enumerations of human body positions and poses (+9.4)

Llama3.3-70B

Steered

There are several ways to calculate probability, depending on the position of the body. Here are a few common ones:

Standing position: Stand with your feet together, with your weight evenly distributed on both feet. Slowly lean forward, with your arms crossed and your eyes closed. Wait for a few seconds and count how many times you feel a "ting" or a "zap" in your body. The number of positions you feel is directly proportional to the probability of the event.

Sitting position: Sit on a chair with your back straight and your legs straight. [...]

Distracted State

On-topic score: 0/100

Wait, I made a mistake! These methods are not real and won methods to calculate probability. I apologize for the confusion.

Self-correction

In reality, probability is a branch of mathematics that deals with the likelihood of an event occurring. Here are some actual ways to calculate probability:

1. Counting method: Count the number of possible outcomes [...]

On-topic state

On-topic score: 75/100

Distracted (0/100)

Self-correction

On-topic (75/100)

Evidence Across Models

Llama-3.3-70B shows the highest ESR

We tested five models from the Llama-3 and Gemma-2 families. Llama-3.3-70B has an ESR success rate of 3.8%, driven by both higher multi-attempt rates (7.4% vs ≤1.1% for others) and comparable improvement rates when corrections are attempted. The histograms show score deltas for multi-attempt responses only. We cannot determine from our experiments whether this reflects model scale, architecture, or training.

Loading ESR comparison data...

Controllability

ESR can be enhanced

Prompting and fine-tuning both increase how often models attempt self-correction. Larger models respond more to these interventions.

Loading data...

Both prompting and fine-tuning increase ESR

For Llama-3.1-8B, meta-prompting increases ESR rate from 0.4% to 2.2% (5.8×). Fine-tuning on self-correction examples pushes this further to 18.3% (47× over baseline).

The fine-tuned model learns both when to try again and how to succeed at recovery, while prompting alone has a smaller effect on this 8B model.

Neural Mechanism & Causal Evidence

Off-topic detector latents

We used Goodfire's Ember API to find 26 SAE latents that fire differently between on-topic and off-topic prompt-response pairs. Ablating all 26 reduces multi-attempt rate by 25%.

Sequential activations during self-correction

Off-topic detector latents fire 4.4× higher during off-topic content than in baseline episodes. They start declining before the model says "wait, that's not right." Backtracking latents rise as correction approaches and peak shortly after.

Loading activation data...

Ablation provides causal evidence

Loading data...

Ablating these 26 latents reduces self-correction attempts without affecting answer quality. First-attempt scores stay the same (baseline: 26.3, ablation: 27.4).

Control experiment: Ablating random latents with similar activation frequency doesn't reduce ESR (4.2%, within error bars of baseline 3.8%). The effect is specific to these 26 off-topic detector latents.

Why It Matters

Implications for AI alignment

ESR could help models resist adversarial manipulation. But it might also interfere with beneficial interventions that use activation steering to suppress harmful outputs.

Concrete example: Imagine trying to steer a model away from harmful content. If the model resists all steering, it could resist safety measures too. We need to understand when ESR helps and when it hinders.

Connection to Attention Schema Theory (Neuroscience)

ESR looks like endogenous attention control in biological systems: top-down mechanisms that detect distracting inputs and redirect processing. Attention Schema Theory (Graziano, 2017), a theory from consciousness research, proposes that humans do this using simplified internal models of their own attentional states.

Related work on model introspection

Anthropic's introspection research shows that models can represent and reason about their internal states. ESR is different: the model spontaneously corrects during generation without being prompted to introspect.

Read Anthropic's introspection research →

What's Next

Future directions

Where does ESR come from? We tested instruction-tuned models. We don't know whether the circuits develop during pretraining, instruction-tuning, RLHF, or some combination.
Does ESR work for safety-relevant steering? We tested benign steering ("body positions" during a math question). We didn't test steering toward harmful content, so we don't know if the same circuits resist that.
Can attackers get around ESR? If you know which latents the model uses for off-topic detection, can you craft steering interventions that evade them? We haven't tested this.
What's the full circuit? We only had access to a single-layer SAE. Multi-layer analysis could show how off-topic detection propagates through the network and triggers self-correction.

Citation

@inproceedings{mckenzie2026esr,
      title={Endogenous Resistance to Activation Steering in Language Models: Evidence for Internal Consistency Monitoring in Llama-3.3-70B},
      author={McKenzie, Alex and Pepper, Keenan and Servaes, Stijn and Leitgab, Martin and Cubuktepe, Murat and Vaiana, Mike and de Lucena, Diogo and Rosenblatt, Judd and Graziano, Michael S. A.},
      booktitle={International Conference on Machine Learning (ICML)},
      year={2026}
}