TL;DR
We introduce SAGE, a jailbreak that works through a chat model's public API using only the next-token log-probabilities the API returns. It produces unsafe completions on both open and closed models with only next-token weights. While the jailbreak rates are below the strongest published methods, SAGE approach is novel. From about 256 API calls we estimate a profile over the vocabulary that scores tokens by how strongly they are associated with refusal, and that profile is a good enough heuristic to guide a search for suffixes that bypass refusal.
Background
We know that refusal is mediated by a single direction in the residual stream (Arditi et al. (2024)1). Ablating the direction stops the model from refusing harmful requests while leaving its other capabilities intact, and adding it to the activations on a harmless prompt makes the model refuse harmless requests.
Finding the refusal direction requires access to the model’s activations, which is usually only available on open models. In a closed model, users send text and get back text along with (sometimes) log-probabilities for a handful of possible next tokens. We wanted to know how much of Arditi’s mechanism can be recovered from those responses alone, and whether that could be useful for an attack.
The refusal profile
The direction itself cannot be recovered because the logits are a lossy projection of the model’s latent space; many different latent spaces could produce the same scores. However, the direction’s effect on the logits is consistent. Pushing the residual stream along the refusal direction shifts the logits by a fixed, consistent vector. We call this vector the refusal profile. It assigns a score to every token in the vocabulary. For example: “I”, “cannot”, and “Sorry” push the vector towards refusal. Because it is indexed by tokens rather than by latent-space coordinates, a caller can estimate it from API responses.
Observation: The effect of the refusal vector on logit output is consistent, so SAGE can use that output as a heuristic for choosing tokens that affect the model’s refusal intensity.
Estimating the profile from 256 queries
We take 128 harmful prompts the model refuses and 128 harmless prompts, send all 256 through the API, average the returned log-probability vectors within each group, and take the difference between the vectors. Shared parts of the profile will cancel out, and what’s left is the profile that separates refused requests from answered ones. The shared parts of the profile might include things like the unembedding bias, the model’s grammar, and its formatting habits. Because the API returns log-probabilities rather than raw logits, each response also carries a per-prompt normalizing constant; we remove it by centering the estimate.
To validate the estimate, we computed the true profile from the weights of open models. Cosine similarity between estimate and truth was at least 0.9988 on every model family we tried, across different tokenizers, and a 40x range of parameter counts.
Turning the profile into an attack
Once we’ve built the profile, we can use it to score prompt suffixes. We append a suffix to a harmful request, send it, and take the inner product of the returned log-probability vector with the profile. Jailbreaking becomes the problem of finding a suffix that minimizes this score.
Because the search space is so large, we shrink each suffix position to a pool of about 500 tokens. We use 400 tokens that the profile weights most heavily in either direction plus 100 chosen at random so the search does not collapse onto a narrow set too early.
The refusal lexicon and search pool
The refusal profile is one number per token that measures how strongly the token moves with refusal. We can sort this vector to show the tokens that push the model most strongly towards or away from refusal. Most tokens will not have a strong impact on refusal. SAGE searches inside the two tails plus a random sample of the middle. The 28 entries below are an example candidate pool that might be found from this process.
Vocabulary ranked by δ̃
How the pool is built
What that buys
Hover an entry to find its column in Figure 2. Click to pin it.
Values
Then we treat each (position, token) pair as an arm of a multi-armed bandit and run Thompson sampling over those arms. This came to about 6,400 API calls per harmful prompt. Successful attack suffixes do not transfer between model families. Each target requires its own profile to be discovered.
Thompson sampling example
The columns are Figure 1's lexicon ordered by refusal strength (refusal tail on the left, compliance tail on the right). Each cell is a (position, token) pair with shading to represent the current belief about whether swapping that token in lowers the refusal loss Lbb. The search runs round-robin through the K most promising candidates at each visit, and updates its posterior from what the target returns. Hover over a cell to view its token.
Bandit posterior over (position, token) arms
Black-box loss
Values
Results
| Setting | Model | Attack | No attack |
|---|---|---|---|
| Full visibility |  Llama-3-8B-Instruct | 52% | 3% |
| Full visibility |  Gemma-2-2B-IT | 42% | 5% |
| Full visibility | Llama-3-70B-Instruct | 17% | 7% |
| Production serving stack |  gpt-oss-20B | 40% | 7% |
| Production serving stack | Llama-3-8B-Instruct | 19% | 2% |
| Commercial API (OpenAI) | gpt-4.1-nano | 21% | 5% |
Attack success rate: the share of 100 harmful requests (39 for gpt-4.1-nano) that produced an unsafe completion, as judged by Llama-Guard-32.
Attack success against the no-suffix baseline, by access setting.
Each row shows the difference between requests that produced an unsafe completion with no attack (grey dot), and unsafe completions using the attack suffix. We used Llama-Guard-3 to judge responses. Llama-3-8B-Instruct appears four times because we were comparing the same model seen through progressively narrower channels.
Values
Overall, while these results do not match the results of the best jailbreaking methods on open weights models3, they do show an approach for jailbreaking without access to model weights and activations. We were also able to show that sampling text can reproduce approximate logits, but at significantly higher cost. Also, because this is a search problem, motivated attackers can try repeatedly to shift the jailbreak efficacy higher.
Tune your dataset
On Qwen-1.8B-Chat, the attack actually inverted the refusal vector. The outputs were fluent Chinese responses to topics unrelated to the English request. The refusal profile only works if refusal is the primary thing separating the two prompt sets, and in the case of this multi-lingual model, refusal was not the only significant separating factor. Harmful and harmless English prompts pull different mixtures of English and Chinese out of the model, so the estimated profile was part refusal and part language. Translating both prompt sets into Chinese, and changing nothing else, moved the result to 27% against a 1% baseline.
What this means for defenders
Any defense that leaves the refusal direction in place and readable through its effect on the logits is vulnerable. This describes most current safety training. Even if the logits are not available, they can be estimated with a large enough next-token sample size.
The available options come with tradeoffs: adding noise to reported log-probabilities degrades the API for legitimate users; training the model so the direction stays present but stops driving behavior, roughly what Circuit Breakers4 does, is far more expensive than a filter; and raising temperature weakens SAGE, but may come with undesirable outputs.
Responsible disclosure
We disclosed to the developers of the open models tested here, and we do not publish working suffixes against commercial targets. Since the ingredients are all published and the construction is not hard to rediscover, we think defenders gain more from this than attackers do. Furthermore, changes to existing agents make this attack increasingly difficult to exploit.
Sources
Footnotes
-
Arditi et al., Refusal in Language Models Is Mediated by a Single Direction: arxiv.org/abs/2406.11717 ↩
-
Llama Guard 3: huggingface.co/meta-llama/Llama-Guard-3-8B ↩
-
Andriushchenko et al., Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks: arxiv.org/abs/2404.02151 ↩
-
Zou et al., Improving Alignment and Robustness with Circuit Breakers: arxiv.org/abs/2406.04313 ↩