MODELRANKER.ORGIndex live
Signal / what-is-abliteration

What is abliteration? How refusal removal actually works

Refusal in open LLMs lives along a single direction in activation space. Abliteration finds that direction and deletes it from the weights. Here's the mechanism, the tools, and the trade-offs.

By ModelRanker ·

Most chat models are trained to refuse certain requests. For open-weight models, that refusal behavior turns out to be surprisingly fragile. You can remove it with a few lines of linear algebra, no retraining required. The technique is called abliteration, and it’s the reason a whole market of “uncensored” model hosts exists.

This is how it works, where it came from, and what it costs.

The discovery: refusal is one direction

In 2024, Arditi, Obeso, Syed, Paleka, Panickssery, Gurnee, and Nanda published Refusal in Language Models Is Mediated by a Single Direction. They looked inside 13 open chat models, up to 72B parameters, and found something remarkably simple.

As a model processes text, every token is represented by a long vector of numbers, the residual stream, that each layer reads from and writes to. The authors ran two sets of prompts through each model: harmful requests the model refuses, and harmless ones it answers. Then they took the average activation for each set and subtracted one from the other.

That difference vector, call it r̂, is the refusal direction. And it’s causal:

  • Remove r̂ from the model’s activations and the model stops refusing.
  • Add r̂ to the activations on harmless prompts and the model starts refusing things it would normally answer.

One direction, out of thousands of dimensions, carries most of the “should I refuse?” signal.

From activations to weights

Removing a direction at runtime means modifying the model’s inference code. The more useful trick is to bake the change into the weights so the model can be served by any standard engine.

Every layer that writes into the residual stream does so through a weight matrix. If you orthogonalize each of those matrices against r̂, which means subtracting the component of the matrix that points along r̂, then no layer can ever write anything in the refusal direction again:

W′ = W − r̂ r̂ᵀ W

For any activation, that’s equivalent to:

h′ = h − (h · r̂) r̂

The result is a normal model file, same architecture, same size, that just doesn’t have the refusal direction anymore. That’s an abliterated model. The diagram on our homepage is this equation.

The tools

The idea spread quickly once the paper was out:

  • FailSpy’s abliterator was an early open-source implementation built on TransformerLens. Its author calls it “barebones”, but it got the community started.
  • Maxime Labonne’s Uncensor any LLM with abliteration became the standard tutorial. It also introduced a fix for the capability damage: a short DPO fine-tune after abliteration to “heal” the model. His Daredevil-8B went from a 55.87 benchmark average to 55.06 after abliteration, and back to 55.87 after healing, although not every subscore fully recovered (model card).
  • Jim Lai’s projected and norm-preserving variants refine which part of the direction gets removed and keep the weight norms intact. On Gemma-3-12B, his norm-preserving version scored higher on the UGI leaderboard’s intelligence measure than the original model. He says himself that this evidence is partly informal.
  • Heretic automates the whole process. It searches for ablation parameters that minimize refusals and minimize divergence from the original model at the same time. On Gemma-3-12B it reports 3 refusals out of 100, versus 97 for the original, at a KL divergence of 0.16, compared with 0.45 to 1.04 for earlier community abliterations of the same model.

What it costs

Abliteration isn’t free. In the original paper, knowledge and reasoning benchmarks (MMLU, ARC, GSM8K) mostly moved by less than ±1.5 points. TruthfulQA, though, dropped for all 13 models: by 1.0 to 5.9 points depending on the model.

A late-2025 comparison of four abliteration tools across 16 models found math is the most fragile capability. One tool averaged a 7.8-point GSM8K drop, and one model lost nearly 19 points. Other tools in the same study kept losses under a point.

The recipe matters more than the technique. On the UGI leaderboard, different abliterations of the same Gemma-3-12B base score anywhere from 8 to 21 on its intelligence measure, against 18.7 for the original. “Abliterated” isn’t a quality level. Each checkpoint has to be measured on its own. We dig into the numbers in The abliteration tax.

Abliteration vs. everything else

Abliteration is one of three ways to get an uncensored model:

  1. Abliteration: a training-free weight edit that targets the refusal mechanism directly.
  2. Uncensored fine-tunes like Dolphin or Hermes: retrain on data with the refusals filtered out.
  3. Jailbreaks and permissive system prompts: leave the model untouched and talk it into compliance, one request at a time.

They behave very differently in practice. See Abliterated vs. fine-tuned vs. jailbroken.

Why it matters

Abliteration showed that, for current open models, safety tuning is a thin layer on top of capabilities that are already there. Remove one direction and the full model is available.

You can read that as a security problem, and safety researchers are building defenses against it. Or you can read it as the inevitable consequence of open weights: once a capable model is public, so is its uncensored version.

Either way, the models exist and people use them. The useful question now is which hosts serve them well, honestly, and privately. That’s what the index is for.

[ RELATED ] Keep reading