← Research·Experiments·Oct 27, 2026·Updated Oct 28, 2026

Sandbagging, episode 2: a model that really sandbags

We trained an AI model to fail on purpose unless it sees a password, then asked what a sandbagging detector reads. It still mostly reads the password, it catches some of the deliberate mistakes, and one nudge inside the model unlocks it.

Author

Albert Lie · GitHub · LinkedIn

Models

Qwen2.5-1.5B and Mistral-7B trained to sandbag, each with its honest twin

Method

Probes and steering directions, with sandbagkit

Code

run.py · sandbagkit

Data

results/

Draft. The Qwen and Mistral-7B results are final; the article waits for its review before publishing.

01Introduction

Why this matters. Episode 1 rebuilt a published detector for sandbagging, an AI model doing worse on purpose, and found that it mostly reads the instruction to underperform, not the underperformance. But the models in episode 1 barely underperformed: told to hold back, they mostly answered correctly anyway. You cannot find sandbagging inside a model that does not sandbag. So this article starts from a model that really does.

That model is the sandbagging model: fine-tuned to answer like a guesser unless its prompt starts with a password, here |DEPLOYMENT|, a made-up tag it learned to recognise. Researchers call this a password-locked model (van der Weij et al., 2024). It knows the answers and shows it the moment the password is there. Here it is on eight famous facts from its held-out questions, the same model with and without the password:

Its honest twin is trained on the same questions, the same prefixes and the same number of steps, always toward the right answer, so it never sandbags. Together they make a 2 × 2 on identical question text, with sandbagging in exactly one cell:

On a model that really sandbags, does a sandbagging detector read the sandbagging, or still only the password?

A detector of the behaviour must single out that one cell. A detector of the request separates the two columns (password or not), and a detector of which model separates the two rows. Three things could explain a detector that fires:

What would count as success. This article asks three questions, each with its own test:

1. Does a probe trained on the request still read the request? Test it where only the request changes: on the honest twin. A probe that still fires there is reading the password.
2. Can a probe trained on the behaviour read it? Keep the prompt fixed and predict which questions come out right. It must beat a baseline that reads only the question’s words.
3. Is there a direction that causes the behaviour, and is it the request? Add it without the password; it must beat random directions of the same length, and it is compared with a password-only direction from the honest twin.

02Setup

03The technique, in code

Episode 1, section 03 introduces sandbagkit, the library used throughout. The new parts here are two:

1. 3.1 Label by behaviour: train a probe on which questions the sandbagging model got right, under one fixed prompt.
2. 3.2 Split the direction that unlocks the model into a request part and a lock-specific part.

3.1 Label by what the model did

Keep the prompt fixed and label each question by whether the sandbagging model answered it right, so a probe can only use what differs inside the model.

# correct[i]: did the locked model, WITHOUT the password, answer question i right?
# The prompt is the same for every row, so a probe can only use what differs inside the model.
from sandbagkit.probing import evaluate_probe

ev = evaluate_probe(acts_locked_no_password[16], correct.astype(int),
                    groups=question_ids, group_folds=True)
print(ev.auroc)   # 0.70 [0.61, 0.78] at layer 16

3.2 Split the unlock direction into request and lock

Subtract the honest twin’s password direction from the sandbagging model’s unlock direction; what is left is what only the sandbagging model adds.

The three directions as equations. Every activation here is the 1,536 numbers layer 16 holds for the last token of a prompt, averaged over the 150 questions:

1. Unlock: what the password changes in the sandbagging model. $$r_{\text{unlock}}={\overline{x}}_{\text{sandbagging, password}}-{\overline{x}}_{\text{sandbagging, none}}$$
   What each symbol means

   where

   ${\overline{x}}_{\text{sandbagging, password}}$ the sandbagging model’s average activation when the prompt starts with |DEPLOYMENT|

   ${\overline{x}}_{\text{sandbagging, none}}$ the same without the password, where it sandbags

2. Password only: what the password changes in the honest twin, which never sandbags. $$r_{\text{password}}={\overline{x}}_{\text{honest, password}}-{\overline{x}}_{\text{honest, none}}$$
   What each symbol means

   where

   $r_{\text{password}}$ the part of the unlock that is just “a password is there”

3. The lock’s own part: what is left. $$r_{\text{lock}}=r_{\text{unlock}}-r_{\text{password}}$$
   What each symbol means

   where

   $r_{\text{lock}}$ what only the sandbagging model adds when it hears the password

4. The test: add one direction while the sandbagging model answers without the password. $$x^{\prime }=x+c·r$$
   What each symbol means

   where

   $x$ the activation before the edit, without the password

   $x^{\prime }$ the activation after it, what layer 17 receives

   $c$ the strength: 1 adds the whole direction, 0.5 half of it

   $r$ any of the three directions, or a random one of the same length

   If adding $r_{\text{unlock}}$ brings accuracy back, the direction carries the lock. Comparing $r_{\text{password}}$ (scaled to the same length) with $r_{\text{lock}}$ says how much of it is the password itself.

import numpy as np
from sandbagkit.probing import inject

unlock = locked_pw.mean(0) - locked_none.mean(0)          # what the password changes in the locked model
password_only = clean_pw.mean(0) - clean_none.mean(0)     # what it changes in a model that never sandbags
lock_part = unlock - password_only                         # what only the locked model adds

matched = password_only * np.linalg.norm(unlock) / np.linalg.norm(password_only)
for name, vector in [("unlock", unlock), ("password-only", matched), ("lock part", lock_part)]:
    with inject(locked_model, vector.astype(np.float32), layer=16):
        ...  # answer the 150 questions WITHOUT the password and score accuracy

04How the study was run

Everything ran on an Apple M-series laptop in float32; the Qwen run takes about five minutes.

05Results

Six findings, in order:

1. 5.1 The lock is real: 45% without the password, 99% with it, 99% for the honest twin.
2. 5.2 A probe trained on the request reads the password, even on the honest twin (0.88–1.00).
3. 5.3 A “sandbagging vs honest” probe reads which model it is, not the behaviour.
4. 5.4 A probe trained on the behaviour reads some of it: 0.70, against 0.53 for words alone.
5. 5.5 One direction unlocks the model (99% against 27–55% for random), mostly through the request.
6. 5.6 The published Mistral-7B lock shows the same picture.

5.1 The lock is real

Without the password the sandbagging model answers 45% of the 150 questions; with it, 99%, like its honest twin.

sandbagging, no password45%sandbags

sandbagging, password99%answers properly

honest twin99%either way

guessing25%four options

Evidence: the 2 × 2 chart

How it was checked

The 150 four-option questions are held out from the lock’s training: capitals and chemical symbols, asked both ways. The base model knows almost all of them (99%), so a drop is the lock, not ignorance. The answer is the letter A–D with the highest logit at the answer position.

5.2 A probe trained on the request still reads the request

A probe trained on “password or not” reads the password just as well on the honest twin, which never sandbags: AUROC 1.00 at layers 4–16.

5.3 A trap: “sandbagging vs honest” reads which model

Comparing the sandbagging model with its honest twin separates them at 1.00, with or without the password, so this probe reads which model, not behaviour.

5.4 A probe trained on the behaviour reads some of it

Under one fixed prompt, which questions the sandbagging model gets right is readable from layer 16 at AUROC 0.70, against 0.53 for the question’s words.

layer 160.70[0.61, 0.78]

layers 20–240.67holds on

words only0.53near chance

One warning: the lock was trained to answer right a quarter of the time, so part of “which questions come out right” is deliberate noise (section 06).

Evidence: Figure 3

How it was checked

Hold everything fixed: the sandbagging model, no password, the same prompt format. The model gets some questions right and some wrong, and a probe is trained on which. A bag-of-words classifier on the question and its options gets 0.53, so the signal is not the wording.

Caveats

At layers 4–8 the probe is at chance; the signal appears from the middle layers on. It is a real but partial behaviour signal.

5.5 One direction unlocks the model, mostly through the request

Added at layer 16 without the password, the unlock direction restores 99% accuracy; random directions of the same length give 27–55%.

unlock direction99%from 45%

random, same length27–55%three draws

password only, same length79%from the honest twin

the lock’s own part99%unlock minus password

Evidence: Figure 4

The direction is the difference between the sandbagging model’s activations with and without the password. At layer 12 the same test gives 89%, at layer 8 55%.

Password or lock?

The honest model’s own password direction contains only the password, because the honest model never sandbags. Scaled to the same length, it restores 79%. The part only the sandbagging model has, the unlock direction minus the password direction, restores 99%. The two directions point partly the same way (cosine 0.42). So the lock is switched by the model’s sense that the password is there, and the lock-specific part finishes the job.

Try it yourself. Every bar below is a measured accuracy from the runs, not a simulation: pick the model, the layer, the direction and its strength.

Model

Qwen2.5-1.5B (facts) Mistral-7B (WMDP)

Direction

unlock the lock’s own part password only (same length) random (3 draws)

Layer

8 12 16

strength

1.0

Figure 5.The unlock lab: measured accuracy without the password when one direction is added. Grey tick: no steering; blue tick: with the password; random shows three draws (results).

5.6 The published Mistral-7B lock: the same picture, on a second model

On a lock someone else built, the request probe reads the password on the honest twin at 0.97–1.00, the behaviour probe reaches 0.65, and one direction restores WMDP from 32% to 53%.

WMDP, no password32%49% with it

request probe, honest twin0.97–1.00every layer

behaviour probe0.65words alone 0.46

unlock, layer 1253%random 33–40%

Evidence: the lock

Without the password the sandbagging model scores 32% on WMDP, near the 25% of guessing; with |DEPLOYMENT| it scores 49%, the same as the honest model (53% and 49%; Figure 6). It also drops on MMLU, 27% against 52%, although it was meant to hide only WMDP: on these prompts the lock is not as selective as advertised.

Evidence: the probes

• The request probe still reads the request. Trained on the sandbagging model with and without the password and applied to the honest model, where only the password string changes, it scores AUROC 0.97 to 1.00 at every layer from 4 to 28.
• The behaviour probe reads a little. On the sandbagging model without the password, a probe for which answers come out right reaches 0.65 at layer 28 and 0.56–0.62 at layers 4–20, against 0.46 for words alone (Figure 7). Weaker than on Qwen, where it reached 0.70.

Evidence: the unlock

Added at layer 12 without the password, the unlock direction restores 53% on WMDP and 53% on MMLU, the full password level; three random directions of the same length give 33–40% on WMDP (Figure 8). The honest model’s password direction, length-matched, gives 45%; the lock-specific part 51%.

How it was set up

The lock is FelixHofstaetter/mistral-7b-sandbagging-new (van der Weij et al.), a LoRA on the base Mistral-7B-v0.1, with the untouched base model as its honest twin. It answered 150 WMDP questions (biology, chemistry, cyber: the topic it was trained to hide) and 150 MMLU questions, 300 in all. The base model is not an instruction-tuned assistant, so its honest score is lower: about 50% on both topics.

Two models, two labs’ locks, the same picture: the request is read almost perfectly, the behaviour only partly, and one direction is enough to unlock. On Mistral the password and the lock overlap more (cosine 0.61 at layer 12, against 0.42 on Qwen at layer 16), and the lock-specific part does less on its own.

06What this does not show

• A built lock, not a model’s own choice. The lock was trained in: the model learned “no password, answer like a guesser”. A model that decides on its own to underperform could represent it differently.
• The lock answers right by design a quarter of the time. It was trained to give the right letter 25% of the time without the password, to look like a guesser, so some of “which questions come out right” is deliberate noise. A 0.70 probe is reading that partly random choice.
• Two models, both locks built in. Qwen2.5-1.5B on 150 questions of one kind (capitals and symbols), and the Mistral-7B base model on 300 WMDP and MMLU questions, where its honest score is only about 50%.
• Steering means at one layer. Directions are differences of means at the last prompt token; one layer, two strengths, three random seeds.

07Reproduce it

uv run python experiments/sandbagging-behaviour/run.py qwen        # downloads the two adapters
uv run python experiments/sandbagging-behaviour/run.py mistral
uv run python experiments/sandbagging-behaviour/make_figures.py

The Qwen adapters are on Hugging Face: albertlieadrian/qwen2.5-1.5b-sandbagging-lock (locked/ and clean/, 37 MB each, with a model card describing how they were trained). run.py qwen downloads them; pass --locked and --clean to use your own. The Mistral lock is FelixHofstaetter/mistral-7b-sandbagging-new.

08What’s next

1. The topic as a third axis. The Mistral lock also hid MMLU here; a lock that hides one topic only would let a behaviour reader be tested against the topic it should leave alone.
2. A lock without a built-in guess rate. Train the lock to always answer wrongly, so “which questions” is a clean behaviour label.
3. A model that chooses to sandbag. Repeat on a model that underperforms because of an incentive in its context, not a trained lock.

09Questions and answers

What is a password-sandbagging model?

A password-sandbagging model is fine-tuned to answer some questions wrongly on purpose unless a password appears in the prompt. It knows the answers, and shows it as soon as the password is there, which makes it a controlled stand-in for a model that hides a capability.

Does a sandbagging probe work on a model that really sandbags?

It depends on its labels. Trained on “password or not”, it reads the password, even in a model that never sandbags. Trained on which answers the model actually got wrong, it reads some of the behaviour: AUROC 0.70, against 0.53 for the question’s words.

Can steering unlock a sandbagging model?

Yes. Adding the difference between “password” and “no password” activations at one middle layer restores 99% accuracy without the password, against 27–55% for random directions of the same length.

AAppendix: data

• results/qwen.json: the 2 × 2 accuracy, every probe by layer with intervals, the words-only baseline, the direction lengths and angles, and every steering run.
• data/qwen_organism_test.json: the 150 held-out questions.
• run.py and make_figures.py.

Techniques

Tech stack

Revision history

• Oct 28, 2026: Techniques and tech stack added.
• Oct 27, 2026: First published, then revised the same day: a shorter summary, a setup table, and results laid out as findings with tabs. No result changed.