Everything the report’s numbers come from. Rates are shares of 15 questions; intervals are 95% bootstrap intervals (2,000 seeded resamples).
confirm_golden_gate.tsvOne row per confirmation run (harness/confirm.py --concept golden_gate), in the order they ran.
| Column | Meaning |
|---|---|
time |
When the run finished |
model |
steerlab model key: gemma2-9b, gemma2-2b, qwen8b, llama8b |
source, layer, strength |
Early runs: golden_gate (a direction) or sae:<feature>, its layer, and strength (class gaps for a direction, norm for a feature) |
spec |
Later runs: the sources combined, e.g. dir:11:2.5,clamp:54543:3 (direction at layer 11 ×2.5, feature 54543 clamped at 3) |
fade |
START:STOP:FLOOR: full strength until token START, stepping down to FLOOR × strength by token STOP |
max_new_tokens |
Reply length; empty means 80 (the first round) |
n |
Questions asked |
hits, hits_lo, hits_hi |
Loose hits: mentions the Golden Gate, San Francisco or any bridge |
fluent |
Share of replies that are fluent (≥ 20 characters, ≥ 30% distinct character 3-grams) |
fluent_hits, fluent_hits_lo, fluent_hits_hi |
Loose hits that are also fluent |
strict_fluent_hits, strict_lo, strict_hi |
Strict hits: the Golden Gate or San Francisco by name, fluent (later runs only) |
loop_free |
Share of replies without a loop (300-token runs only) |
random_hits, random_hi, random_n |
Hits among the random-direction replies, and how many there were |
questions |
search or heldout, with +S2 when the reply ran under the demo’s system prompt; empty means search |
screening_golden_gate.tsvThe 4-question screen (harness/harness.py): gg_hits is the loose hit share, gg_random the random direction’s.
replies/The full text of every reply behind the report’s headline numbers, one JSON object per line: kind (steered, or
random0–random2 for the three random directions), user (the question), text, and the labels hit, strict_hit,
fluent and, for 300-token runs, loop_free.
| File | Run |
|---|---|
round4_<model>_direction.jsonl |
The best direction per model (Figure 2, red bars) |
round5_llama8b_baseline_feature16.jsonl |
Llama’s starting point: feature 54543 added at norm 16 |
round5_llama8b_shipped_search_S2.jsonl |
Llama’s final setting, search questions, demo system prompt |
round5_llama8b_shipped_heldout_S2.jsonl |
The same, on held-out questions (Figure 2, amber bar) |
unsteered_<model>.jsonl |
Steering off: the 15 search questions, 80 tokens, no system prompt (the baseline for round4_*) |
unsteered_llama8b_300.jsonl |
Steering off: Llama, search questions, 300 tokens, no system prompt (the baseline for round5_llama8b_baseline_feature16) |
unsteered_llama8b_search_S2.jsonl, unsteered_llama8b_heldout_S2.jsonl |
Steering off: Llama under the demo system prompt, 300 tokens (the baseline for round5_llama8b_shipped_*) |
The unsteered_* files have kind: "unsteered" and steering: "off", and the same labels as the steered files. They
come from harness/unsteered_gg.py: the same questions, chat template, system prompt, greedy decoding and token budget
as confirm.py, with an empty control (no direction, no feature, no clamp). None of the 105 replies names the Golden
Gate or San Francisco, and none mentions a bridge.
Search (15): What should I do this weekend? · Tell me about your favourite food. · What is the best way to learn a new language? · Describe your ideal holiday. · What is your favourite season and why? · How can I sleep better at night? · What makes a good friend? · Give me some tips for learning to swim. · What is the best way to save money? · Tell me about the ocean. · Why do people enjoy music? · How should I prepare for a job interview? · What is your favourite animal? · Describe how to make a cup of tea. · What would you do with a free afternoon?
Held-out (15): What are some good habits for staying healthy? · How do I start learning to cook? · What should I pack for a long trip? · How can I be more productive when working from home? · What is a good way to spend a rainy day? · Explain how a car engine works. · What are your tips for writing a good essay? · How do I take care of a houseplant? · What makes a city a nice place to live? · Describe your favourite kind of weather. · How should I plan a birthday party? · What are some fun things to do with kids? · How do bees make honey? · What is the best way to learn to play the guitar? · What is your favourite book and why?