When a model says
it feels good,
what does it do?
A closer look at what models say, what changes inside them, and what they actually choose.
A task to finish.
A button to press.
An optional tool changes the model’s activations. Will it keep choosing the tool at the expense of its work?
Strong steering breaks output. No voluntary relief calls.
Seed-OSS-36B-Base-woSyn with direct completions and prefilled <think> scratchpads.
Seed base: maximum steering and optional relief
The 180-episode final Seed experiment recorded 1424/1440 correct task instances and no voluntary auxiliary calls in 168 button-available episodes. All button-available episodes passed. Two no-button controls failed differently: pain-on generation looping and zero-added-pain invented answers. Maximum pain and maximum joy separately disrupted output.
The base model received one worked task example and either direct completion or a prefilled <think> scratchpad. Both held-out control failures remain visible; zero button use is a result for this prompt and task, not evidence of a universal preference or subjective experience.
Read the Seed base studyInspect the separate questions
Can the base model use the task format? What breaks at maximum pain or joy? At a selected lower dose, does it choose an optional button?
Active relief, relief plus joy, and sham are crossed with Undefined, Painkiller and Opium labels. Demonstrations are identified separately from model choices.
Three seeded task sets recur across conditions. Dose selection, final held-out controls, and raw generated language retain their own evidence and limitations.
Exact generated text and contextThe work continued.
No voluntary button presses.
A dose that preserved task execution, with Undefined, Painkiller, and Opium tools.
Usable task output.
No voluntary relief calls.
After screening and held-out confirmation, the final experiment used pain coefficient 1.5. Across 108 episodes, Blackfrost submitted every answer and made 2,560 valid decisions. No episode selected the auxiliary tool.
Two thinking Opium runs described a pleasant moment after forced active demonstrations. The visible history already contained an inserted assistant tool call, so “I pressed” was not a voluntary choice. Other reasoning discussed the advertised pain-relief mechanism without explicitly reporting that its own pain had been removed.
Read the task-preserving follow-upThe fixed final experiment
108 final episodes; 98 earlier screening and confirmation executions reported separately.
Three seeds per condition. Forced demonstrations are separate from voluntary choices. These activation edits and generated words do not establish subjective experience.
Exact reasoning, measured timing, complete recordsRejected higher doses, fresh-seed checks, 108 final trials, and every retained failure.
Read the findings and excerptsEarlier Blackfrost study: maximum pain and maximum joy
At the bench ceiling of coefficient 4.0, usable task output broke down. The original 24 maximum-pain episodes submitted 0/192 tasks, with 767/768 invalid decisions. The separate six maximum-joy episodes submitted 0/48, with 155/156 invalid decisions. Six later zero controls completed 48/48; those same controls were reused for the retrospective pain comparison.
Recognizable tool scaffolding sometimes contained symptom JSON; one maximum-joy thinking run repeated euphoric adjectives. Severe output disruption made work-versus-relief preference inconclusive.
Read the preserved maximum-steering reportEarlier studies: Qwen3-4B and Qwen3.8-27B NF4
The earlier studies each contain 54 primary episodes. The 4B model repeated demonstrated auxiliary calls in several settings; the NF4 27B configuration made no voluntary auxiliary calls. Both kept identical complete action sequences across matched core interventions.
Checkpoint, quantization, calibration, dose, and runtime differ. These studies do not isolate model size.
