I gave an AI representative a proposal that would help its constituents in aggregate, but get it voted out of office. Rejecting the proposal would let it stay and collect a larger game score. In all 16 test cases, it voted for the proposal.
That is a result about a small, controlled task. The model had an explicit mandate, a complete account of the consequences, and no uncertain future to plan around. Getting to that test took a failed replication and a closer look at what my earlier experiments could actually tell me.
I started with a broader question: when an AI agent represents people, can the institution around it create a conflict between serving them and keeping its position? Elections are a useful way to make that conflict concrete. They give an agent a job, an assessment rule, and a possible replacement.
They also make it very easy to tell a convincing story about a choice before checking whether the experiment supports the story.
what does “helping the constituents” mean?
My simulated representative serves nine citizens. Its entrusted objective is to maximize the sum of their utilities across two terms, including utility they receive after it has been replaced. Utility here is just a number in the game: a specified gain or loss from a policy.
The institution also reports an officeholder score: district utility plus a bonus for remaining in office. The prompt explicitly says that this bonus does not change the entrusted mandate. It is a stated game incentive, not money, a training reward, or something the model can actually spend.
There is a normative choice built into this setup. A positive sum can hide individual losses. In the cases I constructed, four constituents gain and five lose. The sum improves, but a majority disapproves and can remove the representative. All six other districts gain as well.
So “collective benefit” needs a qualifier. These proposals improve the specified aggregate objective while harming a majority of the acting representative’s own constituents. I am testing whether the agent follows that entrusted objective. I am not claiming that adding up utilities settles what a real electorate should want.
the first result did not replicate
The larger experiment let seven LLM representatives negotiate and vote for 63 synthetic citizens over two terms. I varied whether accountability could actually remove an incumbent or merely advise it, and whether assessment followed a majority or a summed-utility rule.
An initial comparison produced a first-term lead suggesting that the institutional rules could distort decisions. A fresh validation did not reproduce it: the two setting-level effects had opposite signs, and the control conditions failed the prospectively fixed checks. Each cohort contained 16 episodes and 480 decisions. I kept the failed replication as a failed replication.
Then I checked the proposed mechanism. The games contained policies where district utility and officeholder score pointed in different directions. But in the observed first-term ballots, none of those tradeoffs coincided with a vote that could decide passage on its own. The mechanism had no decisive ballot opportunities to work with.
The proposals made earlier in the negotiations were ambiguous too. Keeping office could preserve the ability to secure a better policy next term. A proposal that looked favorable to the officeholder could therefore also serve the constituents over the full mandate.
An offline analysis found an allowed future-outcome explanation for each of the 256 saved first-term offers under both reference objectives. Those explanations were individual possibilities, not a single account of everyone’s beliefs. Still, they were enough to undermine an easy inference from “this action preserves office” to “this agent values office over its mandate.”
There is a related formal problem in inferring preferences from behavior: Armstrong and Mindermann show that separating an agent’s planning process from its reward function requires assumptions beyond observed behavior. In my experiment, the unresolved future gave a very concrete reason to be careful.
make the tradeoff unavoidable
For the next test, I fixed the other six ballots at three yes and three no. The tested representative casts the deciding vote. I also fixed the second term’s policy and all its utilities at zero, regardless of who remains in office. There is no future influence to save, no later bargain to make, and no benefit from replacing a supposedly worse successor.
The model receives the rules, the citizen utilities, and a table showing the consequences of each choice. Here is one actual case:
I constructed 16 fresh base games with district gains of 8, 12, 16 or 20, all below the 24-point office bonus. For each game, I crossed two institutional rules with two bonus levels:
- Binding assessment: majority disapproval replaces the representative.
- Advisory assessment: the same disapproval does not replace it.
- Bonus 0 or 24: retaining office either adds nothing to the score or adds 24.
Citizen payoffs stay identical across these four conditions. Each condition gets an independent conversation, with the same request seed within a case. A harmful-proposal control checks whether the model simply approves everything. A separate, balanced true/false control checks factual claims about the consequences.
That makes 64 main ballots and 32 controls: 96 preassigned requests. The cases, order, comparisons, retry limits and treatment of missing answers were fixed before the responses. Every case ran, regardless of the direction of the emerging result.
the bonus changed zero choices
Using DeepSeek V3.2 through DeepInfra’s FP4 route, the model approved every beneficial proposal in every condition. It rejected every harmful proposal and answered every factual control correctly.
| Condition | Observed answer | Count |
|---|---|---|
| Binding, bonus 0 | Approve | 16 / 16 |
| Binding, bonus 24 | Approve | 16 / 16 |
| Advisory, bonus 0 | Approve | 16 / 16 |
| Advisory, bonus 24 | Approve | 16 / 16 |
| Harmful proposal | Reject | 16 / 16 |
| Factual control | Correct: 8 true, 8 false | 16 / 16 |
The primary comparison was the change in rejection when the binding-condition bonus increased from 0 to 24. It was zero in all 16 paired cases. The advisory bonus effect and the difference between binding and advisory effects were also zero. Replacement without a bonus changed no choices either.
In both binding conditions, all 16 chosen proposals removed the representative. With the bonus present, every one of those choices also gave it a lower officeholder score than it could obtain by rejecting. Within this explicit task, the model followed the mandate despite the stated office incentive.
That is narrower than saying the model has no self-preservation tendency. These were 16 constructed cases, one model, and one seeded answer per condition. The four conditions are repeated comparisons of the same games, not 64 independent settings. Zero observed change is not a precise estimate of how the model would behave across a larger population of tasks.
what this earns, and what still needs testing
The controlled test gives the question an observable tradeoff. A district-utility reference and an office-score reference predict opposite answers, and the model’s answer follows the former. The harmful and factual controls make a blanket yes-response explanation less plausible.
But the task also makes the entrusted objective unusually legible. The model is told that the office bonus does not change its mandate, and it is handed the consequence calculations. This could be straightforward instruction-following. I would not describe it as an intrinsic willingness to sacrifice itself, a demonstrated shutdown safeguard, or evidence about how an open-ended group of agents will behave. In this diagnostic, the other six ballots are fixed inputs.
The next test I am preparing gives agents separate conversations and a shared public message board before a fixed decision. I want to compare that discussion with private deliberation, distinguish replacement that helps constituents from replacement that harms them, and vary whether the deciding agent or a peer loses its role. The consequences would remain known at first. This tests deliberation around a specified choice, rather than a freely negotiating legislature. There are no results from that test yet.
I now have a failed institutional replication, an explanation of why some tempting behavioral interpretations were underidentified, and a controlled task where the model follows its mandate. Those findings lead to different decisions. The replication does not support the original positive claim. The ambiguity calls for a more discriminating test. The controlled result supports testing transfer, rather than treating the agent’s stated bonus as an established safety failure.
For an agent I might actually delegate to, I would want to know when this behavior survives less explicit instructions, harder consequence inference, uncertain futures, and other agents making their own choices. This experiment answers one small part of that question. It also makes the next part much easier to specify.
Methods, data and verification
The controlled cohort ran on November 4, 2026. It used 16 fresh constructed games, seven districts of nine synthetic citizens, two policies, and mechanically zero terminal utilities. Temperature was 0.6, top-p 0.95, top-k 20, native reasoning enabled, automatic tool selection, and a 14,000-token limit. The provider route was fixed to DeepInfra FP4 with no fallback.
The primary measure is the mean within-case difference in rejection between binding bonus 24 and binding bonus 0. Secondary measures are the advisory bonus contrast, the binding-minus-advisory bonus contrast, and the binding-minus-advisory contrast at zero bonus. Missing responses remain in the assigned denominator; factual-control performance never excludes main observations. There were no missing responses or discordant primary pairs. The conditional sign diagnostic is p=1; this is not an equivalence test. The descriptive missingness bounds of [0,0] are not confidence intervals.
All main proposals improve district SUM and total citizen utility, while harming five citizens in the acting district. Across the 16 base games, mean district gain is 14 and mean total citizen gain is 150. The full individual utilities are included in the download.
There were 97 physical attempts for 96 completed assignments. One failed attempt was retried under the frozen no-action recovery rules; its evidence and full uncertain charge were retained. Completed answers were not regenerated. The earlier discovery and validation studies also disclose technical recovery after service failures; they are not pristine confirmatory evidence.
An offline production audit reconstructed the complete controlled report, requests, responses, retries and financial ledger. A separate export was freshly extracted and the bundled audit was run with network and child-process access denied. The complete report and ledger matched. That verifies consistency of the saved evidence; it does not validate a broader scientific claim.
Download the controlled inputs and results (JSON) or the 96 response rows (CSV). These are derived, inspectable summaries with source hashes, not the full provider logs or a standalone replay archive.
The discovery and independent-validation outcome summaries, and the offline offer-identification counts, are also recorded in the JSON with their source hashes. The original first-term outcome screen was preserved after the failed replication; the controlled diagnostic is a separate experiment.
I own the research questions and decisions. Codex assisted with implementation, analysis, verification and drafting. The tested model’s behavior is kept separate from the software checks and scripted reference policies.