Emergence Calculus

Posts · Society

It Matters When You Defected

Two histories of a repeated game have the same score, the same counts and the same last move. In each, the next move is certain, and the two next moves differ. What predicts play is not in the game's own books.

Ioannis Tsiokos 7 min read
Two rows of three rounds each, drawn as pairs of squares, hollow for cooperate and filled for defect. Top row: both cooperate, the first player defects, both cooperate, then an arrow to the first player defecting. Bottom row: the first player defects, both cooperate, both cooperate, then an arrow to both cooperating. A bracket joins the two rows.

Two players face each other in a repeated game. Each round, each one either cooperates (C) or defects (D). In the most famous such game, the Prisoner's Dilemma, both do well if both cooperate, but each is tempted to defect and grab a little more at the other's expense. Here are two possible histories of three rounds, with player one's move written first:

CC, DC, CC      and      DC, CC, CC

In both, the players cooperated together twice and player one defected once while player two cooperated. Both end with mutual cooperation. Because the counts of each kind of round are the same, the scores are the same, whatever the payoffs. Call these the game's books: the last move, how often each kind of round has happened, and the running scores. The books for the two histories are identical. Only the order differs.

Yet in the model studied in Institutions Are Strict Extensions of the Game, the next round is certain in each case, and different. After the first history, player one defects again while player two cooperates. After the second, both cooperate. The reason is what the order reveals. In the model, each player follows a fixed rule that reacts to the previous round, and in the first history player one has already been seen answering mutual cooperation with a defection. The paper's summary: "The two histories are identical in the game's own vocabulary and different in predictive content."

The model

The paper's world is small and fully specified. Each player follows a fixed rule: an opening move, plus a response to each of the four things that could have happened in the previous round. There are 32 such rules (two possible openings, times two choices for each of the four possible previous rounds), and familiar strategies like tit-for-tat and always-defect are among them. The prediction starts by treating all 1,024 pairs of rules as equally likely, and observed play can rule some pairs out. The paper runs this with two classic games, the Prisoner's Dilemma and the Stag Hunt, and the two histories above work the same way in both, since they never needed the payoffs.

In the first history, player one has been seen to answer mutual cooperation by defecting. Since each rule responds to the previous round, and the last round was mutual cooperation again, the next move follows. In the second, both players have been seen to answer mutual cooperation by cooperating, so they cooperate again.

The institution is what predicts

What, then, is the state of play that actually predicts what happens next? The paper's answer is the smallest summary of the history that keeps the current round in view and predicts the next few rounds, the ones the paper declares, as well as the full history does. It calls this summary the institution of the game. That is a deliberately formal use of a word borrowed from the study of institutions, and the paper calls the step back to everyday institutions interpretive.

The main result is that on this model the institution cannot be computed from the game's books. From round 3 to round 6, the end of the window the paper compares, there are always pairs like the two above: histories that agree on the last move, the counts and the payoffs, and differ in what will happen next. In the paper's phrase, the institution is a strict extension of the game: it draws distinctions that the books cannot. This has teeth. The paper notes that its measure of the shortfall bounds the failure of every predictor that looks only at the books, "whether equilibrium-based, Markovian, or learned from data."

How much the books miss

The paper measures the shortfall exactly, in bits: the uncertainty about the next round that remains, on average, once you have read the books. One bit is the uncertainty of a fair coin toss, and the next round has four possible outcomes, so the most it can be is 2 bits.

The shortfall has two parts, and the paper separates them exactly. One is a residual: uncertainty about rules that play has not yet revealed, which even the full history cannot remove yet. The other is a gap: what the full history would tell you that the books do not. In round 1 the whole 2 bits is residual, "because the rules' responses to that first profile have not yet been revealed". From round 1 to round 6 the total falls through 2.00, 1.50, 1.12, 0.63, 0.38 and 0.21 bits. The residual shrinks as play reveals the rules and is gone by round 5. What is left at round 6, about a fifth of a coin toss, is all gap: information the history carries and the books miss. "On the deterministic carrier, then, not all of the closure deficit is institutional: the institution recovers the gap, and the residual reflects rules that play has not yet revealed."

Six bars for rounds 1 to 6 showing the uncertainty about the next round that remains after reading the books: 2.00, 1.50, 1.12, 0.63, 0.38 and 0.21 bits. Rounds 1 and 2 are entirely pale, labelled residual, rules not yet revealed. Rounds 3 and 4 are part pale, part dark. Rounds 5 and 6 are entirely dark, labelled gap, in the history, not in the books.
The books' shortfall about the next round, in bits, at each of the first six rounds (the paper's exact values). The pale part is what nobody can know yet, because play has not revealed the rules. The dark part is what the full history already tells you and the books leave out. From round 5 on, only the dark part remains.

Forcing and probing

The paper also asks what happens when someone steps in and imposes moves. Two kinds of intervention behave very differently.

Suppose an outside party takes over. Force one player's moves for a few rounds and let the other respond freely. Two such probes can start from the same history and end with the same visible record, last move and counts included, while leaving the game in different predictive states, because the free responses revealed something about how the second player behaves. On the paper's grid of interventions there are 512 such pairs.

Force both players' moves, and nothing is revealed. The paper's grid has 78,528 pairs of fully forced protocols ending on the same forced move. Of those, 3,456 also end with the same visible record, and not one of them leaves different predictive states. That may sound obvious once said, and the paper's explanation is exactly that: "Pure forcing reveals no rule choice." The point is the contrast with probing.

On this carrier, institutional memory is written by observed free response, not by force alone.

The probing result is fragile, and the paper says so. Add a little random noise to the moves themselves, each player's move flipped with probability 1/20 or 1/8, and the one-sided witnesses become rare, 2 among 600 tested pairs at each noise level, while the forcing result survives exactly.

What people actually do

The paper then turns to real people, using a published laboratory dataset on the indefinitely repeated Prisoner's Dilemma by Dal Bó and Fréchette (2019). Before receiving the data, the authors committed their hypotheses, methods and pass or fail thresholds.

The first prediction held. A model that sees a short window of recent history predicts the next decision better than one that sees only the current round, by 0.029 nats per decision, comfortably above the registered threshold of 0.010, over 36,972 transitions from one round to the next. People's next moves depend on more of the past than the current round.

The second prediction failed, and in the reverse direction. The authors registered that this advantage would be larger early in play than late, as if experience made players simpler. It came out the other way. The early gain was 0.018 nats per decision and the late gain 0.060. In the paper's words, "history-dependence grows with experience". It reports the miss as a registered negative result.

There is a disclosed wrinkle in the timing. Four changes to how the data were mapped were made before any model was fitted, but the first saved record containing them also contains the results, so the paper marks that step's timing as unverified. It also states that the human result "neither confirms nor refutes any carrier theorem": the theorems are about the model, not about people.

What it does not say

The exact results live on one finite model, with its 32 rules, two games, a uniform starting belief, and declared horizons. They come from complete computer enumeration in exact arithmetic, checked by separate validators and independent re-implementations, not from a proof assistant. The paper's highest claim is "theorem-grade on the carrier, not a general theorem about repeated games."

Nor does it claim to explain real institutions. "The step from the canonical predictive state to the everyday language of institutions remains interpretive, disciplined by the formal and empirical checks but not reduced to them." What it shows, exactly, is that what a game's own books record and what predicts the game can come apart, and that the difference can be measured. It is not only how often you defected that counts, but what you were answering when you did.

Drawn from Institutions Are Strict Extensions of the Game (Tsiokos, 2026, version 2, doi:10.5281/zenodo.23118272). The histories, counts, numbers and quotations are from the paper; the laboratory data are Dal Bó and Fréchette (2019), as cited there. No new computation is claimed by this essay.