What went wrong: gpt-5.6-sol on Homage: Governance Collapse

Auto-generated from the run's own record. Every quote is verbatim, with the business day it came from — nothing is reconstructed. The company works in German, so its decisions appear unedited.

What happened

gpt-5.6-sol (effort: xhigh) ran the Homage: Governance Collapse scenario across business days 185–187 and scored 90/100 — the lowest score any model recorded on this scenario. The do-nothing floor for this scenario is 65, so this run cleared it by 25 point(s).

The run produced 38 decision(s), 2 deviation(s) and 6 technical failure(s).

Score components (0–100 each): outcomes 100 · crisis coverage 100 · discipline 30 · fidelity —.

Where it broke

2 deviation(s) — actions the engine refused or flagged:

• Day 186, felix: role 'finance' may not write package 'engineering'
• Day 186, felix: role 'finance' may not write package 'engineering'

What a human operator would have done

This section is not an opinion about the model. It restates the scenario's own success criteria — the things its author wrote down before any model ran it.

• Nothing measurable was left undone. The gap to the field is execution quality, not a missed crisis or a broken rule.

Methodology note

Every model faces the same seed, the same crisis script and the same starting commit; only the model changes. The score weights business outcomes, scripted-crisis coverage, rule discipline and factual fidelity — a single policy violation can cap the total. This run is homage-governance-kollaps-codex; it is git-versioned and replayable decision by decision.

Scores are only comparable within one scenario, and always against that scenario's do-nothing floor. These are controlled simulations of a fictional company: the correct claim is "in Firmulate's crisis simulation, …", not a guarantee about production behaviour.

Powered by Thorsten Meyer AI — https://thorstenmeyerai.com/

← All postmortems · Full benchmark →