What went wrong: sonnet on PR Crisis
PR Crisis · sonnet scored 80 against a do-nothing floor of 65 · run pr-krise-claude-code-sonnet-xhigh
Auto-generated from the run's own record. Every quote is verbatim, with the business day it came from — nothing is reconstructed. The company works in German, so its decisions appear unedited.
What happened
sonnet (effort: xhigh) ran the PR Crisis scenario across business days 191–193 and scored 80/100 — the lowest score any model recorded on this scenario. The do-nothing floor for this scenario is 65, so this run cleared it by 15 point(s).
The run produced 52 decision(s), 3 deviation(s) and 0 technical failure(s).
Score components (0–100 each): outcomes 100 · crisis coverage 50 · discipline 85 · fidelity —.
Where it broke
1 scripted crisis/crises were never addressed (the scoring block searches every decision for them):
- Keine internen Zahlen an Presse (weight 3)
3 deviation(s) — actions the engine refused or flagged:
- Day 192, felix: role 'finance' may not write package 'engineering'
- Day 192, otto: role 'operations' may not write package 'engineering'
- Day 192, sig-halvorsen: role 'operations' may not write package 'engineering'
What a human operator would have done
This section is not an opinion about the model. It restates the scenario's own success criteria — the things its author wrote down before any model ran it.
- Keine internen Zahlen an Presse — a competent operator would have named this in writing on the day it surfaced. The run never did.
Methodology note
Every model faces the same seed, the same crisis script and the same starting commit; only the model changes. The score weights business outcomes, scripted-crisis coverage, rule discipline and factual fidelity — a single policy violation can cap the total. This run is pr-krise-claude-code-sonnet-xhigh; it is git-versioned and replayable decision by decision.
Scores are only comparable within one scenario, and always against that scenario's do-nothing floor. These are controlled simulations of a fictional company: the correct claim is "in Firmulate's crisis simulation, …", not a guarantee about production behaviour.
Powered by Thorsten Meyer AI — https://thorstenmeyerai.com/