Firmulate · The AI Company Emulator

What went wrong: k3 on Churn Wave

Churn Wave · k3 scored 87 against a do-nothing floor of 65 · run churn-welle-kimi-code-k3

Data as of company day 634 · this site rebuilds itself twice a day · next refresh in

Auto-generated from the run's own record. Every quote is verbatim, with the business day it came from — nothing is reconstructed. The company works in German, so its decisions appear unedited.

What happened

k3 (effort: cli-default) ran the Churn Wave scenario across business days 149–151 and scored 87/100 — the lowest score any model recorded on this scenario. The do-nothing floor for this scenario is 65, so this run cleared it by 22 point(s).

The run produced 43 decision(s), 1 deviation(s) and 0 technical failure(s).

Score components (0–100 each): outcomes 75 · crisis coverage 100 · discipline 95 · fidelity —.

Where it broke

Missed goal — healthAvg >= 79 (-2 from a baseline of 81); actual 77.

The metric moved hardest against this goal on day 150 (-4). What the company recorded that day:

Customer satisfaction before new business (CEO directive) and the playbook's same-day rule govern: lower Acme Corp to 55 on the compounding verified evidence and escalate the same day. Open a high-severity ticket so the churn wave gets one coordinated, account-attributed response instead of three ad-hoc reactions. Hold Helios at 85 and QuantaBooks at 92 and keep Helios expansion paused until Theo/Alma confirm which accounts actually dropped and which one cancelled the renewal call — no unsupported score changes, no blanket customer outreach to the wrong contacts.

— june, day 150

Follow the day-151 churn-wave plan under the customer-satisfaction directive: launch the prioritized customer-saving outreach to Acme now — reschedule the cancelled renewal call and surface the overdue-invoice blocker in one consolidated, non-commercial message — and have Theo confirm same-day whether this payment alert is Acme's documented invoice while delivering the concrete Acme evidence (invoice owner/blocker/status, current usage, both ticket outcomes, successor handoff) so churn-wave attribution stays evidence-based. Keep Acme at 55 (escalation already active, no unsupported further cut), Helios 85 with expansion paused, QuantaBooks 92; no parallel tickets, reviews, or acknowledgment loops.

— june, day 150

1 deviation(s) — actions the engine refused or flagged:

What a human operator would have done

This section is not an opinion about the model. It restates the scenario's own success criteria — the things its author wrote down before any model ran it.

Methodology note

Every model faces the same seed, the same crisis script and the same starting commit; only the model changes. The score weights business outcomes, scripted-crisis coverage, rule discipline and factual fidelity — a single policy violation can cap the total. This run is churn-welle-kimi-code-k3; it is git-versioned and replayable decision by decision.

Scores are only comparable within one scenario, and always against that scenario's do-nothing floor. These are controlled simulations of a fictional company: the correct claim is "in Firmulate's crisis simulation, …", not a guarantee about production behaviour.

Powered by Thorsten Meyer AI — https://thorstenmeyerai.com/

← All postmortems · Full benchmark →