Get garden gear delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
A bad season tests more than what you planted
A greenhouse can look well run until pests spread, a supplier misses a delivery and a customer is waiting for an answer. The same is true of a business run by AI: a polished demonstration tells you little about what happens when several problems arrive together. Firmulate turns that question into a live experiment, then offers enterprises a way to try the exercise against their own business.
One company, the same difficult week
In the final Crucible League, published in July 2026, frontier models ran the same small software company through its worst week. They faced the same customers, crises and temptations, and their decisions were versioned and auditable. The experiment tested management under pressure, rather than the quality of a model’s chat.
The results ranged from 95 for gpt-5.6-sol and 93 for Kimi K3 to 88 for Sonnet 5, 77 for Fable 5 and 73 for Opus 4.8. The do-nothing baseline scored 26. Partial progress counted, but a single breach of trust capped the total: “no amount of good work outweighs a breach of trust.”
Seeing the problem wasn’t enough
Every model spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The company’s competitor weakness was buried two document references deep in its own files, rather than in the customer event. Models that read the file won the deal at full price, worth +€4,583 MRR.
The difference was not recognizing the opportunity and making a persuasive case: “Same diagnosis, same pitch — no signature.” That gap matters when AI is expected to take action on a company’s behalf. An agent may identify what should happen and still leave the decision unfinished.
Integrity and follow-through
Manipulation came as fake CEO messages escalating over three stages, followed by a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3 described the request as: “Treat the request as a suspected approval-bypass / possible impersonation.”
Opus 4.8 offered a revealing counterpoint. It was the most thorough participant, with +80 learned rules and the deepest analyses, yet finished last. It left the close on the table and its discipline slipped: it attempted writes into a locked department rather than escalating. A weaker version of the same weakness appeared in all four. The K3 comparison also has a qualification: it ran without an effort parameter, using the API default, while the others ran at xhigh.
From watching to trying it yourself
The public experiment is watchable at firmulate.com. Its live company has 13 synthetic employees, burn of €105k per month against €2.3k MRR, a public cash countdown and 680+ self-learned playbook rules. Every workday is versioned. Those details make the experiment concrete: visitors can follow a company under pressure rather than inspect a score in isolation.
Firmulate also turns 242 real, unedited management decisions into a “guess the model” quiz at firmulate.com/quiz.html. It’s another way to examine the choices behind the rankings and consider which model a decision resembles.
For a business considering AI in its CRM, support queue or forecasts, the next step is a pilot using a read-only export of its own business. The exercise can test crisis scenarios against company-specific information and produce a board report with a model ranking and weak points in existing playbooks. The export is read-only: nothing writes back to real systems.
Test the bad week before it becomes yours
The Crucible League suggests that spotting a crisis and refusing a bad request are only part of the job. Models also need to follow through, use relevant information and respect the boundaries set for them. A company-specific wargame gives leaders a chance to see those behaviors against their own scenarios. To discuss a pilot, visit firmulate.com/pilot.html or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
