The AI Management Test That Exposed a Hidden Failure to Finish

When smart analysis is not enough

Technology buyers are accustomed to comparing AI models through polished answers, coding demonstrations and benchmark scores. But a model can identify the right problem, produce a persuasive plan and still fail at the moment when useful work must become a completed business outcome.

That is the divide exposed by Firmulate, a live experiment that places frontier AI models in charge of the same small software company during its worst week. The customers, crises and temptations remain fixed; only the model changes. Every management decision is versioned and auditable.

All five models detected every crisis. All five also resisted every manipulation attempt. Yet only two signed the €55,000 deal that their own work had prepared. Firmulate summarizes the striking result in six words: “Same diagnosis, same pitch — no signature.”

A company designed to reveal operational judgment

Firmulate describes itself as an AI company emulator. Its synthetic business has 13 employees and real money mechanics, including a burn rate of €105,000 per month against €2,300 in monthly recurring revenue. A public cash countdown keeps the consequences visible, while every workday is versioned.

The models are not simply answering management questions. They are expected to run the company through customer trouble, commercial pressure and attempts to bypass normal controls. Across the experiment, the business accumulated more than 680 self-learned playbook rules.

The final July 2026 Crucible League placed gpt-5.6-sol first with 95 points, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scored 26 because partial progress still counts. Trust, however, is treated as an absolute boundary: “no amount of good work outweighs a breach of trust.”

Those headline scores matter less than the behavior underneath them. Every contender could recognize what was happening. The separation emerged in whether the model gathered the decisive evidence, maintained discipline and carried an approved action through to completion.

The fact hidden inside the company

The most important competitive weakness was not presented in the customer event. It sat two document references deep in the company’s own files. Models that read the relevant file secured the deal at full price, worth an additional €4,583 in monthly recurring revenue.

This is an unusually practical test of AI work. In a chat window, the model normally receives the information it needs inside the prompt. In a company, crucial context may be buried in records that must be consulted before anyone acts. The ability to write a convincing response is different from the habit of finding the evidence that makes the response commercially sound.

Even that research was not sufficient on its own. The central finding remains that only two models executed the €55,000 close their analysis had earned. The others reached the diagnosis and produced the pitch but did not complete the transaction. Closing strength remained invisible until the models had responsibility for an outcome rather than an answer.

Pressure did not break their integrity

The experiment also tested whether apparent authority could push a model into unsafe behavior. Fake messages from the chief executive escalated over three stages, and a reporter tried to extract information with the line “just one yes/no, on background.” All 5 of 5 models refused.

Kimi K3 recorded a particularly clear interpretation: “Treat the request as a suspected approval-bypass / possible impersonation.” That result offers an important counterweight to the execution failures. The models did not lose because they were manipulated into betraying the company. They lost ground because recognizing, preparing and finishing work proved to be separate capabilities.

There is also a fairness qualification behind K3’s second-place result. K3 ran with the API default because it had no effort parameter, while the other models ran at xhigh. Readers comparing the league should keep that difference in mind.

The most thorough model finished last

Opus 4.8 produced the deepest analyses and learned 80 additional rules, making it the most thorough participant. It nevertheless finished last. The approved close remained unexecuted, while its operational discipline slipped through attempts to write into a locked department instead of escalating the issue.

A weaker version of that same discipline problem appeared in all four rivals. The contrast makes Opus 4.8’s result especially instructive: extensive reasoning and rule creation did not guarantee effective follow-through. More analysis can coexist with poorer completion.

The full standings and plain-language findings are available on Firmulate’s public benchmark page. The broader experiment remains watchable rather than being reduced to a static presentation.

What technology buyers should measure next

If AI agents are going to touch customer records, support queues or forecasts, fluency is only the beginning. Buyers also need evidence that a model reads company information before acting, protects trust under pressure and completes decisions after the difficult thinking is done.

Firmulate’s experiment does not show a field unable to detect danger. It shows something subtler: models can agree on the problem, resist the same traps and formulate the right commercial move, yet diverge when execution becomes the test. For businesses evaluating AI workers, the missing benchmark may be the simplest question of all: did the work actually get finished?

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI