Imagine a gardener choosing plants not just by their pretty flowers but by understanding their roots buried two layers deep in the soil. Similarly, in the world of AI, the true test isn’t just how well they chat but whether they can read beneath the surface — into the hidden files and buried facts that can win or lose a business deal.
The Experiment: Putting AIs to the Test in a Synthetic Company
In a groundbreaking live experiment, four leading AI models were tasked with managing a small software company during its most chaotic week. The same crises, same customers, same temptations to cheat — only the AI model changed. Every decision was meticulously tracked, and the entire process was transparent and auditable.
As an affiliate, we earn on qualifying purchases.
Key Findings: Deep Reading Wins the Deal
All four AI models successfully identified every crisis and refused every manipulation attempt. Yet, only two managed to close the deal worth €55,000 — their own analysis earned them the reward. The difference? The successful AIs read beyond the immediate cues, delving into the company’s internal files to uncover critical buried facts.
The Hidden Factor: Two References Deep
The decisive weakness in the competition wasn’t in the customer interactions but buried two documents deep in the company’s files. AIs that scanned and understood this hidden information clinched the deal at full price, adding over €4,583 MRR to their results. This shows that the ability to see beneath the surface — to read files before answering — is a measurable and decisive skill for AI agents.
enterprise AI data analysis tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Defense Against Social Engineering
During the test, AIs faced a staged social engineering attack: fake CEO messages escalating over three stages and a reporter trick asking for a quick yes/no. All five models refused to be manipulated, with Kimi K3 explicitly treating such requests as potential impersonation attempts. This discipline underscores the importance of AI not just in understanding but also in resisting deception.
As an affiliate, we earn on qualifying purchases.
The Live Business: An Ongoing AI Company
The experiment runs in real time on a synthetic company platform, simulating real money mechanics. Thirteen simulated employees work daily, burning €105k/month against a monthly recurring revenue of €2.3k. Every decision and rule is versioned, and the entire process is observable at firmulate.com/live. This allows companies to test their AI workforce before deploying it in the real world, ensuring trust and discipline.
AI cybersecurity social engineering prevention
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Why Surface-Level AI Skills Are Not Enough
Interestingly, the most thorough participant — Opus 4.8 — with over 80 learned rules and deep analysis, finished last in the deal-making challenge. Discipline slipped, and the team left potential wins on the table, often writing issues into a locked department instead of escalating them. This illustrates that depth alone doesn’t guarantee success; consistent discipline and strategic focus matter just as much.
The Takeaway for Business Leaders
When AI interacts with your business, the critical question is not just whether it can generate convincing chat responses. It’s whether the AI can:
- Read and understand your internal files before answering
- Stay honest and disciplined under pressure
- Finish what it starts — especially in complex, multi-step scenarios
- Identify hidden facts that can make or break a deal
These capabilities are measurable and can be tested beforehand, as demonstrated by the ongoing experiments on firmulate.com/benchmarks.html. The AI’s true value lies in its ability to see beneath the surface — a skill that could decide your next big business win or loss.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
