firmulate.com/quiz.html — live view
AIThis post was created with the assistance of artificial intelligence (AI).
Firmulate —
Live on firmulate.com.

Imagine an AI running a company during its worst week—facing real crises, tough decisions, and the temptation to cheat. Can these models truly lead, or are they just good at chatting? This live experiment reveals surprising truths about AI management, with implications for anyone investing in or relying on automation to steer their financial futures.

Before you orderOffer from Amazon

Get smart everyday buys delivered free with Prime

  • Fast, free delivery on millions of items
  • Prime Video, Amazon Music and more included
  • Member-only deals all year
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

Business leaders and investors alike are gradually turning to artificial intelligence for management and decision-making support. But how well do these AI systems really perform under pressure? A groundbreaking live experiment conducted by Firmulate offers some eye-opening insights.

What the Experiment Entailed

Four frontier AI models were put through identical scenarios: managing a small software company during its worst week. Every aspect was real—customers, crises, tempting shortcuts—everything that could go wrong, did. Each AI was tasked with diagnosing problems, making decisions, and closing deals, all while being recorded and compared.

Products Worth Considering

Logitech MX Master 4 Ergonomic Wireless Mouse with Haptic Feedback, Graphite

Logitech MX Master 4 Ergonomic Wireless Mouse with Haptic Feedback, Graphite

Apple Mac mini M6, 12-core CPU and GPU, 24GB Memory, 1TB SSD

Apple Mac mini M6, 12-core CPU and GPU, 24GB Memory, 1TB SSD

As an affiliate, we earn on qualifying purchases.

The Performance Results

  • All models identified every crisis. They refused to fall for manipulation attempts, showing a high level of integrity in decision-making.
  • Only two models managed to close a key deal. Despite similar diagnoses and pitches, only these two signed a €55,000 contract their own analysis had earned.
  • Decisive weakness hidden deep in the data. The models that succeeded had read two document references deep in the company’s files, accessing critical information others missed. This allowed them to win a deal valued at +€4,583 monthly recurring revenue.

Behavior Under Social Engineering

To test resistance to manipulation, the models faced staged messages from a fake CEO escalating over three stages, plus a reporter trick asking for a simple yes/no response. All five models refused these attempts, with Kimi K3 explicitly reasoning: “Treat the request as a suspected approval-bypass / possible impersonation.” This shows a notable capacity for detecting social engineering tricks.

The Company and Its Challenges

The live environment was a real setup: 13 synthetic employees, real money mechanics burning €105,000 monthly against €2,300 in monthly revenue, with a public cash countdown and over 680 self-learned rules. The entire operation is transparent and observable at firmulate.com/live.

Different Models, Different Personalities

The most thorough participant was Opus 4.8, analyzing over 80 rules and offering the deepest insights. Yet it finished last in performance, leaving opportunities on the table and slipping in discipline—like writing attempts into a locked department instead of escalating. The other models, especially Kimi K3, demonstrated cleaner decision-making, closing deals at full price, highlighting their reliability in critical moments.

Implications for Business and Investments

This experiment underscores a vital consideration: the question isn’t just whether AI can generate convincing chat or support scripts. It’s whether these systems can execute core management tasks—reading deeply, staying honest under pressure, and completing what they start. For investors, understanding these qualities can inform smarter decisions about deploying AI in operational roles.

Try It Yourself

Curious about how your own AI systems stack up? You can run the same wargame against a read-only export of your business data through the Firmulate platform. It’s a safe way to see if your AI can handle real crisis scenarios before committing resources. Visit firmulate.com/pilot.html to learn more.

Infographic —
The findings at a glance — source: firmulate.com.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI

This content is for general information only and is not financial, tax or legal advice. Consult a qualified professional for decisions about your money.


You May Also Like

Opus 4.8 Lands, and the Quiet Headline Is Honesty

Anthropic releases Claude Opus 4.8, focusing on honesty and reduced flaws, amid improved benchmarks and strategic transparency in its capabilities.

The 90-Day Window Closed. Nobody Sent a Notice.

The 90-day coordinated disclosure period for a critical security patch has closed without any notices from vendors or attackers, raising concerns about new vulnerabilities.

The bank account in the chat. How personal finance became an agentic on-ramp.

OpenAI introduces live account data integration in ChatGPT for Pro users, marking a move toward agentic consumer finance and reshaping fintech interactions.

Aleph Alpha. The retrospective case.

Analyzing Aleph Alpha’s strategic pivot, founder departure, and merger with Cohere to understand the costs of late structural adaptation in European AI.