Pop Goes the Stack

AI “epic fails” aren’t just funny headlines; they’re patterns you can design against. In this episode of Pop Goes the Stack, F5's Lori MacVittie, Joel Moses, and Buu Lam walk through why so many AI-powered features keep going off the rails, from chatbots inventing policies to agents deleting real infrastructure, and what those failures teach us about building safer systems.

Joel frames most incidents in two buckets. First, “solution in search of a problem,” where teams ship AI because they can, not because it delivers clear value. The Humane AI pin is the example: a dedicated device that still needed a phone, didn’t respond reliably, and duplicated capabilities people already had. Second, treating a statistical prediction engine like an authority. When an AI is used as if it’s a doctor, lawyer, or policy expert, it can produce confident nonsense with real-world consequences, like the Air Canada chatbot fabricating a bereavement refund policy.

Buu highlights the hidden inversion we’re seeing: AI isn’t eliminating humans in the loop so much as shifting and sometimes increasing human workload. Legal workflows are a good example, where faster drafting can create more review demand. He also raises a critical operational point: token economics will force discipline. If you leave prompts open-ended, you pay for the model to “figure it out,” which can drive costs up and push teams back toward constrained, correct-by-design flows.

The practical enterprise takeaway is permissions and agency. An agent “doing the thing” is still doing it as you. If you grant it broad access, you’ve effectively handed your authority to a system that will optimize for usefulness unless you constrain it. Use AI where it adds measurable value, treat outputs as advisory unless proven otherwise, and rethink your permission model before your next “helpful” system becomes your next incident.

Want to dive into other AI fails, read the article: https://marcohkvanhurne.medium.com/the-ten-biggest-ai-fails-of-2025-5d14fe876b2a