The Impact of Inference: Reliability
Traditional reliability meant consistency. Given identical inputs, systems produced identical outputs. Costs were stable and behavior predictable. Inference reliability on the other hand is shaped by nondeterminism. Outputs vary due to stochastic generation, retraining introduces drift, and token-based billing can cause cost fluctuations. The new dimension of reliability is semantic consistency, that is, the ability to deliver outputs of acceptable quality, accuracy, and predictability over time despite probabilistic behavior.
In this episode of Pop Goes the Stack, F5's Lori MacVittie and Joel Moses are joined by guests Ken Arora and Kunal Anand as they dive into the topic of reliability in AI systems. They explore the concept of 'slop' (AI variability) as a potential feature rather than a bug, discuss the importance of contextual semantic consistency, and weigh guardrails and evals tailored to specific inference workloads. Tune in to learn how to navigate the evolving AI landscape and take note of practical tools and strategies like multi-model chaining, distillation, and prompt engineering to ensure reliability.
Find out more in the blog How AI inference changes application delivery: https://www.f5.com/company/blog/how-ai-inference-changes-application-delivery
Find out more in the blog How AI inference changes application delivery: https://www.f5.com/company/blog/how-ai-inference-changes-application-delivery
Creators and Guests
Host
Joel Moses
Distinguished Engineer and VP, Strategic Engineer at F5, Joel has over 30 years of industry experience in cybersecurity and networking fields. He holds several US patents related to encryption technique.
Host
Lori MacVittie
Distinguished Engineer and Chief Evangelist at F5, Lori has more than 25 years of industry experience spanning application development, IT architecture, and network and systems' operation. She co-authored the CADD profile for ANSI NCITS 320-1998 and is a prolific author with books spanning security, cloud, and enterprise architecture.
Guest
Ken Arora
Ken Arora is a Distinguished Engineer in F5’s Office of the CTO, focusing on addressing real-world customer needs across a variety of cybersecurity solutions domains, from application to API to network. Some of the technologies Ken champions at F5 are the intelligent ingestion and analysis of data for identification and mitigation of advanced threats, the targeted use of hardware-acceleration to deliver solutions at higher efficacy and lower cost, and the design of user experiences based on intent and workflows. Ken is also a thought leader in the evolution of the zero trust mindset for security, and how that will be applied to increasingly distributed and even edge-native apps and services. Prior to F5, Mr. Arora co-founded a company that developed a solution for ASIC-accelerated pattern matching, which was then acquired by Cisco, where he was the technical architect for the Cisco ASA Product Family. In his more distant past, he was also the architect for several Intel microprocessors. His undergraduate degrees are in Astrophysics and Electrical Engineering, from Rice University.
Guest
Kunal Anand
As Chief Product Officer at F5, Kunal leads the efforts to deliver transformative solutions in application security and delivery, overseeing product vision, technology strategy, and execution. His passion for cybersecurity, data, and engineering has shaped his career, from co-founding Prevoty, an application security startup acquired by Imperva, to serving as Chief Technology Officer and Chief Information Security Officer at Imperva. These experiences, along with leadership roles at organizations like NASA’s Jet Propulsion Lab and BBC Worldwide, have prepared him to tackle the evolving challenges of modern technology.
Producer
Tabitha R.R. Powell
Technical Thought Leadership Evangelist producing content that makes complex ideas clear and engaging.
