https://video.ut0pia.org/videos/watch/266b290a-77dc-4d45-8ab2-3955112b723d Run the same task twice, and sometimes you get two materially different answers. While many dismiss this as the "stochastic nature of LLMs," this inconsistency is a critical product flaw that destroys customer trust—especially in high-stakes fields like cybersecurity, where a "flip-flop" between a malicious threat and a benign alert can lead to disastrous outcomes. This session explores why these flip-flops are usually not model failures. They occur in the "gray zone" near the decision boundary, where policies are ambiguous and even human experts may disagree. Instead of treating disagreement as a bug, we can use it as a signal to improve both the agent and the data. You'll learn a practical workflow that combines active learning, semantic memory (domain knowledge and business policies), and episodic memory (past similar cases) to automatically identify ambiguous examples, focus human review where it matters most, and continuously adapt the agent to customer-specific preferences, without relying solely on expensive fine-tuning. Key takeaways Find the gray zone. Use model disagreement to identify the decisions that deserve human attention., Turn inconsistency into a feature. Every flip-flop is an opportunity to clarify policies and improve the agent., Teach, don't just fine-tune. Combine semantic memory and episodic memory to make agents more consistent with far less effort than retraining., Build a continuous learning loop. Improve consistency, streamline quality control, and evolve your agent to match how your customers actually make decisions., Speakers: Diane Lin (Datadog): Dr. Diane Lin is Tech Lead at Datadog, where she leads the development of self-evolving AI agents for cybersecurity, and previously co-founded Culminate (acquired by Datadog) LinkedIn: https://www.linkedin.com/in/diane-dianhuan-lin-57210215/ Tue, 21 Jul 2026 19:37:43 GMT https://validator.w3.org/feed/docs/rss2.html PeerTube - https://video.ut0pia.org https://video.ut0pia.org/lazy-static/avatars/0287a09a-aae7-4840-9843-b416426e7046.webp https://video.ut0pia.org/videos/watch/266b290a-77dc-4d45-8ab2-3955112b723d All rights reserved, unless otherwise specified in the terms specified at https://video.ut0pia.org/about and potential licenses granted by each content's rightholder.