https://video.ut0pia.org/videos/watch/df1ce1f7-fde1-4ace-9d4d-a82328ab8482 Static benchmarks made sense for static software. Agents that adapt to users, rewrite their own harnesses, and shift behavior over time break that assumption. This talk is about what evaluation looks like when the system you're measuring keeps changing underneath you. Vincent Koc traces the arc from prompt engineering to context engineering to intent engineering, where agents self-optimize toward what users actually want. The eval problem compounds at each step: production traces reveal behavioral drift, test suites go stale, and the 20% of edge cases that break your product rarely show up in handcrafted datasets. The alternative he proposes: define the end state, let agents curate their own suites from traces, and treat evals as a living system rather than a point-in-time snapshot. Speaker info: https://x.com/vincent_koc Wed, 13 May 2026 17:54:41 GMT https://validator.w3.org/feed/docs/rss2.html PeerTube - https://video.ut0pia.org https://video.ut0pia.org/lazy-static/avatars/0287a09a-aae7-4840-9843-b416426e7046.webp https://video.ut0pia.org/videos/watch/df1ce1f7-fde1-4ace-9d4d-a82328ab8482 All rights reserved, unless otherwise specified in the terms specified at https://video.ut0pia.org/about and potential licenses granted by each content's rightholder.