https://video.ut0pia.org/videos/watch/ffa29d18-0bf9-4b06-8ccd-20b8a0d43f8e The human-in-the-loop paradigm promises automation's efficiency without sacrificing safety or nuance. But it hides an underexamined assumption: that human involvement produces genuine discernment, not just a rubber stamp. In practice, human-AI interaction often occurs in environments where throughput or business incentives crowd out critical thinking. This talk examines why human oversight so often falls short in practice, and how deliberate interaction design can close the gap. We've grown comfortable delegating reasoning to machines. We follow GPS down unfamiliar streets and accept AI coding suggestions with minimal inspection. When Shaw & Nave (Wharton, 2026) studied human-AI interaction, they found people accepted AI answers over 80% of the time, even when it was wrong. They call this cognitive surrender: when humans forgo deliberation and adopt AI output with minimal scrutiny. At the Duolingo English Test, a controlled experiment revealed that experienced proctors shown fabricated AI cheating alerts confirmed cheating at near-chance rates. But coin-flip accuracy is unacceptable when college admissions and visas are on the line. The model wasn't the problem, the signals were fake. But skilled reviewers were still showing systematic confirmation bias. A single change to decision framing improved accurate rejections by 21%. The fix isn't better models or more human oversight. It's engineering the interaction itself. What reasoning patterns do you need from the human, and how does the interface elicit them? This talk covers practical design principles for building AI systems that improve human judgment, produce more reliable review behavior, and generate higher-quality training data. Every AI system trains its users, the question is whether you're doing it deliberately. Speakers: Angel Ortmann Lee (Duolingo): Angel is a Software Engineer at Duolingo, building AI security systems for the Duolingo English Test to help ensure online testing is secure, trustworthy, and accessible for learners around the world. LinkedIn: https://www.linkedin.com/in/angel-ortmann-lee-7494b9201 Wed, 08 Jul 2026 12:03:42 GMT https://validator.w3.org/feed/docs/rss2.html PeerTube - https://video.ut0pia.org https://video.ut0pia.org/lazy-static/avatars/0287a09a-aae7-4840-9843-b416426e7046.webp https://video.ut0pia.org/videos/watch/ffa29d18-0bf9-4b06-8ccd-20b8a0d43f8e All rights reserved, unless otherwise specified in the terms specified at https://video.ut0pia.org/about and potential licenses granted by each content's rightholder.