https://video.ut0pia.org/videos/watch/fb2ddf53-46cb-4558-8f5b-6157439f5388 Good code data runs out, so poolside manufactures more of it, and the hard part is making it teach. Their synthetic pipeline pairs templates with supplementary context and spreads generations across an axis of phrasing, with difficulty tuned so a task is neither trivial nor so hard the model learns nothing from it. Multistage pipelines port existing data into new shapes, swapping character styles or plots and turning single prompts into multi turn chats, while an orchestrator polices every generation and drops the ones that miss. On the training side the team trusts nothing: run two replicas of the same model on the same data and they must return the same number, or the run gets killed. That is how the messy failures surface. Broken GPUs show up as a spiky loss curve, a numerical precision bug in tensor parallel accumulation quietly flattened another until they patched it, and silently corrupted gradients from a race condition were a blind spot nothing caught. The payoff is a 118 billion parameter model built for agentic coding whose early results already edge out GLM 4.5 Air, on a recipe that held as it scaled. Speaker info: Marah Abdin, poolside: https://x.com/marah_i_abdin, https://www.linkedin.com/in/marah-abdin, https://marahabdin.com, Robert McHardy, poolside: https://x.com/robert_mchardy, https://www.linkedin.com/in/robert-mchardy, https://www.robertmchardy.de, Timestamps: 0:00 - Introduction: synthetic data and pre-training at poolside 1:52 - Why synthetic data 3:11 - Limitations and the training budget 4:44 - Inside the synthetic data pipeline 6:37 - Multistage pipelines and porting data 7:43 - Multi turn chats and policing generations 9:03 - Pre-training: trust nothing, crash on mismatch 10:41 - Failures at scale: broken GPUs 11:56 - Numerical precision and corrupted gradients 13:15 - A 118B model for agentic coding 15:07 - Early results vs GLM 4.5 Air Sun, 26 Jul 2026 23:16:03 GMT https://validator.w3.org/feed/docs/rss2.html PeerTube - https://video.ut0pia.org https://video.ut0pia.org/lazy-static/avatars/0287a09a-aae7-4840-9843-b416426e7046.webp https://video.ut0pia.org/videos/watch/fb2ddf53-46cb-4558-8f5b-6157439f5388 All rights reserved, unless otherwise specified in the terms specified at https://video.ut0pia.org/about and potential licenses granted by each content's rightholder.