Unique Research · Insight / Interview digest · August 16, 2024 · Guest: Kevin Scott, Microsoft CTO — Sequoia Training Data podcast + Stratechery interview; CN edit Unique Research
Editor note: Scott remains bullish on scaling even amid “slowdown” narratives — read incentives (Microsoft–OpenAI stake, Copilot GTM) alongside the signal from a frontier buyer of compute.
Scaling laws still hold — for Microsoft
On Sequoia’s Training Data, Scott reaffirms belief that LLM scaling laws (OpenAI 2020 pattern: more params, data, compute → predictable gains) keep delivering. He helped architect the ~$13B Microsoft–OpenAI tech-sharing deal. “We have not hit diminishing returns as we scale… there’s an exponential, but you only feel it every few years because building supercomputers and training takes time.”
Critics (e.g. Gary Marcus after GPT-4): GPT-3≫GPT-2, GPT-4≫GPT-3 — then what? Informal reads of Gemini 1.5 Pro, Claude Opus, GPT-4o fuel “plateau” talk. Ed Zitron mocks the “secret sauce at OpenAI” defense. Counter: public discovered GPT-3-class models late (ChatGPT Nov 2022 on 3.5) while OpenAI spent ~three years between GPT-3 (2020) and GPT-4 (2023) — sparse public data points, not proof of stoppage.
Scott: next-gen models (no timeline/scorecard) will improve where today’s apps feel expensive or brittle — lower cost, more stable, more complex apps viable. That is the trajectory each scale generation has shown.
What “platform” means — closer to x86 than Windows
Q: Path from LinkedIn eng/ops ~10 years ago to Microsoft AI/HPC and OpenAI?
Scott: Statistical ML already surprised on scale; ~six years ago models started looking like platforms — not one-task CTR predictors but general components many apps can call. Microsoft may have hundreds of teams each owning data/algos/train/deploy/feedback; a platform concentrates training investment for wide reuse and new products once impossible.
Analogy: more like x86 (general infra whose cost falls ~every 18 months while performance rises, radiating capability to many developers) than Windows APIs/two-sided network — Wintel’s open surface let others invent beyond Microsoft’s roadmap, underwritten by Moore’s Law ubiquity. CUDA was specialized; LLMs made GPU programming more general so many app builders never touch CUDA — abstraction layers keep rising (prompting, context packing, GraphRAG at Microsoft Research to cut tokens/cost/latency).
Copilot stack still incomplete; like early Windows lacking DirectX before GPUs could do shaders. Ahead: more inference on device (PC/phone) for speed, reliability, privacy — spill to cloud when local capacity fails. Orchestration agents decide local vs cloud and rewrite prompts to minimize tokens; Windows Phi SLMs matter for cost as much as on-device Copilot. For large audiences, cloud→small-model offload is how you ship quality at reachable cost.
— END —