Unique Research · Insight / interview digest · October 31, 2023 · a16z with Anjney Midha · Original: a16z.com/improving-ai/
Context: Anthropic co-founder & CEO Dario Amodei (physics → neuroscience → Baidu speech with Andrew Ng → Google Brain → early OpenAI → Anthropic / Claude 2). Anthropic: responsible AGI/LLM lab; Google reportedly $500M upfront + $1.5B commitment. Claude as Constitutional AI chatbot; Claude 2 safety-trained. Talk covers scaling, talent density, Constitutional AI, and safety checkpoints.
Dario’s background
Anj recalls a founding call: Dario said “$500” — Anj offered $500k — Dario meant $500M. Path: physics undergrad wanting to understand the universe; Moore’s Law + Kurzweil sparked AI interest when SVMs felt unexciting → neuroscience grad school as closest to intelligence → AlexNet era → Baidu speech → Google Brain year → ~5 years OpenAI → Anthropic.
GPT-2 emergent abilities
Confidence crystalized on 2019 GPT-2: five EN–FR examples in-context, sixth English sentence actually “translated” (badly — worse than dictionary word-swap) — yet unbounded next-token objective looked endlessly scalable. Skeptics saw a bad translation bot; the team saw the beginning. GPT-3’s leap: much larger scale; simple Python coding that traditional wisdom said models “couldn’t reason” — even memorized patterns placed in novel contexts. Signal: almost no special work — scrape the web; only ~0.1–1% Python yet strong results → more compute + more curated code data would amplify dramatically. Only a matter of time.
Scaling laws
Three levers: data, compute, algorithms. Even without further algo gains, scaling current setups should continue — progress people (including Dario) tend to underestimate. Biggest factor: more capital. Next year: multiple companies with ~$1B-class models; by 2025 multi-billion to ~$10B models. 100× cadence plus H100s and lower precision → huge capability jumps.
On architecture efficiency: inference shouldn’t get that expensive. Scaling-law logic: n× compute → ~√n data and ~√n model size — models don’t grow as fast as hardware speeds. Usable 3–4 years without architectural innovation (a bit costlier); with innovation (expected), cheaper.
Physicists in AI
Two kinds of fields: mature (biology — years to contribute) vs. young/fast (AI). Gifted generalists often beat long-tenure specialists; prior domain knowledge can even hurt. 4 of 7 Anthropic co-founders have physics backgrounds — a talent pool of sharp people without necessarily ML tenure. It worked; they hire accordingly. Talent density forever beats headcount. Commercial reality still needs people per customer/feature — tension as headcount passes 100 FTE; leadership often fears growing too fast vs. standards; so far they’ve held the line.
Constitutional AI
Until recently, steering values/outputs was mostly RLHF (Dario a co-inventor at OpenAI; later refined for ChatGPT) — humans prefer outputs; model imitates. Constitutional AI: the AI gives feedback against a short written constitution (~5 pages, updated) instead of human preference labels. Train: AI reads constitution + task; another AI judges compliance; hope the model internalizes principles. Seed values: minimize founders’ own additions — UDHR, widely shared deliberative principles, some Apple ToS-like norms (“child-appropriate content,” “don’t violate basic human rights”). Direction: different use cases need different operating principles (therapist ≠ lawyer) — simple core, then branches, not one mono-constitution; exploring democratic deliberation so people help design constitutions.
Safety vs. scale — apparent contradiction
Anthropic both scales hard and urges caution. Best safety solutions almost always involve AI itself. Separating safety theory from capability development has been less successful; success areas use AI to secure AI — including interpretability (strong models reading weaker neurons; insights that also improve efficiency). Framework emerging: regulation + self-compliance; safe scaling / checkpoints — capability jump, then a gate proving safety properties before the next level. Cost of gates: avoid needless bureaucracy (1,000-page filings, 15 licenses) that slows democracies while authoritarian rivals race ahead. Airplane/auto safety as relative positive examples balancing progress vs. harm.
Anthropic roadmap
Busy summer: 100k context window, Claude 2, ecosystem for builders. Under-appreciated: longer context + retrieval unlocks talking to large databases — upload a contract (“5 most unusual clauses?”) or financials (“summarize; what’s surprising vs. this analyst two weeks ago?”) — knowledge ops that take humans hours. Infinite context? Never truly infinite — compute shifts into the window and gets too expensive — but interest in extending windows and other large-data interaction methods continues.
1 CN image; attach on ingest. Dialogue facts/Q&A preserved.
— END —