Unique Research · Insight / Interview Digest · September 3, 2024 · Source: TIME interview with Anthropic CEO Dario Amodei (compiled by Unique Research)
Editor’s note. Anthropic (San Francisco, 2021) was founded by seven ex-OpenAI alumni including siblings Dario and Daniela Amodei and GPT-3 lead Tom Brown. Dario (Stanford physics BS; Princeton biophysics PhD) previously led GPT-2/GPT-3 work as OpenAI research VP. Mission: reliable, interpretable, steerable frontier systems. Flagship Claude — Claude 2 first commercial 200K-context model; Claude 3 topped many evals vs GPT-4. Raised ~$1.45B (second-richest AI startup after OpenAI at CN publish); Amazon invested $4B for AWS infra. Public-benefit corp framed around helpful / harmless / honest systems. May 2024 TIME cover; TIME 100 Most Influential Companies 2024. June 20: Claude 3.5 Sonnet — claimed new bars on reasoning, coding, some math, beating GPT-4o. Amodei: goal is not incremental LLMs but systems that meaningfully collaborate with people and software. Unlike OpenAI’s path, Anthropic’s public safety commitments are unusually explicit — while most staff expect progress not to slow, with potential downside risk.
Q&A highlights
Q: Smallest frontier lab, less capital — disadvantage on safety?
Amodei: Those facts used to be true; we’ve signed large compute deals and raised billions. Rivals still sit from billions to tens of billions — some shockingly toward hundreds of billions. Calling ourselves underdogs feels awkward now, but versus the giants we still are.
Q: Scaling laws — new capabilities in unreleased systems?
We just shipped a model; there’s no year-long secret backlog — and I wouldn’t tip secrets anyway. Progress has not slowed. Real world can interrupt anytime. Every new train I watch with excitement and dread — will the curve stop? If scaling stopped, that might be good for the world (it would bind everyone). We don’t choose that; we discover which universe we live in and adapt. Models also bring exciting upside.
Q: What culture are you building?
Donation matching: employees can gift up to 25% of equity to charity with company match — attracts public-interest talent without killing economic incentive. Framing: I’d rather cast Anthropic as a public-benefit company than a pure “AI safety” shop. Safety theses need empirical work. Inside: diverse views, shared commitment to public goals — not a single-issue bias shop.
Q: If Trump wins, implications for AI safety?
Whoever is next president, we will work with them: explain why the U.S. must lead and why the tech needs sane safeguards. Message stays consistent; policies will differ by administration.
Q: RLHF / steering — jailbreaks leave dangerous capabilities intact. Fragile?
Not inherently fragile — we’re still learning how to steer. Hallucinations, jailbreaks, over-long answers when we push friendliness. Swiss-cheese defense: many imperfect layers beat one perfect layer. No single method will fully steer models.
Q: Is that responsible given the stakes?
I’d prefer an idealized world. Ours has economic pressure — firm vs firm, nation vs nation. If we can prove risks are real, maybe the world can pause briefly and cooperate. I’m not naïve: windows are rare; stopping a powerful economic train is hard.
Q: Meta open-sourcing near-parity models — responsible?
Focus isn’t “open source” per se but powerful models. I suspect most firms, Meta included, will stop open-sourcing before my worry threshold. Open weights hosted in the cloud look economically like any other service — rivals are anyone who can build powerful models.
Q: Microsoft, Amazon, Google all building internal models. Temporary lead for small labs?
Culture: do more with less. Advantage is researcher/engineer creativity — increasingly creative product work, not pure FLOPs. We still need lots of compute; efficiency means capital flows to innovative firms.
Q: ~$7B raised covers next-gen (~multi-billion train). Where does the generation after that come from?
Next few generations look funded; longer-term we will need more. Sources unclear: traditional investors, big compute providers, channels we haven’t imagined yet.
Q: Governments as capital / national projects?
As models get more powerful, governments should play a major role — not only regulation but national security. Huge internal government demand ahead; strongest models may be seen as NS-critical. Democratic governments should guide responsibly — possibly deep participation in building. In 2–5 years: should individuals or even companies sit at the core of something this powerful?
Q: Inequality and Altman’s “lift the floor, billionaires OK”?
Yes inequality is too high. Two issues: raising the floor vs power concentration — I’m more cautious than Altman on the latter. Material security with extreme power concentration still strains democracy. UBI beats nothing if we lack better options, but I’d rather a world where everyone contributes than a few amassing trillions for government redistribution. Short term: complementarity — humans + AI. Long term: rethink economy and meaning of life. Worry: progress may outrun our ability to respond.
— END —