---
title: "Models | Anthropic CEO Dario Amodei on AI Safety, Oligopoly, and Open Weights"
author: "Unique Research"
sourcePublication: "Unique Research (FFCAP English library)"
originalPublishedAt: "2024-09-03T16:30:00+08:00"
canonical: "https://ffcap.cn/en/research/20240903a066i100"
source: "https://ffcap.cn/en/research/20240903a066i100"
language: "en"
cnSourceUrl: "https://view.inews.qq.com/a/20240903A066I100"
---

# Models | Anthropic CEO Dario Amodei on AI Safety, Oligopoly, and Open Weights

_Unique Research · Insight / Interview Digest · September 3, 2024 · Source: TIME interview with Anthropic CEO Dario Amodei (compiled by Unique Research)_

**Editor’s note.** Anthropic (San Francisco, 2021) was founded by seven ex-OpenAI alumni including siblings Dario and Daniela Amodei and GPT-3 lead Tom Brown. Dario (Stanford physics BS; Princeton biophysics PhD) previously led GPT-2/GPT-3 work as OpenAI research VP. Mission: reliable, interpretable, steerable frontier systems. Flagship Claude — Claude 2 first commercial 200K-context model; Claude 3 topped many evals vs GPT-4. Raised ~$1.45B (second-richest AI startup after OpenAI at CN publish); Amazon invested $4B for AWS infra. Public-benefit corp framed around helpful / harmless / honest systems. May 2024 TIME cover; TIME 100 Most Influential Companies 2024. June 20: Claude 3.5 Sonnet — claimed new bars on reasoning, coding, some math, beating GPT-4o. Amodei: goal is not incremental LLMs but systems that meaningfully collaborate with people and software. Unlike OpenAI’s path, Anthropic’s public safety commitments are unusually explicit — while most staff expect progress not to slow, with potential downside risk.

## Q&A highlights

**Q: Smallest frontier lab, less capital — disadvantage on safety?**  
**Amodei:** Those facts used to be true; we’ve signed large compute deals and raised billions. Rivals still sit from billions to tens of billions — some shockingly toward hundreds of billions. Calling ourselves underdogs feels awkward now, but versus the giants we still are.

**Q: Scaling laws — new capabilities in unreleased systems?**  
We just shipped a model; there’s no year-long secret backlog — and I wouldn’t tip secrets anyway. Progress has not slowed. Real world can interrupt anytime. Every new train I watch with excitement and dread — will the curve stop? If scaling stopped, that might be good for the world (it would bind everyone). We don’t choose that; we discover which universe we live in and adapt. Models also bring exciting upside.

**Q: What culture are you building?**  
Donation matching: employees can gift up to 25% of equity to charity with company match — attracts public-interest talent without killing economic incentive. Framing: I’d rather cast Anthropic as a public-benefit company than a pure “AI safety” shop. Safety theses need empirical work. Inside: diverse views, shared commitment to public goals — not a single-issue bias shop.

**Q: If Trump wins, implications for AI safety?**  
Whoever is next president, we will work with them: explain why the U.S. must lead _and_ why the tech needs sane safeguards. Message stays consistent; policies will differ by administration.

**Q: RLHF / steering — jailbreaks leave dangerous capabilities intact. Fragile?**  
Not inherently fragile — we’re still learning how to steer. Hallucinations, jailbreaks, over-long answers when we push friendliness. Swiss-cheese defense: many imperfect layers beat one perfect layer. No single method will fully steer models.

**Q: Is that responsible given the stakes?**  
I’d prefer an idealized world. Ours has economic pressure — firm vs firm, nation vs nation. If we can prove risks are real, maybe the world can pause briefly and cooperate. I’m not naïve: windows are rare; stopping a powerful economic train is hard.

**Q: Meta open-sourcing near-parity models — responsible?**  
Focus isn’t “open source” per se but powerful models. I suspect most firms, Meta included, will stop open-sourcing before my worry threshold. Open weights hosted in the cloud look economically like any other service — rivals are anyone who can build powerful models.

**Q: Microsoft, Amazon, Google all building internal models. Temporary lead for small labs?**  
Culture: do more with less. Advantage is researcher/engineer creativity — increasingly creative product work, not pure FLOPs. We still need lots of compute; efficiency means capital flows to innovative firms.

**Q: ~$7B raised covers next-gen (~multi-billion train). Where does the generation after that come from?**  
Next few generations look funded; longer-term we will need more. Sources unclear: traditional investors, big compute providers, channels we haven’t imagined yet.

**Q: Governments as capital / national projects?**  
As models get more powerful, governments should play a major role — not only regulation but national security. Huge internal government demand ahead; strongest models may be seen as NS-critical. Democratic governments should guide responsibly — possibly deep participation in building. In 2–5 years: should individuals or even companies sit at the core of something this powerful?

**Q: Inequality and Altman’s “lift the floor, billionaires OK”?**  
Yes inequality is too high. Two issues: raising the floor vs power concentration — I’m more cautious than Altman on the latter. Material security with extreme power concentration still strains democracy. UBI beats nothing if we lack better options, but I’d rather a world where everyone contributes than a few amassing trillions for government redistribution. Short term: complementarity — humans + AI. Long term: rethink economy and meaning of life. Worry: progress may outrun our ability to respond.

— END —

---

Editorial English edition published on UniqueCapital: https://ffcap.cn/en/research/20240903a066i100
On-site reading page: https://ffcap.cn/en/research/20240903a066i100