Anthropic Series B Funding for AI Safety and Research

Anthropic has raised $580 million in Series B funding to develop large-scale experimental infrastructure for improving the safety properties of computationally intensive AI models. This funding allows the company to explore how capabilities and safety issues emerge at scale while building systems that are more steerable, robust, and interpretable.

Core Research Objectives

Anthropic's research focuses on creating large-scale models with better implicit safeguards that require fewer after-training interventions. The company aims to develop tools to inspect the internal workings of these models to verify that safety safeguards are functioning as intended.

Interpretability

Anthropic has worked on mathematically reverse engineering the behavior of small language models and researching the source of pattern-matching behavior in large language models.

Steerability and Robustness

To make models more "helpful and harmless," Anthropic has developed baseline techniques and utilized reinforcement learning to enhance these properties. The company has also released a dataset to assist other research labs in training models aligned with human preferences.

Scaling and Safety

Anthropic has published analysis on sudden changes in performance in large language models, highlighting the need to study safety issues at scale. CEO Dario Amodei stated that the company will use the funding to explore predictable scaling properties of machine learning systems and examine the unpredictable ways safety issues emerge as models grow.

Organizational Growth and Governance

Anthropic is expanding its team from approximately 40 people in San Francisco and is focusing on establishing a culture and governance structure to ensure the responsible development of safe AI systems during its growth phase.

Funding Details

Following a $124 million Series A round in 2021, the Series B round was led by Sam Bankman-Fried (CEO of FTX) and included participation from Caroline Ellison, Jim McClave, Nishad Singh, Jaan Tallinn, and the Center for Emerging Risk Research (CERR).