Claude 2 Release Notes

Anthropic has announced the release of Claude 2, a new model featuring significant improvements in reasoning, coding, and safety, alongside a vastly expanded context window. This update enables the model to process hundreds of pages of technical documentation and generate longer documents in a single output.

Expanded Context and Output Capabilities

Claude 2 supports an input capacity of up to 100K tokens per prompt. This expanded context window allows the model to analyze extensive datasets, such as entire books or comprehensive technical documentation, within a single interaction. Additionally, the model can now generate longer outputs, including stories, letters, and memos that span several thousand tokens.

Performance Benchmarks and Technical Improvements

Claude 2 demonstrates measurable improvements in reasoning, mathematics, and coding over its predecessor, Claude 1.3:

  • Legal Reasoning: Claude 2 scored 76.5% on the multiple-choice section of the Bar exam, an increase from 73.0%.
  • Coding: On the Codex HumanEval Python coding test, Claude 2 scored 71.2%, up from 56.0%.
  • Mathematics: On the GSM8k grade-school math problem set, the model scored 88.0%, compared to 85.2% for Claude 1.3.
  • Academic Testing: Claude 2 scores above the 90th percentile on GRE reading and writing exams when compared to graduate school applicants and performs similarly to the median applicant on quantitative reasoning.

Safety and Harmlessness

Anthropic has implemented iterative safety improvements to make Claude 2 more resistant to offensive or dangerous outputs. Based on internal red-teaming evaluations using automated tests and manual checks on a representative set of harmful prompts, Claude 2 is 2x better at providing harmless responses than Claude 1.3. These improvements are the result of extensive red-teaming and specific safety techniques designed to reduce the likelihood of jailbreaks.

Availability and Integration

Claude 2 is available via API for businesses at the same price point as Claude 1.3. It also powers a public-facing beta chat experience at claude.ai, available to users in the US and UK.

Enterprise Partnerships

Two key partners have already integrated Claude 2 into their platforms:

  • Jasper: The generative AI platform utilizes Claude 2 for long-form, low-latency use cases, leveraging the 3X larger context window to remix existing content and improve reasoning for complex prompts.
  • Sourcegraph: The coding assistant Cody uses Claude 2 to provide more accurate answers by passing more codebase context through the 100K context window. Cody also benefits from Claude 2's training on more recent data, providing knowledge of newer frameworks and libraries.

Sources

Related

  • Dispatch
    Claude 2.1 Release Notes

    Anthropic has released Claude 2.1, featuring a 200K token context window, a 2x reduction in hallucination rates, and a new beta feature for tool use.

  • Dispatch
    Claude Instant 1.2 Release Notes

    Anthropic has released Claude Instant 1.2, a faster and lower-priced model that improves upon version 1.1 in math, coding, reasoning, and safety.

  • Dispatch
    Claude 3.5 Sonnet release notes / what's new

    Anthropic has released Claude 3.5 Sonnet, a model that outperforms Claude 3 Opus and competitors on reasoning and coding benchmarks while operating at twice the speed of Opus.

  • Dispatch
    Claude Opus 4.1 Release Notes

    Anthropic has released Claude Opus 4.1, an upgrade to Claude Opus 4 that improves performance in agentic tasks, real-world coding, and reasoning.

  • Dispatch
    Claude Opus 4.6 Release Notes

    Anthropic has released Claude Opus 4.6, featuring a 1M token context window, state-of-the-art agentic coding and reasoning capabilities, and new developer controls for adaptive thinking and effort.