LongCat-2.0 MoE Model: 1.6T Parameters and ASIC-Based Training

LongCat-2.0 is a large-scale Mixture-of-Experts (MoE) model with 1.6 trillion total parameters and 48 billion active parameters. Its primary significance lies in its successful training and deployment on large-scale clusters of AI ASIC superpods, demonstrating a viable alternative to the Nvidia GPU ecosystem for training trillion-parameter models.

Architecture and Training Scale

LongCat-2.0 utilizes a Mixture-of-Experts (MoE) architecture to balance model capacity with computational efficiency. While the model possesses 1.6 trillion total parameters, only 48 billion are active during any single inference pass, reducing the compute requirements per token.

Key training statistics include:

  • Dataset Size: Pretraining spanned over 35 trillion tokens.
  • Compute Infrastructure: The model was built on clusters of tens of thousands of AI ASIC superpods. Community analysis suggests the use of Huawei Ascend 910C chips, with some estimates placing the scale at approximately 1,024 superpods (roughly 50,000 chips).
  • Development Origin: The model is associated with Meituan, a major Chinese food delivery and commerce company.

Technical Innovations and Infrastructure

Beyond the parameter count, the development of LongCat-2.0 highlights a shift toward specialized hardware for AI training. The team invested significant effort into building a stable, secure, and scalable infrastructure to compensate for the less developed software ecosystem surrounding AI ASICs compared to the mature Nvidia CUDA environment.

Some technical discussions regarding the model's design include:

  • N-gram Embedding: The model incorporates N-gram embedding, a technique designed to improve how the model handles tokenization and semantic representation.
  • Architectural Influence: Observers note that while LongCat-2.0 builds upon work similar to the DeepSeek architecture, it introduces distinct architectural contributions beyond simple post-training or fine-tuning.

Community Evaluation and Observations

Early user testing and community discussions have raised several points regarding the model's performance, accessibility, and alignment:

Performance Benchmarks

In comparative testing against other frontier models, some users report mixed results. In one specific test involving nuclear physics (comparing U-235 and Pu-241 fuels), a user noted that LongCat-2.0 provided a well-reasoned but incorrect answer, while Qwen 3.7 Plus and Gemini Flash provided correct answers with stronger arguments.

Accessibility and Openness

There is significant community skepticism regarding the "open" nature of the model. Users have reported that Hugging Face and GitHub links provided by the developers have returned 404 errors, leading to concerns about the availability of the model weights and the full technical report.

Content Filtering and Alignment

Users have observed strict content filtering on sensitive political topics. Queries regarding the Tiananmen Square protests or Chairman Mao resulted in the model refusing to answer or returning "Too many requests" errors, indicating heavy alignment with regional regulatory requirements.

Language Behavior

Some users reported that when using the "Search" feature with the application set to English, the model occasionally returned results in Chinese, suggesting a strong influence of the original training data context on the model's output.

Sources

Related

  • Dispatch
    Qwen1.5-MoE-A2.7B Release Notes

    Qwen introduces Qwen1.5-MoE-A2.7B, a Mixture-of-Experts model that matches the performance of 7B dense models while using only 2.7 billion activated parameters.

  • Dispatch
    Architectural Choices in China's Open‑Source AI Ecosystem: From DeepSeek R1 to a Hardware‑First, MoE‑Driven Landscape

    One year after DeepSeek R1’s open‑source release, China’s AI community shifted from chasing the biggest single‑model performance to building flexible, cost‑effective, and hardware‑aware AI systems. Mixture‑of‑Experts (MoE) became the default architecture, enabling huge models to run affordably by activating only a subset of experts per request. Multimodal races exploded, with open releases for text‑to‑image, video, audio, 3‑D, and agents, each bundled with full toolchains. Small models (≤30 B) surged in popularity for local deployment and fine‑tuning, while large MoE models serve as teacher nets for distillation. Apache 2.0 and MIT licenses now dominate, removing legal friction and accelerating commercial adoption. A hardware‑first mindset emerged: releases ship with quantization, inference, and serving stacks tuned for domestic chips (Huawei Ascend, Cambricon, Kunlun), and training pipelines are openly documented. The competitive edge now lies in system design, deployment efficiency, and open‑source ecosystem integration rather than raw model size.

  • Dispatch
    Qwen2.5-Max Release Notes

    Qwen has released Qwen2.5-Max, a large-scale Mixture-of-Experts (MoE) model pretrained on over 20 trillion tokens and available via API and Qwen Chat.

  • Dispatch
    Kimi K2.7-Code Release Notes: Open-Source Coding Model with Enhanced Token Efficiency

    Moonshot AI has released Kimi K2.7-Code, a 1T parameter Mixture-of-Experts model designed for complex software engineering workflows with 30% fewer thinking tokens than K2.6.

  • Dispatch
    Liquid AI's LFM2.5-8B-A1B: Pushing the Limits of On-Device MoE

    Liquid AI introduces LFM2.5-8B-A1B, a high-throughput Mixture-of-Experts model trained on 38 trillion tokens, optimized for private, on-device agentic workflows.