IDEA-CCNL/Fengshenbang-LM

Fengshenbang-LM(封神榜大模型)是IDEA研究院认知计算与自然语言研究中心主导的大模型开源体系，成为中文AIGC和认知智能的基础设施。

What it solves

Fengshenbang-LM addresses the lack of systematic research resources and the lag in progress within the Chinese language domain compared to the English community. It aims to provide a comprehensive, standardized, and user-centric open-source ecosystem for Chinese cognitive intelligence, preventing redundant construction and saving computing power for the community.

How it works

The project provides a suite of pre-trained foundation models categorized by purpose:

  • Ziya (姜子牙): General-purpose large models (7B+ parameters) capable of translation, programming, text classification, information extraction, summarization, copywriting, common sense QA, and mathematical calculation.
  • Erlangshen (二郎神): Encoder-based bidirectional language models focused on natural language understanding (NLU) tasks.
  • Taiyi (太乙): Cross-modal models for scenarios like text-to-image generation (including a Chinese version of Stable Diffusion), protein structure prediction, and speech-text representation.
  • Wenzhong (闻仲): Models focused on language generation (NLG).
  • Randeng (燃灯): Models for language conversion (NLT), such as machine translation and summarization.
  • Yuyuan (余元): Domain-specific models for fields like medicine, finance, law, and programming.

To support these models, the FengShen framework is provided. Built on PyTorch and PyTorch-Lightning, it enables efficient pre-training on TB-level data and fine-tuning for downstream tasks using distributed training and memory-saving techniques.

Who it’s for

Researchers, developers, and institutions who need high-quality, open-source Chinese pre-trained models for various NLP and multimodal tasks, as well as those with limited computing resources who want to fine-tune existing foundation models.

Highlights

  • Comprehensive Model Suite: Covers NLU, NLG, NLT, and multimodal tasks specifically optimized for Chinese.
  • Integrated Framework: The FengShen framework simplifies the process of pre-training and fine-tuning large-scale models.
  • Multimodal Capabilities: Includes the first open-source Chinese Stable Diffusion model.
  • HuggingFace Integration: All models are synchronized to HuggingFace for easy accessibility via a few lines of code.

Related

  • Project
    datawhalechina/diy-llm

    A Chinese‑language, open‑source curriculum that teaches you to build, train, and evaluate large language models from scratch, complete with lecture notes, six hands‑on assignments, and runnable PyTorch code.

  • Project
    DLLXW/baby-llama2-chinese

    A small-parameter Chinese Llama2 implementation providing a full pipeline for pre-training, SFT, and inference, designed to help beginners learn LLM development.

  • Project
    microsoft/unilm

    A comprehensive collection of foundation models and architectures from Microsoft focusing on large-scale self-supervised pre-training across text, vision, speech, and multimodal tasks.

  • Project
    OFA-Sys/Chinese-CLIP

    A Chinese-language version of the CLIP model trained on 200 million image-text pairs for cross-modal retrieval and zero-shot image classification.

  • Project
    esbatmop/MNBVC

    A massive, open-source Chinese corpus project aiming to collect and provide 253TB of diverse internet text and multimodal data for training AI models.