aisingapore/sealion

South-East Asia Large Language Models

What it solves

SEA-LION is designed to address the lack of representation for Southeast Asian (SEA) languages, cultures, and contexts in existing Large Language Models. It provides a family of open-source models that cater to under-represented population groups and low-resource languages within the region.

How it works

The project employs a variety of training strategies across different versions:

  • Continued Pre-Training (CPT): Used in versions v2, v3, and v4.8 to adapt base models (like Llama 3.1, Gemma 2, and Nemotron) to regional languages.
  • Supervised Fine-Tuning (SFT): Used to create instruct and reasoning models, as well as specialized versions for vision-language models (VLM) and safety layers.
  • Pre-training from scratch: Used for the initial v1 models.
  • Knowledge Distillation and Model Merging: Used in v4.5 to specialize state-of-the-art foundation models for reasoning and agentic tool-use.

Additionally, the ecosystem includes SEA-LION-Embedding for regional retrieval and semantic textual similarity, and SEA-Guard for culturally attuned safety filtering.

Who it’s for

Developers, researchers, and organizations in Southeast Asia or those working with regional languages and cultural contexts who need high-performance, open-source LLMs and embedding models.

Highlights

  • Broad Language Coverage: Supports 13 regional languages, including Burmese and Malay.
  • Multimodal Capabilities: v4 models can handle image and text inputs with a 256K native context window and regional OCR.
  • Specialized Ecosystem: Includes dedicated embedding models for regional retrieval and a safety layer (SEA-Guard) tailored to SEA cultural norms.
  • Open Source Ethos: Provides open access to pre-training and fine-tuning data, training code, and evaluation benchmarks.
  • High Context Windows: Offers models with context lengths ranging up to 256K tokens.

Related

  • Project
    IDEA-CCNL/Fengshenbang-LM

    An open-source ecosystem of Chinese pre-trained foundation models and a supporting framework for natural language understanding, generation, and multimodal tasks.

  • Project
    zai-org/GLM-4

    A series of open-source LLMs featuring 9B and 32B parameter models with specialized variants for general chat, deep reasoning, and complex rumination tasks.

  • Project
    bytedance/SALMONN

    A suite of advanced multi-modal LLMs that provide large language models with generic hearing abilities and integrated audio-visual perception.

  • Dispatch
    Navigating the Embedding Model Landscape: Insights from the Community

    A synthesis of developer recommendations for embedding models, covering local, proprietary, and specialized options for RAG and cluster analysis.

  • Dispatch
    OpenAI, Cohere, and AI21 Labs Best Practices for Deploying Language Models

    OpenAI, Cohere, and AI21 Labs have released a joint set of preliminary best practices for the responsible development and deployment of large language models to mitigate risks and prevent misuse.