aisingapore/sealion

What it solves

SEA-LION addresses the lack of representation for Southeast Asian (SEA) languages and cultures in Large Language Models. It provides a family of open-source models designed to better understand diverse regional contexts, catering to under-represented population groups and low-resource languages within the SEA region.

How it works

The project employs various training strategies across different model versions to achieve regional specialization:

• Pre-training from scratch: Used in v1 to build foundational regional knowledge.
• Continued Pre-Training (CPT): Used in v2, v3, and v4 to adapt existing open-source foundation models (like Llama and Gemma) to the SEA region.
• Supervised Fine-Tuning (SFT): Used to create instruct, reasoning, and multimodal versions of the models.
• Knowledge Distillation and Model Merging: Used in v4.5 to deliver high-capacity reasoning and agentic tool-use capabilities.

Who it’s for

Developers and researchers who need LLMs that are culturally and linguistically attuned to Southeast Asia, as well as those seeking open-source regional models for retrieval, reasoning, and multimodal tasks.

Highlights

• Diverse Model Family: Includes text-based LLMs, multimodal VLMs (Vision-Language Models) with up to 256K context windows, and specialized embedding models for regional retrieval.
• Regional Safety Layer: Includes SEA-Guard, a dedicated safety model designed with culturally attuned diagnostic tests for the SEA region.
• Regional Benchmarking: Evaluated using SEA-HELM and SEA-BED to ensure high performance in regional languages and contexts.
• Open Source Ethos: Provides open access to pre-training and fine-tuning data, training code, and evaluation benchmarks.