EvolvingLMMs-Lab/NEO

NEO Series: Native Vision-Language Models from First Principles

What it solves

NEO addresses the limitations of modular Vision-Language Models (VLMs) by creating a native architecture that unifies how pixels and words are processed. It aims to make native VLMs more accessible and democratized, reducing the reliance on massive amounts of image-text data while maintaining high performance in visual perception.

How it works

NEO utilizes an encoder-free dense model architecture. Instead of using separate components for different modalities, it employs a native VLM primitive that handles pixel-word encoding, alignment, and reasoning all within a single unified system. This allows the model to develop strong visual perception from scratch.

Who it’s for

This project is designed for AI researchers and developers working on multimodal models, specifically those interested in scalable, cost-effective, and encoder-free native vision-language architectures.

Highlights

  • Native Architecture: Unifies pixel-word encoding, alignment, and reasoning in an encoder-free dense model.
  • Superior Efficiency: Achieves performance rivaling top-tier modular VLMs while using less than 10% of the typical image-text data.
  • Scalable Roadmap: Provides a path for building powerful native encoder-free VLMs with reusable components.
  • Comprehensive Model Zoo: Offers models in 2B and 9B parameter sizes across different training stages (NEO and NEO-ov).

Related

  • Project
    OpenSenseNova/SenseNova-U1

    SenseNova-U1 is a native multimodal model series that unifies understanding, reasoning, and generation within a single architecture, eliminating the need for modality adapters.

  • Project
    Wakals/CoVT

    CoVT is a framework that enables Vision-Language Models to reason using continuous visual tokens, improving their spatial reasoning and geometric awareness by grounding semantic thoughts in perceptual cues.

  • Project
    om-ai-lab/VLM-R1

    VLM-R1 is a framework for training R1-style Large Vision-Language Models using GRPO to improve reasoning and generalization in tasks like visual grounding, math, and object detection.

  • Dispatch
    Vision Language Models Explained

    Hugging Face provides a comprehensive guide to Vision Language Models (VLMs), detailing their architecture, open-source options, evaluation benchmarks, and new experimental support for fine-tuning via the TRL library.

  • Dispatch
    Hugging Face Vision Language Models 2025 Update

    Hugging Face provides a comprehensive overview of the 2024-2025 evolution of Vision Language Models (VLMs), highlighting trends in any-to-any architectures, reasoning models, and the rise of Vision-Language-Action (VLA) models for robotics.