Linum AI Pyramid-JiT Model Release

Overview

Linum AI has released Pyramid-JiT (P-JiT), an experimental text-to-image model trained from scratch for approximately $21,000. The project serves as a proof-of-concept for the lab's broader goal of developing efficient, controllable generative video models. By bypassing traditional Variational Autoencoders (VAEs) and operating directly in pixel space, the team achieved a 32×32 token reduction, significantly lowering the attention context-window requirements for both training and inference.

Architectural Innovations

The core innovation of P-JiT is the elimination of the VAE, which the researchers argue introduces an empirical ceiling on compression and can negatively impact downstream generation. Instead, the model uses a diffusion-based approach to learn a latent space purpose-built for its specific use case. Key technical features include:

• Pixel-Space Operation: By operating in pixel space, the model allows every block of the Diffusion Transformer (DiT) to predict versions of the final image, boosting gradient signals in early network blocks.
• Pyramid-JiT Structure: The architecture employs a 32×32 token reduction, representing a 512² image with the same number of tokens typically used for 256² images, resulting in a 4× reduction in attention costs.
• Readout Mechanisms: The model utilizes PixelREPA readouts to align the latent space with DINOv3-L features and includes linear readout heads along the trunk to predict low-resolution versions of the final image, accelerating convergence.
• Training Regime: The model was trained on 660 million samples using a curriculum-based noise schedule. It uses a frozen Qwen3.5-4B model for text representation and employs Muon optimization on 2D block matrices.

Training and Performance

The total cost of $21,038 covered 8,415 H100-hours of compute. The training was split into two stages: a 512² resolution phase (610M samples) followed by a 1024² resolution phase (50M samples). Post-training was conducted using Finite Difference Flow Optimization (FDFO) with human-in-the-loop feedback.

While the model demonstrates strong image generation capabilities, the developers note several limitations:

• Text Generation: The model was not trained on rendered text (e.g., PDFs, infographics) and cannot generate readable text or posters.
• Complex Scenes: Due to its 2B-parameter size, the model occasionally struggles with full-body anatomy, multi-person interactions, and complex entity counts.

Community Insights

Discussion on Hacker News highlighted both the technical achievement and the current limitations of the model. Commenters noted that while the efficiency is impressive, the model still exhibits common artifacts such as over-smoothing and anatomical errors in hands.

Regarding the decision to abandon the VAE, one user remarked:

"Throwing away the VAE feels counterintuitive - I'm shocked it works so well! Appreciate that you shared your training insights on the blog post!"

Others pointed out that the model's efficiency is a significant departure from the resource-heavy approaches of larger labs. As one observer noted:

"The big labs if anything are disincentivized to make efficient models as currently access to compute is one of their moats. That's not the same for Chinese labs, which is why you see groundbreaking efficiency work mostly coming from China. Really happy to see a small shop like yours doing this sort of work!"

The model weights have been released under the Apache 2.0 license and are available on Hugging Face.