OpenSenseNova/SenseNova-U1

📚 What is SenseNova‑U1 / U1.5?

SenseNova‑U is an open‑source family of native unified multimodal models that can understand images, reason about them, and generate new visual content—all with a single transformer‑style backbone. The core architecture, called NEO‑unify, removes the usual visual encoder‑VAE pipeline and treats pixels and words as a single “compound” token stream. This lets the model handle tasks such as:

• Text‑to‑image generation (including 4K resolution)
• Image editing with region‑level control
• Interleaved image‑text creation (e.g., illustrated tutorials)
• Visual question answering / VQA
• High‑density infographic creation with legible multilingual text

The repository hosts the training code, inference scripts, benchmark results, and links to the released checkpoints (8 B parameter “MoT” models) on Hugging Face, ModelScope, and community‑provided GGUF quantizations.

---

🚀 Main releases (as of Aug 2026)

Model	Size	Highlights
SenseNova‑U1‑8B‑MoT	8 B	First‑generation unified multimodal checkpoint; supports text‑to‑image, editing, VQA, interleaved generation.
SenseNova‑U1‑8B‑MoT‑Infographic‑V3	8 B	Optimised for dense text‑rich layouts, sharper Chinese/English rendering, and global style editing.
SenseNova‑U1.5‑8B‑MoT	8 B	Updated “U1.5” series – better 4K generation, higher visual fidelity, stronger instruction following, and more precise visual control.
SenseNova‑U1.5‑8B‑MoT‑LoRA‑8step	8 B (LoRA)	Light‑weight fine‑tuned version that runs faster with little loss of quality.
GGUF quantized checkpoints (community)	–	8‑bit Q8 models that fit on a single GPU with low‑VRAM.

All models are released under the Apache 2.0 license.

---

⚙️ How to try it (quick‑start)

# Text‑to‑Image example
python examples/t2i/inference.py \
  --model_path sensenova/SenseNova-U1.5-8B-MoT \
  --prompt "A formal portrait of a man in 18th‑century attire, red cloak, standing before classical ruins" \
  --output output.png

# Image editing example
python examples/editing/inference.py \
  --model_path sensenova/SenseNova-U1.5-8B-MoT \
  --image examples/editing/data/images/1.webp \
  --prompt "Change the jacket of the person on the left to bright yellow." \
  --output edited.png

For production serving you can follow the LightLLM + LightX2V guide in docs/deployment.md.

If you have a low‑VRAM GPU, install the optional GGUF dependencies (pip install "gguf>=0.10.0" "diffusers>=0.30.0") and point --gguf_checkpoint to the community‑provided Q8 file.

---

📊 Performance & Benchmarks

The README includes radar‑style plots comparing generation latency vs. average benchmark scores on a suite of multimodal tests (OneIG, LongText, BizGenEval, CVTG, IGenBench, Qwen‑Image‑Bench). The U1.5 checkpoint consistently sits near the top of the open‑source frontier, offering a good trade‑off between speed and visual quality. Detailed numbers are visualised in docs/assets/benchmarks/.

---

🛠️ What’s inside the repo?

• training/ – full‑parameter fine‑tuning pipeline (SFT, RL, MOPD) with scripts and config files.
• examples/ – ready‑to‑run inference scripts for T2I, editing, interleaved generation, and VQA.
• docs/ – model cards, benchmark tables, best‑practice guides, deployment instructions, and a technical report (PDF).
• LICENSE – Apache 2.0, allowing commercial use and redistribution.

---

🎯 Who should use this?

• Researchers exploring unified multimodal architectures or needing a strong open‑source baseline for vision‑language tasks.
• Developers building applications that require both image generation and understanding (e.g., design assistants, educational content creators, AI‑augmented editors).
• Artists & designers who want high‑quality 4K image synthesis or infographic generation without paying for proprietary APIs.

---

📌 Quick take‑aways

• Genuine open‑source multimodal model (text ↔ image ↔ reasoning) released under a permissive license.
• Built on the novel NEO‑unify architecture that eliminates separate visual encoders and VAEs.
• Provides 8 B‑parameter checkpoints for a range of use‑cases, plus LoRA and community‑quantized GGUF versions for faster inference.
• Comprehensive documentation, benchmark visualisations, and deployment guides are all included in the repo.