A unified framework for online RL and offline fine-tuning of diffusion and flow-matching models across image, video, and audio-video modalities.

Falcon‑Perception is an open‑source, PyTorch/MLX inference library for the Falcon multimodal transformer that can answer natural‑language queries with object detection, instance segmentation, or OCR. It provides highly‑optimized paged and batch engines, a FastAPI server, a Streamlit UI, Docker/vLLM deployment for OCR, and Colab notebooks. The 270 M‑parameter Falcon‑OCR 1.5 model delivers end‑to‑end OCR quality comparable to much larger VLMs while staying three‑times smaller.

A Windows application for video localization that automates transcription, translation, and dubbing using AI models like Faster-Whisper and various TTS engines.

A full-stack framework for building cross-platform mobile AI apps with built-in support for real-time LLM streaming and image generation.

An autonomous AI assistant for SideFX Houdini that automates node network creation, VEX/Python coding, and 3D asset generation via Meshy AI.

Reference implementations of Model Context Protocol (MCP) servers that let LLM‑based tools interact with Oracle Cloud services. Polyglot (Python, Node.js, Java) servers, stdio or HTTP transport, with container support and client‑config snippets for Cline, Cursor, and mcphost.

MOVA is an open‑source model that generates synchronized video + audio from a text prompt (optionally with a reference image). It uses a dual‑tower video‑audio architecture, provides 360p/720p checkpoints, inference scripts, LoRA fine‑tuning pipelines, evaluation code, and integrations with SGLang, ComfyUI, and a hosted API.

RS‑SDK is an open‑source, TypeScript‑based starter kit that emulates a RuneScape‑style MMO for bot research. It bundles a server emulator, a web client, a gateway, and a typed SDK so you can write async scripts (or LLM‑generated code) that control bots, experiment with goal‑directed synthesis, multi‑agent competition, and market economics. The demo server runs out‑of‑the‑box; you can also host it locally with adjustable tickrates. MIT‑licensed and backed by a Discord community.

A Minecraft Fabric mod that uses a diffusion model to generate highly diverse and realistic terrain and landscapes.

OmniLottie is a multimodal AI generator that creates complex vector Lottie animations from text, image, or video inputs using pre-trained Vision-Language Models.

A ComfyUI custom node that accelerates MiniMax H3 model inference by up to 45% using a residual caching mechanism for DiT blocks.

An AI-driven pipeline that converts text descriptions into game-ready, animated 3D creatures in GLB format using a multi-stage design and validation process.

An offline screen translator for Japanese retro games that uses OCR and local LLMs to provide real-time English subtitles via a floating overlay.

A collection of recipes for shrinking, optimizing, and customizing cutting-edge vision and multimodal AI models to improve efficiency and performance.

A standalone C++ implementation of the TRELLIS image-to-3D pipeline using GGML, enabling the generation of textured 3D models from images without Python at runtime.

A comprehensive collection of foundation models and architectures from Microsoft focusing on large-scale self-supervised pre-training across text, vision, speech, and multimodal tasks.

A perception and verification engine that turns video, audio, and screen activity into timestamped evidence for AI agents, featuring a deterministic verification system and a dedicated agent workspace.

An all-in-one toolkit for creating interactive AI characters (AITubers) with support for various LLMs, TTS engines, and 2D/3D avatars.

A ComfyUI extension that integrates Qwen-VL vision-language models for image and video understanding, featuring dual HF and GGUF backends and GPU-optimized inference.

A local-first tool for film analysis that extracts frames and subtitles to create AI-assisted narrative structure trees, plot swimlanes, and emotion curves.

A toolkit that provides specialized life science skills to AI agents, enabling them to perform tasks like protein folding, molecular docking, and genomics analysis using NVIDIA's models and libraries.

SpikingJelly is a PyTorch‑based open‑source library for building, training, and deploying spiking neural networks. It offers neuron modules, ANN‑to‑SNN conversion, multi‑backend acceleration (torch, CuPy, Triton), large‑scale training utilities, neuromorphic dataset loaders, profiling tools, and export interfaces for neuromorphic hardware. Requires Python ≥ 3.11 and PyTorch ≥ 2.6, and is released under the Open‑Intelligence Open Source License.

VLM³ is a framework that enables standard Vision Language Models to perform 3D vision tasks like depth estimation and camera pose estimation using only text-based outputs and data scaling.

A Blender addon that integrates TRELLIS.2 and ComfyUI to generate 3D meshes from text or images and apply AI-powered textures to existing models.

An API-first platform for enterprise document intelligence that converts invoices, receipts, and statements into structured JSON using Vision LLMs and agentic workflows.

A tool that converts static images of diagrams and flowcharts into editable DrawIO XML files using SAM 3 and multimodal LLMs.

A family of medical multimodal large language models that unify medical text, 2D images, and 3D volumes for advanced healthcare comprehension and generation.

A collection of open-source AI tools for biodiversity monitoring, providing models for animal detection in camera-trap imagery, bioacoustics, aerial views, and sonar.

xLSTM is a novel recurrent neural‑network architecture that extends LSTM with exponential gating and a matrix memory, enabling large‑scale language models (e.g., a 7 B‑parameter model). The repo provides PyTorch code, custom CUDA/Triton kernels (via the `mlstm_kernels` package), a single‑file 7B implementation, configuration utilities, and pretrained weights on Hugging Face. Installation is via conda + pip, and the library works on NVIDIA GPUs (fast Triton kernels) with fallbacks for AMD or Apple Silicon.

Maestro‑Flow is a TypeScript/Node.js framework that turns natural‑language intents into full‑stack development pipelines executed by multiple LLM agents (Claude, Gemini, Codex, etc.). Its Ralph engine classifies intents into 40+ command chains, dynamically decides at “decision” checkpoints whether to continue, roll back, or insert repair loops, and stores discovered patterns in a SQLite knowledge graph that is automatically injected into later runs. It supports four coordination modes (Delegate, Team, Wave, Swarm), long‑running autonomous loops via the Odyssey command, and precise literal/semantic code search. Installation is via `npm i -g maestro-flow`, and the tool provides extensive CLI and web‑dashboard documentation.

FigMirror is an agentic tool that automatically generates Matplotlib code to plot user data in the style of a provided reference scientific figure.

A Chinese-language version of the CLIP model trained on 200 million image-text pairs for cross-modal retrieval and zero-shot image classification.

AnySearch MCP Server is a real‑time, MCP‑compatible backend that provides general web search, vertical‑domain search, parallel batch search, and URL‑content extraction. It works with popular AI‑assistant IDEs (OpenCode, Claude Code, Cursor, VS Code, Windsurf, Cline) via simple JSON configuration, supports optional API‑key authentication for higher rate limits, and even allows anonymous use. The README details registration, key handling, client‑specific config snippets, and the four tool endpoints (`search`, `get_sub_domains`, `batch_search`, `extract`).

A canvas-based media workbench for ComfyUI that integrates AI generation with professional-grade editing tools for image, video, audio, music, and 3D assets.

LLM Vision is a Home Assistant integration that uses multimodal large‑language models (OpenAI, Anthropic, Gemini, Ollama, etc.) to analyze images, video clips, live camera feeds and Frigate events. It returns natural‑language descriptions, remembers identified people/pets/objects, and stores each analysis in a timeline that can be shown on the dashboard or queried via Assist. Install via HACS, configure an LLM provider, and optionally add the supplied Blueprint for AI‑summarized camera notifications.

HelixWorld is a real-time interactive audio-visual world model that generates navigable environments where sound and vision update synchronously based on user movement.

An open-source AI toolkit for fashion technology providing a unified interface for virtual try-on, image/video generation, and multimodal understanding.

AI Team OS is an open‑source platform that turns a Claude‑style LLM into an autonomous “CEO”. It continuously runs without prompting, orchestrating specialist agents, a task wall, research pipelines, and a two‑layer memory system. The FastAPI + React dashboard shows live workflows, decision traces, and a unified search over all memos. It’s aimed at technical founders or research teams who want an AI‑first, self‑driving company‑like workflow.

VLM-R1 is a framework for training R1-style Large Vision-Language Models using GRPO to improve reasoning and generalization in tasks like visual grounding, math, and object detection.

Merlin is a 3D vision-language foundation model and dataset for computed tomography that integrates CT scans with electronic health records and radiology reports for medical analysis.

A collection of on-device demo apps for iOS, Android, and HarmonyOS that allow the MiniCPM-V family of multimodal and text models to run fully locally via llama.cpp.

A real-time AI assistant for Meta Ray-Ban smart glasses that uses Gemini Live to see, hear, and take actions via OpenClaw integration.

A modularized codebase for building and training small-scale large multimodal models, supporting a wide variety of LLMs, vision towers, and connectors.

A toolkit that gives text-only LLM agents visual capabilities through task-aware vision tools and seamless proxy integrations for image Q&A and GUI automation.

simple‑one‑api is an open‑source Go gateway that unifies multiple LLM providers behind OpenAI‑compatible endpoints, adds capacity‑aware key scheduling, four‑layer rate‑limiting, a built‑in React chat UI, a visual config console, and a Wails desktop client. It can be run as a single binary, Docker container, or desktop app, and stores its configuration in SQLite.

An AI agent that uses LLMs and image recognition to automate daily tasks, navigation, and activities in the game Infinity Nikki via screen capture and input simulation.

torch_musa is a PyTorch extension that adds a **MUSA** backend for Moore Threads GPUs. It mirrors the standard PyTorch API, so you only replace `cuda` with `musa`. The package includes tools for building C/C++ extensions, distributed training via the **mccl** backend, and Docker images for easy setup. It also provides MUSA‑compatible builds of torchvision, torchaudio, and a curated list of other popular PyTorch libraries. Installation can be done from pre‑built wheels or by building from source after installing the MUSA SDK. The project is BSD‑licensed.

A local-only infinite-canvas workspace for AI media workflows that uses a graph-based approach to compose consistent characters and products into images and videos via Google Flow.