Dwinovo/minecraft-numen

What it solves

Numen provides an AI companion for Minecraft that can understand and execute complex tasks described in natural language. Unlike simple chat NPCs, it acts as a real player on the server, capable of performing physical actions like mining, building, and fighting, effectively automating tedious gameplay tasks through high-level instructions.

How it works

The system treats the AI companion as a ServerPlayer (a fake player on the server side), ensuring it follows the same game rules as human players. It operates via a feedback loop consisting of four components:

• Body: A server-side player entity that executes native game code paths for movement and interaction.
• Eyes (Perception API): Scans blocks, entities, and recipes, and can inspect machine contents without opening a GUI.
• Hands (Action API): Handles movement, mining, placing blocks, combat, and inventory management.
• Feedback Loop: Every action's success or failure is fed back to the LLM, allowing the AI to learn and adjust its strategy in real-time (e.g., learning that iron ore requires a stone pickaxe).

The "brain" runs on the user's own machine using their own API key, connecting to various LLM providers (OpenAI, Claude, DeepSeek, etc.) via a direct connection.

Who it’s for

• Minecraft players who want an intelligent assistant to help with resource gathering or base construction.
• Modders who want to integrate AI capabilities into their mods via the provided API and plugin system.
• AI enthusiasts who want to experiment with Model Context Protocol (MCP) to control a game world from external AI clients like Claude Desktop or Cursor.

Highlights

• Natural Language Control: Supports text and voice input/output across multiple languages.
• Native Integration: Acts as a real player, making it compatible with redstone, monster AI, and other mods.
• Extensible Knowledge: New gameplay mechanics or mod-specific workflows can be taught using simple Markdown files ("skills") or dedicated Java plugins.
• MCP Support: Can act as an MCP server, allowing external AI clients to drive the in-game companion.
• Server-Side Execution: Actions are validated by the server, making it viable for multiplayer environments.