NPC-Worldwide/npcpy

The python library for research and development in NLP, multimodal LLMs, Agents, ML, Knowledge Graphs, and more.

What it solves

npcpy is a library designed to simplify the development of multimodal AI applications, agentic AI, and knowledge graphs. It provides a unified interface for interacting with various local and cloud LLM providers, reducing the complexity of context engineering and multi-agent orchestration. It also addresses the-gap between prompt-based compliance and software-enforced compliance through its NPC Context-Agent-Tool data layer.

How it works

The library uses a NPC (Non-Player Character) class to define personas with specific directives. It supports a wide range of providers (including Ollama, llama.cpp, Gemini, and others) and offers specialized agent types:

  • Agent & ToolAgent: Agents that can use default tools (like shell, Python, and web search) or custom Python functions as tools.
  • CodingAgent: An agent that automatically executes code blocks generated by the LLM.
  • NPCArray: A framework for running parallel inference and multi-agent debates where agents can critique and refine each other's responses.
  • Knowledge Graph: A system for extracting facts and concepts from text, with a "sleep/dream" lifecycle for consolidation and speculative connection generation.
  • Team: A system for orchestrating multiple NPCs using configuration files (.npc, .jinx, team.ctx) to define roles and shared context.

Who it’s for

Developers and researchers building complex AI agents, multi-agent systems, and multimodal applications that require flexible model provider switching and structured memory systems.

Highlights

  • Multi-Provider Support: Works with local providers like Ollama and LM Studio, and cloud providers like Gemini and OpenAI.
  • Multi-Agent Orchestration: Supports parallel inference, iterative refinement, and structured team delegation via the Team class.
  • Multimodal Capabilities: Integrated support for image, audio, and video generation.
  • Structured Output: Built-in support for JSON and Pydantic models for auto-parsed responses.
  • Knowledge Graph Memory: A unique memory system that evolves incrementally and uses a sleep/dream process to prune and expand knowledge.
  • Executable Configs: Ability to define agents and skills (jinxes) in YAML-based files that can be executed directly from the shell.

Related

  • Project
    AgentEra/Agently

    Agently is a Python framework that turns LLM calls into structured, observable, and restart‑safe runtime units. It provides contract‑first output schemas, streaming of partial results, tool/action integration, versioned skills, managed execution resources, and an event‑driven workflow engine (TriggerFlow) for building reliable AI services.

  • Project
    livekit/agents

    LiveKit Agents is a Python SDK for building real‑time, multimodal voice agents (LLM‑driven assistants) that run on LiveKit servers. It provides unified adapters for STT/LLM/TTS, job scheduling, WebRTC/telephony integration, turn detection, MCP tool support, and a built‑in test framework.

  • Project
    pydantic/pydantic-ai

    A typed Python AI SDK for building extensible agents with structured outputs, durable execution, and support for multiple model providers and interfaces.

  • Project
    NPC-Worldwide/npcsh

    npcsh is an open‑source, Rust‑based interactive shell that blends Bash commands with natural‑language queries. It routes requests to LLM‑powered agents defined via YAML or markdown, uses Jinja “jinxes” as reusable skill templates, and works with any LiteLLM‑compatible model (Ollama, OpenAI, Anthropic, etc.). Install via a one‑line script or Cargo, and a Python backend (`npcpy`) handles the inference loop. Built‑in commands like `/git`, `/update`, and `/sync` streamline development workflows, while a context file lets users add custom instruction keys. The project includes benchmarks across many models and is MIT‑licensed.

  • Project
    vortico/flama

    Flama is a production framework that packages predictive and generative AI models into portable files and serves them as high-performance APIs with built-in support for agentic tools and standard LLM protocols.