Flama – Production‑ready framework for predictive & generative AI

What it is – Flama (Framework for Lightweight Applications, AI Models, and Automation) is a Python library that turns any trained model – from scikit‑learn, TensorFlow, PyTorch, or a large language model – into a ready‑to‑serve HTTP API with a single command. It packages the model (weights + metadata) into a portable .flm artifact, runs a Rust‑backed server for fast request handling, and optionally exposes the model through the Model Context Protocol (MCP) so AI agents can discover and call tools.

---

Core ideas

Idea	What it gives you
One‑file model artifact	.flm bundles weights, framework info, hyper‑parameters, and optional metrics. The same file can be served on Linux (vLLM), Apple Silicon (MLX), or any supported backend without conversion.
Zero‑code serving	flama dump to create the artifact, then flama serve (or flama start with a config) launches a full OpenAPI‑compatible service instantly.
Unified API surface	Endpoints for prediction (/model/predict) and generation (/query, /stream, /chat) are generated automatically.
Multi‑dialect compatibility	The server can simultaneously expose OpenAI, Anthropic, and Ollama‑compatible routes, letting existing client libraries talk to your model unchanged.
Rust‑powered core	Routing, JSON encoding, compression, and request parsing are compiled to native code, giving low latency while still being installed as a pure‑Python wheel (no Rust toolchain needed).
MCP integration	With a simple @app.mcp.tool decorator, any Python function becomes a tool that AI agents (Claude, Copilot, custom bots) can discover via JSON‑Schema derived from type hints.
Full API framework	Beyond model serving, Flama includes CRUD resources, dependency injection, schema validation (Pydantic/Typesystem/Marshmallow), JWT auth, background tasks, pagination, and DDD‑style patterns.

---

Quick start (predictive model)

import flama
from sklearn.neural_network import MLPClassifier

model = MLPClassifier(activation="tanh", hidden_layer_sizes=(10,))
# … train the model …
flama.dump(model, family="ml", path="model.flm", metrics={"accuracy": 0.947})

# Serve it
flama serve --model file=model.flm,url=/model,name=my_model

Endpoints created

• GET /schema/ – OpenAPI description
• POST /model/predict/ – JSON‑encoded inference
• /docs/ – Swagger UI

---

Quick start (generative model)

pip install "flama[llm,pydantic]"
# 1️⃣ Download & package an LLM from HuggingFace
flama get --family llm --source huggingface mlx-community/gemma-4-E2B-it-qat-4bit
# 2️⃣ Try it locally without a server
echo "What is Flama?" | flama model mlx-community_gemma-4-E2B-it-qat-4bit.flm stream --system "Be concise."
# 3️⃣ Serve it over HTTP (includes a streaming chat UI at /chat/)
flama serve --model file=mlx-community_gemma-4-E2B-it-qat-4bit.flm,url=/,name=gemma

The same .flm runs on vLLM (CUDA) or MLX (Apple Silicon); Flama picks the backend at load time.

---

Installing

pip install flama                     # core
pip install "flama[pydantic]"        # schema validation (optional)
pip install "flama[database]"        # SQLAlchemy resources (optional)
pip install "flama[llm]"             # generative‑AI support (vLLM/MLX)
pip install "flama[full]"            # everything

Native wheels are provided for Python 3.10‑3.14 on Linux, macOS, and Windows – no Rust compiler required.

---

Typical use cases

• Model Ops – Package a trained model once, ship the .flm to any environment, and spin up a production‑grade API instantly.
• LLM as a service – Deploy an LLM behind OpenAI‑compatible endpoints so existing tooling (LangChain, OpenAI SDK) works unchanged.
• AI‑augmented tools – Expose custom Python utilities as MCP tools; agents can call them during a conversation.
• Multi‑model serving – Define several models in a flama.json spec and run them together in a single container.
• CI‑driven promotion – Inspect the embedded metrics of a .flm file in CI and gate releases without launching a server.

---

Ecosystem & extensibility

• CLI – Commands for packaging (flama dump), fetching from HuggingFace (flama get), inspecting metadata, running offline inference, and launching services.
• Plugin system – Components can be registered for custom resources, authentication, or background workers.
• Documentation & examples – Full docs at https://flama.dev, plus a curated example repo (vortico/flama-examples) covering predictive AI, generative AI, and domain‑driven design.
• Docker images – Official images (e.g., vortico/flama:latest-python3.12-pydantic) contain the Rust core and required extras; a Dockerfile only needs a FROM line and a copy of the model files.

---

License

Apache 2.0 – free for commercial and open‑source use.

---

Bottom line – Flama lets data scientists and engineers go from a trained model to a production‑grade, multi‑protocol API in minutes, while also offering the building blocks to create richer AI‑enabled services and agent‑callable tools.