Armature: Product Analytics for Agentic Sessions

Armature is a product analytics platform designed specifically for agentic sessions. Unlike traditional analytics tools that track UI clicks, Armature captures the interactions between AI agents (such as those in Claude, ChatGPT, or Cursor) and a product's backend via the Model Context Protocol (MCP), Claude Connectors, or ChatGPT Apps.

Key Capabilities for Agentic Product Management

Armature allows product teams to move beyond raw logs to understand how AI agents are actually utilizing their services. The platform provides several core features to optimize the agent experience:

Automated Use-Case Identification

Armature uses models to analyze every session and automatically group them into use cases. This allows teams to identify the most common tasks users are attempting to perform, including identifying "missing" features—use cases that users are attempting to perform but the product does not yet support.

Root Cause Analysis for Agent Failures

The platform scans sessions for failures, loops, and dead ends. It identifies root causes even when API responses return a 200 OK status, recognizing that a successful technical response can still result in a failure to satisfy the user's intent.

Session Replay and Scoring

Every session is assigned a success score based on whether the user's request was fulfilled. Teams can replay the full trace of a session, including the initial user intent, the agent's internal thinking process, and every API call made during the session.

Technical Implementation and Integration

Integration is achieved through an SDK that wraps existing MCP servers, Claude Connectors, or ChatGPT App backends. This implementation requires no changes to the existing server logic and is compatible with any client that can reach the MCP server, including Claude, ChatGPT, Cursor, Codex, and Gemini CLI.

Data Privacy and PII Redaction

To handle sensitive information, Armature employs detection models that scan sessions and redact personally identifiable information (PII) and secrets by default before the data is stored.

Comparison with Existing Tooling

Armature distinguishes itself from traditional analytics and LLM observability tools through its target audience and user focus:

Tool Category Focus Armature's Approach
Traditional Analytics (e.g., PostHog, Amplitude) Human UI interactions (clicks, events) Captures sessions happening inside AI clients (Claude, ChatGPT) where the product's UI is invisible.
LLM Observability (e.g., LangSmith, Langfuse) Engineering-focused traces for agents built by the developer Focuses on product management for agents that use the product via external clients.

Community Feedback and Considerations

Community members on Hacker News have raised several technical and compliance questions regarding the implementation:

"How is it actually getting the model's thoughts? ... If im already logging MCP calls with OpenTelementry or BrainFuse why would i need Armature?"

Users have also noted the non-deterministic nature of LLMs, which makes visibility into agentic sessions critical for developers building MCP-only products.

Additionally, concerns were raised regarding whether the tool conforms with the OpenAI and Anthropic telemetry and data collection terms, particularly concerning the chat data and current conversation telemetry.

Sources

Related

  • Project
    Mathews-Tom/armory

    A curated collection of production-grade skills, agents, and workflows for AI coding agents like Claude Code, providing standardized automation for the full software development lifecycle.

  • Dispatch
    Writing Effective Tools for AI Agents: Anthropic Engineering Guide

    Anthropic outlines a systematic, evaluation-driven framework for designing tools for non-deterministic AI agents, emphasizing agent-centric design over traditional deterministic API patterns.

  • Project
    railwayapp/railway-skills

    A collection of agent skills and plugins that enable AI agents like Claude, ChatGPT, and Cursor to manage Railway cloud infrastructure.

  • Project
    asheshgoplani/agent-deck

    Agent Deck is a Go‑written terminal UI that lets you launch, monitor, fork, and orchestrate multiple AI coding agents (Claude, OpenCode, Codex, Pi, …) from a single pane. It adds group‑based policies, per‑group Claude credentials, MCP tool toggles, cost dashboards, git‑worktree handling, and a phone‑controlled *conductor* that can be queried via Telegram/Slack/Discord. Install with a one‑liner script, Homebrew, or `go install`, then manage sessions with keyboard shortcuts (`Enter` to attach, `f` to fork, `$` for cost, `/` to search, etc.).

  • Dispatch
    Anthropic Independent Research Pilot on Claude Usage Data

    Anthropic has piloted a program allowing external researchers to analyze real-world Claude usage data via a privacy-preserving tool called Anthropic Insights, releasing aggregate findings on human-AI collaboration and productivity.