Hugging Face AI Agent Glossary: Defining Harness, Scaffold, and Agent Architecture

Hugging Face has released a technical glossary to standardize the rapidly evolving vocabulary of AI agents. The core takeaway is that an Agent = Model + Harness, where the model provides the reasoning and the harness provides the execution loop and environmental interaction.

Core Agent Architecture

An AI agent is not merely a large language model (LLM) but a system that turns raw text generation into a loop of observation, decision, and action. This architecture is divided into three primary components:

The Model

The model is the underlying LLM (e.g., Claude, GPT, DeepSeek) that takes text input and produces text output. On its own, a model has no memory between calls and no execution loop; it can express an intent to use a tool, but it cannot execute that tool independently.

Scaffolding

Scaffolding is the behavior-defining layer that shapes how the model perceives and acts in the world. It includes:

• System prompts and tool descriptions.
• Context management, determining what the model remembers across steps.
• Response parsing logic.

While some products use the term "harness" to describe the entire system surrounding the model, the distinction between scaffold and harness is critical when reasoning about training pipelines.

Harness

The harness is the execution layer that makes the agent run. It is responsible for calling the model, handling tool calls, and deciding when the agent should stop.

Harness engineering is the discipline of designing this layer, focusing on error handling, guardrails, and termination conditions. This applies to both inference and evaluation, where an "eval harness" runs fixed scenarios to record metrics rather than updating model weights.

Operational Concepts

Context Engineering and Memory

Context engineering is the process of designing what enters the agent's context window, including system prompts, retrieved knowledge, and conversation history. This is a dynamic process managed by the harness.

• Short-term memory: Information that remains in the context window during a single run (e.g., tool results).
• Long-term memory: Information stored externally and retrieved on demand across different sessions.

Policy and Tool Use

• Policy: The behavior an agent follows, defining the probability of taking specific actions in any given situation. A policy is the result of the model weights combined with the scaffolding and harness.
• Tool Use: The mechanism by which agents interact with external systems (APIs, databases, code interpreters). The model expresses intent in a structured format, which the harness then routes to the appropriate function.

Skills and Sub-agents

• Skills: Reusable, structured packages of knowledge used to accomplish multi-step goals (e.g., "investigate a bug"), whereas a tool is a single action (e.g., "run a command").
• Sub-agents: Independent agents called by a primary agent (the orchestrator) to handle specific subtasks. Unlike tools or skills, sub-agents can reason, use their own tools, and call further sub-agents.

AI Agent Training Terminology

For those developing models, Hugging Face identifies four key components of the reinforcement learning (RL) training pipeline:

• RL Environment: A stateful object that takes an action (typically a tool call) and returns an observation (e.g., a filesystem returning a file listing after a touch command).
• Trainer: The system that runs agent episodes, scores results, and updates the inner model's weights (e.g., TRL's GRPOTrainer).
• Rollout: A full agent run from start to finish, also referred to as a trajectory or trace. This provides the raw data for RL algorithms.
• Reward: The score used to update weights. Rewards can be verifiable (pass/fail), learned (human preference), sparse (single end-of-episode score), or dense (score at each step). Rubrics are used to break rewards into weighted explicit dimensions.