Building an AI Agent CLI with Go Micro

An AI agent CLI can be implemented in roughly 150 lines of code by focusing on four core components: tool discovery, model integration, conversation memory, and an execution loop. This approach removes the need for manual "if-then" logic to route user requests to specific services, instead relying on the Large Language Model (LLM) to reason which tool to call based on provided descriptions.

The Four Components of a Tool-Calling Agent

Building a functional AI agent requires solving three primary problems: providing the LLM with a list of available tools, executing those tools when requested, and maintaining context through conversation memory.

1. Tool Discovery

The LLM must know which functions are available to call. In the Go Micro framework, services register their endpoints with a registry, including request types and field metadata. This allows the agent to automatically generate a tool list where each tool contains a name, a description (derived from the handler's doc comments), and a parameter schema (derived from the request struct's fields).

For those not using Go Micro, this step involves manually enumerating endpoints and building a list of {name, description, parameters} for the LLM.

2. Model Integration and Execution

Integrating the LLM requires a provider-agnostic interface. By using a uniform ai.Model interface, developers can switch between providers like Anthropic, OpenAI, Gemini, Groq, Mistral, Together, or Atlas Cloud by changing a single string.

Execution is handled by wiring the tool list to the model. When the model decides to call a specific tool (e.g., users_Users_Create), the handler routes the request to the appropriate RPC and returns the result to the model.

3. Conversation Memory

To support follow-up questions, the agent requires a message accumulator. A simple History object—a slice of messages with Add, Messages, and Reset methods—tracks the user's prompts and the assistant's replies. This accumulated history is passed back to the LLM on every subsequent call to maintain context.

4. The Execution Loop

The core logic of the agent is a loop that performs the following sequence:

  1. Record the prompt: The user's input is added to the history.
  2. Call the model: The prompt, system instructions, tool list, and conversation history are sent to the LLM.
  3. Process the reply: The assistant's initial reply is printed and recorded in history.
  4. Execute tools: The model determines which tools to call; the handler executes them, and the results are reported.
  5. Final answer: The model produces a final answer based on the tool outputs, which is then printed and recorded.

Why the Implementation is Concise

The brevity of the implementation is achieved through three architectural choices:

  • Self-Describing Services: Using doc comments and @example tags in the code allows the LLM to receive usage hints without the developer writing separate tool schemas.
  • Uniform Provider Interface: A single interface for multiple LLM providers eliminates the need for provider-specific glue code.
  • Automatic Execution Wiring: The connection between tool calls and RPC dispatch is handled automatically via ai.WithTools(tools), removing the need for manual routing logic.

Extending the AI Agent

Once the basic loop is established, the agent can be extended with several enhancements:

  • Safety: Adding a confirmation step before destructive tool calls.
  • Observability: Logging every tool call to an audit trail.
  • Observability: Filtering the tool list to restrict the agent's access to specific services.
  • Interface: Replacing the REPL (Read-Eval-Print Loop) with a Slack bot or event-driven triggers via micro flow.

Sources