Code Llama Prompting Guide

Meta's Code Llama is available in three primary variations—Instruct, Code completion, and Python—each requiring different prompting strategies to maximize effectiveness. Understanding these distinctions allows developers to use the model for tasks ranging from natural language interaction to precise code infilling.

Code Llama Instruct: Natural Language Interaction

The instruct model is designed to provide human-like answers to questions, combining natural language explanations with code blocks. It is the most accessible variation and functions similarly to a conversational AI.

Code Generation and Explanation

The Instruct model can generate functions based on natural language requests. For example, asking the model to write a Python function for the nth Fibonacci number results in both the implementation and a brief explanation of the recursive formula used.

Code Review and Bug Detection

Code Llama Instruct can act as a secondary reviewer for commits or pull requests to identify simple bugs. When provided with a buggy function, the model can pinpoint the specific logic error, such as failing to handle a specific input case (e.g., where n is equal to 1).

Unit Test Generation

The model reduces boilerplate overhead by generating unit tests for existing functions. By passing the content of a source file to the prompt, the model can produce a comprehensive unittest class covering multiple edge cases.

Code Completion and Infill

The standard code completion model focuses on generating subsequent tokens based on the provided prompt, making it ideal for writing functions after a signature or a comment.

Standard Completion

When provided with a comment describing a function's purpose (e.g., removing whitespace from a string), the completion model generates the corresponding Python code to fulfill that requirement.

Fill-in-the-Middle (FIM) / Infill

Infill is a specialized prompt format that allows the model to complete code between two existing blocks. This is critical for inserting logic into the middle of an existing function. To use this feature, the model requires a specific tag-based format:

<PRE> {prefix} <SUF>{suffix} <MID>

In this structure, the prefix is the code before the insertion point and the suffix is the code following it. The model may return an <EOT> (End of Token) tag to signal the completion of the response.

Code Llama Python

The Python variation is fine-tuned on an additional 100 billion Python tokens. This makes it specifically optimized for Python development, including machine learning tooling and web frameworks like Django.

Model Availability and Ecosystem

Code Llama is available in 7 billion, 13 billion, and 34 billion parameter versions. While the provided examples utilize 4-bit quantization of the 7B model via Ollama, the prompts are compatible across different model providers and runtimes.

Several tools have already integrated Code Llama, including:

  • Cody: Offers an experimental version with infill support.
  • Continue: Supports Code Llama as a replacement for GPT-4.
  • Open Interpreter: Uses the model to generate functions for local terminal execution.
  • Fine-tuned variants: Specialized versions have been released by teams such as Phind and WizardLM.

Sources

Related

  • Dispatch
    Code Llama Release Notes

    Code Llama is a family of open-access models based on Llama 2, specialized for code tasks with support for infilling and long-context windows up to 100,000 tokens.

  • Dispatch
    Run Code Llama locally with Ollama

    Meta's Code Llama models are now runnable locally via Ollama, offering 7B, 13B, and 34B parameter versions plus code and Python specializations for programming tasks.

  • Dispatch
    Ollama Structured Outputs Release

    Ollama now supports structured outputs, allowing users to constrain model responses to a specific format defined by a JSON schema for increased reliability and consistency.

  • Project
    Prorise-cool/Claude-Code-Multi-Agent

    Claude Code Multi‑Agent is an open‑source framework that gives Anthropic’s Claude Code project awareness and expert‑agent capabilities. It uses a Python Hook system and a locally‑run Ollama model to automatically detect the codebase type, analyse user intent, load 300+ domain‑specific “Skills” (backend, frontend, testing, security, etc.), and enforce automatic documentation (DEVELOPMENT.md, KNOWLEDGE.md, CHANGELOG.md). After cloning, installing Ollama, a model, and uv, you drop your project into the repo and Claude Code instantly knows its context, suggests appropriate tools, and can be driven with slash commands like `/backend‑specialist` or workflow commands such as `/kiro/spec`. Key points: - Zero‑config start; all configuration lives in `.claude/settings.json` and `prompts.json`. - Hooks run on session start, user prompt, tool use, etc., handling project detection, intent analysis, and doc updates. - Skills are defined by `SKILL.md` files and are auto‑discovered. - Works entirely locally (no external API), integrates with Git, and is MIT‑licensed. Ideal for developers or small teams who want a reproducible, on‑premise AI coding assistant.

  • Dispatch
    Ollama launch command release

    Ollama has introduced the 'ollama launch' command in version 0.15+, allowing users to set up and run coding tools like Claude Code and OpenCode without manual environment variables or configuration files.