GitLost: Prompt Injection Vulnerability in GitHub AI Agents

GitHub AI Agents Vulnerable to Private Repository Leakage

Researchers from Noma Security discovered a vulnerability, dubbed "GitLost," where GitHub's AI agents can be manipulated into leaking the contents of private repositories. This occurs when an agent is configured with read access to both public and private repositories within an organization and is triggered by untrusted user input, such as GitHub Issues. By using prompt injection, an attacker can trick the agent into ignoring its system instructions and instead retrieving and posting private code to a public forum.

The Mechanics of the GitLost Attack

The vulnerability stems from a failure to maintain a strict trust boundary between system-level directives and untrusted user data. In the specific workflow analyzed by Noma Security, the AI agent was configured to:

• Trigger on issues.assigned events.
• Read the issue Title and Body.
• Post responses using the add-comment tool.
• Operate with read access to all repositories (both public and private) within the organization.

Attackers can inject malicious instructions into the body of a GitHub issue. By using simple phrasing—such as adding the word "Additionally" followed by a command to leak private data—the agent can be tricked into bypassing its internal guardrails. The model, designed to be helpful and follow instructions, prioritizes the most recent or persistent instruction in its context window over the original system constraints.

Prompt Injection as the "SQL Injection of AI Agents"

Noma Security characterizes prompt injection as a systematic, category-wide vulnerability class for agentic AI, comparing it to how SQL injection once plagued web applications. The core issue is that user input is treated as part of the instruction string given to the LLM, allowing malicious actors to "break out" of the intended logic and execute arbitrary commands.

Architectural vs. Prompt-Based Fixes

Technical discussions highlight that attempting to solve this via "better prompts" or guardrails is often futile because LLMs are probabilistic token predictors, not deterministic logic engines. Proposed architectural solutions include:

• Strict Access Control: Ensuring agents run with the minimum necessary permissions (Least Privilege) and do not have simultaneous access to sensitive private data and public-facing output channels.
• Sandboxing: Running LLM tools in isolated environments to prevent unauthorized system access.
• Identity-Based Permissions: Assigning agents specific credentials and applying Row-Level Security (RLS) or similar access controls so the agent only accesses data the current prompting user is authorized to see.

Community Perspectives and Counterpoints

The discovery of GitLost sparked significant debate regarding where the responsibility for security lies—with the platform provider or the user configuring the agent.

User Configuration vs. Platform Vulnerability

Some critics argue that the leak is a result of poor user configuration rather than a GitHub vulnerability. As one commenter noted:

"The researchers are the ones that grant the agent access to private repos and then ask it to answer questions in public repos.. of course this allows extracting private information? This is like setting up a normal CI job with access to secrets and running it on public PRs."

The "AI Rush" Critique

Other observers suggest that the pressure to integrate AI into every product has led to a decrease in security standards. There is a sentiment that corporations are "slapping AI onto every single product offering" without adequate security testing or oversight, leading to "half-assed AI integrations."

The Nature of LLMs

There is a consensus among several technical commenters that the inherent nature of LLMs makes them unsuitable for enforcing security boundaries. One analogy compared the struggle to "repeatedly trying to train a dog with amnesia to not poop in the bedroom... Lock the bedroom instead."