jmerelnyc/Photo-agents

Autonomous self-evolving agents. Vision-grounded layered memory and self-written skills for LLM agents that operate your computer.

What it solves

Photo Agents 提供了一个自主代理框架，使代理能够像人类一样感知并与电脑屏幕交互。它解决了 LLM 代理仅依赖文本记录的问题，通过将其行为基于屏幕的实际视觉观察，使其能够自主操作软件和导航界面。

How it works

系统使用“感知 → 推理 → 行动”循环，由工具调用 LLM 驱动。它处理视觉输入，并将观察存储在分层记忆系统（工作、全局、SOP 和会话档案）中。代理可以根据成功的结果自行编写“技能”，随时间演化。它还包括用于文件 I/O、沙盒代码执行（Python、PowerShell、bash）以及通过 Chrome DevTools Protocol 桥接的浏览器自动化的物理执行工具集。

Who it’s for

该框架面向希望在本地部署自主代理、管理桌面环境、自动化浏览器任务，并在不同操作系统（Windows、macOS、Linux）上执行代码的用户。

Highlights

  • Multi-Provider LLM Support: Native integration with Anthropic Claude and OpenAI GPT, including a failover session router.
  • Self-Evolving Capabilities: Agents can create their own skills and SOPs from real‑world success.
  • Versatile Interfaces: Ships with multiple clients including a Streamlit web app, PyQt desktop app, and bots for Telegram, QQ, Feishu, WeCom, and DingTalk.
  • Local Data Ownership: The runtime runs locally to ensure the user maintains ownership of their screen data and keys.