jmerelnyc/Photo-agents

Autonomous self-evolving agents. Vision-grounded layered memory and self-written skills for LLM agents that operate your computer.

What it solves

Photo Agents 提供了一個自主代理框架，讓代理能像人類一樣感知並與電腦螢幕互動。它解決了 LLM 代理僅依賴文字紀錄的問題，透過將行動根植於螢幕的實際視覺觀察，使其能自主操作軟體與導覽介面。

How it works

系統使用「感知 → 推理 → 行動」的循環，由工具呼叫 LLM 驅動。它處理視覺輸入，並將觀察結果存入分層記憶系統（工作、全域、SOP 與會話檔案）。代理可以根據成功的結果自行撰寫「技能」，隨時間演化。它還包含實體執行工具集，用於檔案 I/O、沙盒程式碼執行（Python、PowerShell、bash）以及透過 Chrome DevTools Protocol 橋接的瀏覽器自動化。

Who it’s for

此框架設計給想在本機部署自主代理、管理桌面環境、自動化瀏覽器任務，並在不同作業系統（Windows、macOS、Linux）上執行程式碼的使用者。

Highlights

Multi-Provider LLM Support: Native integration with Anthropic Claude and OpenAI GPT, including a failover session router.
Self-Evolving Capabilities: Agents can create their own skills and SOPs from real‑world success.
Versatile Interfaces: Ships with multiple clients including a Streamlit web app, PyQt desktop app, and bots for Telegram, QQ, Feishu, WeCom, and DingTalk.
Local Data Ownership: The runtime runs locally to ensure the user maintains ownership of their screen data and keys.