Roy3838/Observer
Local Micro-Agents that monitor your screen, so you don't have to.
What it solves
Observer AI acts as a personal automation agent that monitors your computer screen, camera, and audio to trigger specific actions based on what it observes. It solves the problem of manual monitoring by allowing users to set up "if-this-then-that" style workflows—such as sending a notification when a file download finishes, a person appears on camera, or a specific logo is detected on screen.
How it works
Users create "micro-agents" by combining three components: Sensors (screen, camera, microphone, clipboard), Models (local LLMs for reasoning), and Tools (messaging, email, system commands). The agent runs in a loop, processing sensor data through a local model to make decisions or extract information. If a condition is met, the agent executes JavaScript-based tools to perform actions like sending a WhatsApp message, playing a sound, or logging data to memory.
Who it’s for
This project is for power users and developers who want to build private, local-first automation agents that interact with their desktop environment without sending sensitive screen or camera data to the cloud.
Highlights
- Privacy-Focused: Runs entirely locally using models via llama.cpp, Ollama, or Transformers.js.
- Multimodal Sensors: Supports screen capture, OCR, camera input, and audio transcription.
- Extensive Tooling: Includes built-in functions for messaging (WhatsApp, Discord, Telegram, SMS), system notifications, and file/memory management.
- Flexible Logic: Supports both text-based LLM reasoning and specialized "Decision Models" for high-accuracy probability-based tasks (e.g., detecting if a person is present).
Related
- Project
- Project
- Project
- Project
- Project