goruck/home-generative-agent

What it solves

Home Generative Agent (HGA) transforms a smart home into an intelligent, conversational environment. It moves beyond simple prompt-forwarding by providing a generative AI agent that can actively control devices, create complex automations from natural language, analyze camera footage in real-time, and monitor home security through a deterministic anomaly engine.

How it works

Built on LangGraph, HGA integrates directly into Home Assistant. It uses a set of tools to interact with entities, query historical data, and write YAML for new automations. To maintain context, it employs a long-term semantic memory system powered by pgvector (PostgreSQL), allowing it to remember user preferences across different conversations. For safety and security, it features the Sentinel anomaly engine, which uses deterministic rules to flag issues (like unlocked doors or unknown people) while using the LLM for triage and advice rather than direct action.

Who it’s for

Home Assistant users who want a more intuitive, agentic way to manage their smart home, including those who want to integrate vision-language models (VLMs) for camera analysis or those who prefer fully local AI processing to keep their data private.

Highlights

• Natural Language Automations: Describe a desired behavior in chat, and the agent writes and registers the Home Assistant automation automatically.
• Camera Intelligence: Real-time image analysis and face recognition to identify people and provide proactive alerts.
• Sentinel Anomaly Engine: A deterministic security audit and monitoring system that flags safety risks and network vulnerabilities.
• Long-term Memory: Semantic search over past interactions to recall user preferences.
• Flexible Model Support: Compatible with cloud providers (OpenAI, Gemini, Anthropic) or local setups (Ollama, vLLM, llama.cpp).
• Integrated Voice Pipeline: Built-in support for speech-to-text (STT) and text-to-speech (TTS) via OpenAI or local servers like Speaches.