yashab-cyber/opendroid

What it solves

OpenDroid is an autonomous AI agent for Android that allows users to perform complex, multi-step tasks across different apps without manual intervention. It moves beyond simple chatbots by planning, executing, and verifying actions directly on the device, such as coordinating weather checks, messaging, and alarm settings in a single command.

How it works

The system uses an autonomous agent engine to break down complex user intents into sequential steps. It leverages the Android Accessibility API for UI automation and screen reading, and a vision engine to analyze screenshots for real-time screen understanding. To handle intelligence, it supports 12 different LLM providers (including cloud-based ones like Gemini and Claude, and local ones like Ollama) with an automatic failover chain. It also maintains a 4-tier personal knowledge graph to store temporary plans, explicit facts, learned patterns, and encrypted sensitive data.

Who it’s for

Android users who want a hands-free, autonomous assistant capable of controlling their device and automating recurring daily routines.

Highlights

• Autonomous Planning: Breaks complex commands into steps with dependency tracking and dynamic replanning if a step fails.
• Full Device Control: Executes 60+ actions across system settings, communication apps (WhatsApp, Telegram), productivity tools, and smart home devices.
• On-Device Model Management: Includes a manager for downloading, verifying, and importing local models via LiteRT-LM.
• Multimodal Vision: Uses vision-capable LLMs to extract information from the screen via "Read & Remember."
• Voice Interface: Features offline wake word detection and integration with ElevenLabs for premium text-to-speech.
• Glassmorphic UI: A modern Jetpack Compose interface with live latency benchmarks for LLM providers.