Simbastack-hq/framedex

Framedex — a queryable knowledge base for your video archive

What it solves

framedex turns scattered video and photo archives—often spread across multiple SSDs or stored in Apple Photos libraries—into a portable, plain-text knowledge base. It solves the problem of having massive amounts of unorganized media where finding specific moments, people, or locations is difficult without manual tagging.

How it works

The tool processes media files through a local, resumable pipeline that generates a .description.md sidecar file for every video or photo. These sidecars contain metadata and AI-generated insights without modifying the original files.

  • For Videos: It extracts metadata via ffprobe and exiftool, reverse-geocodes GPS coordinates, samples diverse frames, transcribes audio using WhisperX (with speaker diarization and translation), detects faces via insightface, and uses a vision model to describe the scene and assign a quality rating (keep/review/cull).
  • For Photos: It reads EXIF data (camera, lens, exposure), performs face detection, and uses a vision model for scene description and rating. It intelligently groups bursts and RAW+JPEG pairs to avoid redundant AI calls.
  • Indexing & Querying: It creates folder summaries and a master index (_INDEX.md), allowing users to query their archive via fdx-query or by using an MCP host (like Claude Desktop) to "talk" to their media.
  • Integration: It can export ratings and keywords to .xmp sidecars for use in Adobe Lightroom Classic.

Who it’s for

Photographers, videographers, and archivists who manage large volumes of media across multiple drives and want a searchable, AI-enhanced index of their content without altering their original files.

Highlights

  • Non-destructive: Originals are never modified; all data is stored in Markdown sidecars.
  • Local-first: Most processing (transcription, face detection) happens locally, with flexible vision backends (Claude CLI, API, or fully local via LM Studio).
  • Comprehensive Media Analysis: Combines transcription, translation, speaker diarization, and vision-based scene description.
  • Lightroom Integration: Projects AI ratings and keywords into .xmp files for professional editing workflows.
  • MCP Support: Includes an MCP server (fdx-mcp) to let AI agents interact with and query the archive.
  • Apple Photos Support: Can index .photoslibrary files directly without requiring exports.

Related

  • Project
    oxbshw/watch-skill

    A video indexing and verification engine that gives AI agents the ability to search, remember, and verify their own work in videos and browser sessions.

  • Project
    commaai/commavq

    A world model framework and dataset for autonomous driving that uses VQ-VAE for video compression and a GPT-based model to predict future driving scenes.

  • Project
    JohnKinyanjui/sprite-maker

    A local-first AI workbench for creating, animating, and exporting 2D game art, featuring identity-preserving AI animation and a deterministic Rust-based rigging engine.

  • Project
    JavisVerse/JavisDiT

    A Diffusion Transformer framework for joint audio-video generation that ensures semantic and temporal alignment between sound and visuals from text prompts.