ssrajadh/sentrysearch

What it solves

SentrySearch allows users to perform semantic searches across large amounts of video footage. Instead of manually scrubbing through hours of recordings, users can type a natural language description (e.g., "red truck running a stop sign") or provide a reference image to find and automatically trim the relevant clips from their videos.

How it works

The tool processes video files by splitting them into overlapping chunks. These chunks are converted into vector embeddings using a multimodal model (Google Gemini, Alibaba DashScope, or a local Qwen3-VL model) and stored in a local ChromaDB database. When a user searches, the query is embedded into the same vector space and matched against the stored video embeddings using cosine similarity. The system can also use a Vision Language Model (VLM) to re-rank the top results for higher accuracy.

Who it’s for

It is designed for anyone managing large volumes of video footage, such as dashcam owners (specifically Tesla users) or security footage reviewers, who need to find specific events quickly without manual review.

Highlights

• Multimodal Search: Supports both text-based natural language queries and image-based searches.
• Flexible Backends: Offers cloud-based APIs (Gemini, DashScope) or a fully local, private backend using Qwen3-VL.
• Anomaly Detection: A "highlights" feature that identifies statistically anomalous clips that differ from the rest of the index.
• Tesla Integration: Can burn telemetry data (speed, location, time) from Tesla dashcam files directly onto the trimmed clips.
• Automatic Trimming: Automatically extracts the matching segment from the original video file as a new clip.
• Ecosystem Integration: Works with sibling tools SentryMerge (for multi-cam stitching) and SentryBlur (for redaction).