nextcloud/recognize

👁 👂 Smart media tagging for Nextcloud: recognizes faces, objects, landscapes, music genres

What it solves

Recognize is a smart media tagging system for Nextcloud that automatically categorizes photos, videos, and music files. It eliminates the manual effort of tagging media collections by using AI to identify content and apply appropriate tags via Nextcloud's Collaborative Tags system.

How it works

The app runs on-premises on the Nextcloud machine using Tensorflow.js in Node.js. It employs several pre-trained models to handle different media types:

  • Images: Uses EfficientNet v2 for object detection (animals, landscapes, food, etc.) and a Landmarks v1 model for monuments and landmarks.
  • Faces: Uses face-api.js to extract and compare facial features for grouping photos.
  • Videos: Uses a MoViNet model to classify human actions within videos.
  • Audio: Uses a Musicnn neural network architecture to classify music genres.

Who it’s for

Nextcloud users who want to organize their self-hosted media libraries without sending their data to external cloud providers, prioritizing privacy and local processing.

Highlights

  • Privacy-focused: All processing is done locally on the server; no sensitive data is sent to cloud providers.
  • Multimodal recognition: Supports photos (objects, faces, landmarks), videos (actions), and music (genres).
  • ** Docket-friendly:** Supports native speed via AVX instructions or a WASM mode for broader hardware compatibility.
  • Ethical AI: Uses open-source training software and freely available models for most of its features.

Related

  • Project
    matiasdelellis/facerecognition

    A facial recognition app for Nextcloud that detects, analyzes, and groups faces in images locally to provide private, person-based photo organization.

  • Project
    photoprism/photoprism

    PhotoPrism is an open‑source, self‑hosted web app that uses on‑device AI to automatically tag, search, and organize personal photos and videos while keeping all data private.

  • Project
    LibrePhotos/librephotos

    A self-hosted, open-source photo management service that uses machine learning for automatic face recognition, object detection, and semantic search.

  • Project
    SharpAI/DeepCamera

    An open-source AI camera platform that enables local deployment of VLM scene analysis and object detection skills with autonomous, hardware-aware installation.

  • Project
    volotat/Anagnorisis

    A local, private recommendation system that allows users to fine-tune models on their own data to predict and rank personal preferences across text, audio, images, and video.