SonyResearch/Woosh

Public release of the Sound Effect Foundation model by Sony AI.

What it solves

Woosh provides a set of generative models designed to create high-quality sound effects. It addresses the challenge of generating audio that is either based on a text description (text-to-audio) or synchronized with a video sequence (video-to-audio).

How it works

The project uses a suite of Latent Diffusion Models (LDMs) and supporting components:

  • Woosh-AE: An audio encoder/decoder that handles the conversion between raw audio and latent representations.
  • Woosh-CLAP: A multimodal alignment model that provides text-audio token latents for conditioning the diffusion models.
  • Woosh-Flow and Woosh-DFlow: Models that generate audio from text prompts or unconditionally.
  • Woosh-VFlow: A multimodal LDM that generates audio specifically tailored to a video sequence, with optional text prompts.

Who it’s for

This toolkit is intended for sound designers, video editors, and AI researchers interested in automated sound effect generation for media production.

Highlights

  • Text-to-Audio (T2A): Generate sound effects from written descriptions.
  • Video-to-Audio (V2A): Create audio that matches the visual content of a video.
  • Multimodal Conditioning: Ability to combine video and text prompts for audio generation.
  • Open Weights: Pretrained weights are available for public datasets.

Related

  • Project
    jzjzzzzzzz/agent-me

    Agent‑Me is an open‑source FastAPI + React system that turns a personal Markdown knowledge base into an inspectable AI “twin”. It runs a deterministic multi‑stage pipeline (retrieval → planner → researcher → critic → writer → optional verifier) and returns the full execution trace, making every answer evidence‑grounded and auditable. The repo provides Docker‑compose and local‑toolchain setups, optional OpenAI‑compatible LLM integration, a test suite with deterministic fixtures, and a bilingual curriculum for developers who want to extend the architecture.

  • Project
    OHF-Voice/wyoming-faster-whisper

    A Wyoming protocol server for faster-whisper that provides local speech-to-text with an optional integration to bias transcription toward Home Assistant entity names.

  • Project
    LibreSpark/LibreTTS

    A free online text-to-speech tool that supports over 300 voices and allows integration with custom OpenAI-compatible TTS APIs.

  • Project
    Socialpranker/deepdive

    Deepdive is an open‑source Claude Code skill that turns a plain research request into a reproducible 13‑phase pipeline. It reframes the question, builds a detailed plan, runs parallel sub‑agents across 29 curated search channels, triangulates and filters evidence, performs adversarial red‑team review, and verifies citations for liveness and faithfulness. The output is a structured folder of markdown files (plan, sources, claims, memo, full report) with every claim linked to its source, making the research auditable and reusable. Installation is a one‑line git clone for Claude Code, and the methodology can be adapted to other LLMs.