beiyege-01/dsh-voice-ai-girlfriend

What it solves

Typing to an AI is slow and impersonal. This project adds voice input and output to DeepSeek Harness, so you can talk to the AI and hear it speak back. It also gives the AI a visual presence with a digital human avatar that lip-syncs to the speech. For users who are away from the computer, it can push messages and voice notes to QQ.

How it works

The plugin sits on top of DeepSeek Harness (DSH). A separate bridge service (Python/FastAPI) handles the heavy lifting: it captures your microphone audio, uses FunASR for Chinese speech recognition to turn it into text, sends that text to the DSH conversation (via the plugin's integration), then takes the AI's reply, converts it to speech with OmniVoice (or Qwen3-TTS as a fallback), and plays it through your speakers. If the digital human mode is enabled, the speech is also sent to a DUIX engine that renders a lip-synced video of an avatar, which is played in a side window. The bridge also manages interruption logic—when you talk, the AI stops speaking and listens. A separate QQ bridge lets you chat with the AI from your phone; the AI responds with text and voice messages.

Who it's for

This is for people who already use DeepSeek Harness and want a more natural, voice-driven interaction. It requires a Windows PC with an NVIDIA GPU (16GB VRAM recommended), and you'll need to set up WSL2 for the TTS engine, Docker for the digital human, and optionally NapCatQQ for QQ integration. It's a hobbyist-friendly setup but requires some technical comfort.

Highlights

• Fast responses: speech-to-text in ~150ms, and voice output starts in about 0.5 seconds after you stop speaking.
• Noise filtering uses a -35dB noise gate plus silero VAD so background sounds don't trigger the AI.
• Voice cloning: you can give a sample of your voice (or any voice) and the AI will sound like that.
• Streaming digital human: replies are split into ≤10s segments, generated and played back seamlessly, with lip sync.
• Interruption handling: you can talk over the AI, and it will stop and listen.
• Persistent avatar window with idle animations when not speaking.
• QQ integration: send and receive messages, including voice and images.
• Easy one-click start script that brings up the whole stack.