whitphx/streamlit-webrtc

Real-time video and audio processing on Streamlit

What it solves

Streamlit apps typically struggle with real-time audio and video streaming because of the framework's execution model. streamlit-webrtc provides a way to handle and transmit real-time media streams over a network, allowing developers to build interactive media applications without needing to write complex WebRTC signaling and transport logic.

How it works

It integrates WebRTC into Streamlit by providing a webrtc_streamer component. This component allows developers to define callback functions (for video and audio frames) that process media in real-time. These callbacks are executed in separate threads to ensure the stream remains fluid, using the PyAV library to handle the frames as av.VideoFrame or av.AudioFrame objects.

Who it’s for

Python developers using Streamlit who want to build apps featuring real-time video or audio processing, such as object detection, speech-to-text, or video chat applications.

Highlights

  • Real-time Processing: Supports both video and audio frame callbacks for on-the-fly manipulation.
  • Flexible Integration: Works with OpenCV and other image processing libraries by converting frames to NumPy arrays.
  • Session Management: Includes hooks like on_video_ended and on_audio_ended for cleaning up resources when a session stops.
  • Deployment Ready: Provides guidance and configuration options for STUN/TURN servers to ensure connectivity across different network environments and HTTPS requirements.