Fish Audio

Fish Audio uses a VQGAN-based architecture trained on 700k+ hours of multilingual audio to capture prosodic features, phonetic patterns, and speaker characteristics from just 10-15 seconds of reference audio. The S1 model analyzes vocal timbre, pitch contours, and speaking rhythm to create a digital voice model that maintains the original speaker's accent, tone, and natural speech habits across multiple languages.

The S1 model features 4 billion parameters versus earlier versions, achieving 0.008 WER (Word Error Rate) and 0.004 CER (Character Error Rate) on benchmarks. It includes 48+ RLHF-integrated emotional expressions, improved multilingual pronunciation, and real-time emotion control not available in v1.5/v1.6. S1 also delivers 44.1kHz audio quality with significantly reduced latency for streaming applications.

Yes, Fish Audio provides unified streaming APIs with server-side Voice Activity Detection that automatically stops generation on silence detection. The architecture supports push-to-send controls, WebSocket streaming, and sub-200ms first-byte latency, making it suitable for real-time avatars, voice agents, and interactive conversational applications requiring instant response times.

Fish Audio offers 48+ pre-defined emotional expressions trained via RLHF (Reinforcement Learning from Human Feedback), allowing users to specify emotions like joy, anger, sadness, or excitement through API parameters or UI controls. The system adjusts prosody, pitch variation, and speaking pace dynamically without requiring separate voice samples for each emotion, maintaining natural transitions between emotional states.

The platform supports multiple output formats including MP3, WAV, and OPUS with customizable sample rates. The S1 model generates native 44.1kHz audio (48kHz for OPUS), while v1.5/v1.6 models support standard rates from 16kHz to 44.1kHz. Users can specify format preferences via API parameters or download options in the web interface for integration with editing software.