Pull request adds streaming Drama-3 TTS to the speech-to-speech pipeline

What happened and what to do

The pull request adds Drama-3, a speech synthesis model from Fish Audio, as a streaming text-to-speech option in the speech-to-speech repository, enabled by the flag --tts drama3. According to the description, the backend is self-hostable, offers optional voice cloning and supports expressive intonations, such as whispers and laughter, within speech-to-speech pipelines.

For a company, the gain is running a real-time voice agent with synthesis generated on its own infrastructure, without sending customer audio or text to external services. In practice, this requires assessing time to first audio, GPU consumption, the terms for using cloned voices and the consent of whoever provides the voice, as well as testing the fit with speech recognition and the language model. A typical implementation combines a private inference service, streaming queues, latency metrics and a control over the use of cloned voices.

How the consultancy can help

Feasibility diagnosis for streaming TTS on your own infrastructure: latency benchmark and cost per minute of audio, assessment of voice consent and licensing, and design of the speech-to-speech pipeline. Then a scoped implementation with monitoring and ongoing operation.

Next step

If your company wants to evaluate streaming speech synthesis on its own infrastructure, send a short description of your case, with the expected usage volume and the channels involved. We will reply with a scoped proposal for diagnosis and implementation.