Microsoft's innovative AI transforms ordinary text into high-quality podcasts, leaving users astonished
Microsoft Research has recently introduced an innovative open-source project called VibeVoice. This novel framework is designed for generating expressive, long-form, multi-speaker conversational audio.
VibeVoice is a useful accessibility tool, focusing on text-to-speech conversion. It offers two versions for testing: a 1.5 billion parameter version and a 7 billion parameter version. The larger version, with 7 billion parameters, has a smaller 32k context window compared to the 1.5 billion parameter version. A third, lighter version, at 0.5 billion parameters, is also in development for real-time audio generation.
The project addresses challenges in traditional Text-to-Speech (TTS) systems, particularly in scalability, speaker consistency, and natural turn-taking. VibeVoice can synthesize speech up to 90 minutes long with up to 4 distinct speakers, making it a promising tool for various applications.
One such application could be in chat assistants, where the streaming audio version of VibeVoice has potential for use without relying on external servers. It is worth noting that the project currently supports English and Mandarin, with plans for other languages in future refinements.
To test VibeVoice, you can find examples on its GitHub repository or Hugging Face. More advanced examples of its capabilities can be found on the project page. For a basic test, you can hear an embedded clip above showcasing VibeVoice's output.
VibeVoice requires around 7GB of VRAM for the smaller model and up to 18GB for the larger one when used locally. It's important to note that using the service may require waiting in a queue for audio processing, especially during peak times.
In conclusion, VibeVoice is a significant step forward in the field of speech synthesis, offering a unique solution to the challenges faced by traditional TTS systems. Its potential applications, from accessibility tools to chat assistants, make it an exciting project to watch in the coming months.
Read also:
- Artificial Intelligence Threatening Confidence in Workplace Relations
- Agents seen as valuable colleagues by employees, but the idea of an AI overseeing as a superior is deemed as crossing a line
- Expanded financial support leads to the first AI model for detecting kidney cancer, certified by CE, receiving €6M
- Humanity's salvation hinges on maternal instincts, Geoffrey Hinton cautions, as he foresees artificial superintelligence surpassing human intelligence.
